Target contour extraction method, device and system based on SCE-Net and storage medium

Abstract

The application discloses a target contour extraction method, device and system based on an SCE-Net and a storage medium, and relates to the technical field of image processing. The method comprises the following steps: acquiring original point cloud data, constructing an SCE-Net network comprising a point cloud target segmentation module and a point cloud target contour extraction module connected in series, inputting the original point cloud data into the point cloud target segmentation module to obtain point cloud individual data of different categories, and processing the point cloud individual data of different categories through the point cloud target contour extraction module to obtain a point cloud target contour extraction result of the entire scene. The method of the embodiment of the application guarantees the accuracy of segmentation and improves the fineness of contour extraction. The entire network can directly output the segmentation and contour extraction result from the original point cloud data, reduces manual intervention, and improves the degree of automation. Accurate target segmentation is realized through a segmentation classifier, and the point cloud target contour of the entire scene is accurately extracted through a contour classifier.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of image processing, and particularly relates to a target contour extraction method, device and system based on an SCE-Net and a storage medium. BACKGROUND

[0002] With the continuous development of three-dimensional sensors, the collection of point cloud data is becoming easier and easier, and in recent years, with the development of deep learning technology, especially the great advantages of convolutional neural network technology in the field of image processing, using deep learning technology to process point cloud data provides a new solution for point cloud data processing tasks.

[0003] However, the unordered, non-structured, inconsistent density and incomplete information characteristics of point cloud itself also hinder the use of deep learning technology to process point cloud data. First, most existing technologies often take small-scale point cloud as the processing target, and many technologies for processing large-scale point cloud data also need to cut the point cloud, which may cause the loss of overall information of the point cloud; in addition, the mutual occlusion of the scanned objects is easy to cause information loss, which also has a great influence on point cloud data processing; in addition, the noise in the point cloud data will cause the precision of shape recognition and other tasks to decrease, in addition, for the same model, rotating at different angles still represents the same object, but the network recognition result may differ due to the rotation of the point cloud data.

[0004] In addition, with the continuous progress of point cloud data collection technology and the continuous deepening of research on methods for processing point cloud data based on deep learning technology, point cloud data processing related tasks are also constantly proposed, mainly including three-dimensional shape classification, three-dimensional target detection and tracking, three-dimensional point cloud segmentation, three-dimensional point cloud registration, three-dimensional reconstruction and 6 degree of freedom pose estimation. To solve these tasks, some deep learning technologies will voxelize or convert point cloud data into two-dimensional views and then analyze them, however, such technologies have certain limitations, the performance of deep learning technologies based on voxel for point cloud data analysis is often limited by the resolution of the voxel, the method of analyzing point cloud data based on two-dimensional images is superior in performance when completing shape classification and retrieval tasks, but still has certain difficulties when facing complex tasks, and the deep learning technology based on features is limited by the representation ability of the manually extracted features for point cloud data.

[0005] Most of the traditional methods for target contour point recognition of point cloud data are based on prior knowledge or based on local features of point cloud, such as covariance matrix, curvature, normal vector, etc. The results obtained by these methods are often actual geometric oversimplified models. Therefore, the traditional three-dimensional reconstruction technology for point cloud often has good effect for a certain category of data, but the universality is not strong. At present, the related research on point cloud target contour extraction still focuses on a certain entity such as industrial parts and other objects, and does not take the point cloud data of a scene containing multiple objects as the research object of point cloud target contour point extraction technology. Moreover, there are few related technologies that fully combine point cloud target contour extraction technology and point cloud target segmentation technology, and have certain disadvantages in solving the problem of point cloud target contour extraction of scene point cloud. SUMMARY

[0006] The purpose of the present application is to provide a target contour extraction method, device and system based on SCE-Net, which can realize the function of extracting point cloud target contour from scene point cloud data.

[0007] In order to achieve the above purpose, the solution of the present application is:

[0008] In a first aspect, the embodiments of the present application provide a target contour extraction method based on SCE-Net, comprising:

[0009] obtaining original point cloud data; constructing an SCE-Net network, the SCE-Net network comprising a point cloud target segmentation module and a point cloud target contour extraction module, the point cloud target segmentation module and the point cloud target contour extraction module being connected in series;

[0010] The point cloud target segmentation module comprises a first multi-layer perception embedding layer, a first encoder, a first decoder and a segmentation classifier connected in sequence; the first encoder comprises N first sub-encoders, and the first decoder comprises N first sub-decoders.

[0011] The original point cloud data is input into the first multi-layer perception embedding layer for processing to obtain point cloud data features; the point cloud data features are input into the first encoder for processing to obtain different dimension point cloud data features of different scale point cloud data;

[0012] The different dimension point cloud data features of different scale point cloud data are input into the first decoder for processing, each layer of the first sub-decoder in the first decoder taking the point cloud data features output by the previous layer of the first sub-decoder and the point cloud data features input by the corresponding level of the first sub-encoder as input to obtain point cloud data features of all points; the point cloud data features of all points are input into the segmentation classifier for processing to obtain point cloud individual data of different categories;

[0013] The point cloud target contour extraction module comprises a second multi-layer perception embedding layer, a second encoder, a second decoder and a contour classifier connected in sequence; the second encoder comprises N second sub-encoders, and the second decoder comprises N second sub-decoders;

[0014] The point cloud individual data of different categories is embedded into a feature space through the second multi-layer perception embedding layer, and the point cloud individual data of different categories is processed through the second encoder and the second decoder to obtain the composite point cloud data features of each point in the point cloud individual data of different categories.

[0015] The contour classifier performs binary classification on all points in the point cloud individual data of different categories according to the composite point cloud data features, and obtains the point cloud target contour extraction result of the entire scene.

[0016] According to the above method of the embodiment of the application, the following additional technical features can be further provided:

[0017] Further, the original point cloud data is input into the first multi-layer perception embedding layer for processing to obtain point cloud data features, including:

[0018] The first multi-layer perception embedding layer establishes a nonlinear mapping, extracts the feature values of the original point cloud data through the first multi-layer perception embedding layer, and maps the feature values of the original point cloud data to a feature space to output the point cloud data features, and the formula is as follows:

[0019] Γ: R N×M → R N×C

[0020] Wherein N represents the number of points in the original point cloud data, M represents the dimension of the original point cloud data features, and C represents the feature dimension required for the first encoder to extract the point cloud data features.

[0021] Further, the point cloud data features are input into the first encoder for processing to obtain different dimension point cloud data features of different scale point cloud data, including:

[0022] The point cloud data features are processed through downsampling in the first encoder, the first encoder comprises N first sub-encoders, the dimension of the point cloud data features input into each first sub-encoder is C'', and the dimension of the point cloud data features output from each first sub-encoder is 2xC''.

[0023] The scale of the point cloud data input into each first sub-encoder is N', and the scale of the point cloud data output from each first sub-encoder is N' / S, wherein S represents the point cloud data scale reduction ratio in the downsampling process, and after the downsampling is completed, the feature aggregation of the point cloud data features is performed through the graph convolution technology, and the different dimension point cloud data features of different scale point cloud data are extracted.

[0024] Further, the different dimension point cloud data features of the different scale point cloud data are input into the first decoder for processing, and each layer first sub-decoder in the first decoder takes the point cloud data features output by the previous layer first sub-decoder and the point cloud data features input by the corresponding layer first sub-encoder as input to obtain the point cloud data features of all points, including:

[0025] The different dimension point cloud data features of the different scale point cloud data are processed by upsampling in the first decoder, and the first decoder includes N layers of first sub-decoders, the dimension of the point cloud data features input by each layer first sub-decoder is C''', and the dimension of the point cloud data features output by each layer first sub-decoder is C''' / 2;

[0026] The scale of the point cloud data input by each layer first sub-decoder is N'', and the scale of the point cloud data output by each layer first sub-decoder is N''xS, wherein S is the scale amplification ratio of the point cloud data in the upsampling process, and after upsampling, the different dimension point cloud data features of the different scale point cloud data are aggregated by the graph convolution technology and the point cloud data features of all points are extracted.

[0027] Further, the point cloud data features of all points are input into the segmentation classifier for processing to obtain point cloud individual data of different categories, including:

[0028] In the segmentation classifier, the semantic labels of each point in the point cloud data features of all points are predicted, the points with the same semantic labels are divided into the same category, and the points with different semantic labels are segmented to obtain point cloud individual data of different categories.

[0029] Further, in the second sub-decoder of the point cloud target contour extraction module, except for the last layer second sub-decoder, each layer second sub-decoder takes the features output by the previous layer second sub-decoder and the input point cloud data features of the corresponding layer second sub-encoder as input, and the last layer decoder takes the point cloud data features of all points obtained by the point cloud target segmentation module, the point cloud data features input by the corresponding layer second sub-encoder, and the features output by the previous layer second sub-decoder as input to obtain the composite point cloud data features of each point in the point cloud individual data of different categories.

[0030] Further, a composite loss function is introduced in the point cloud target contour extraction module, and the composite loss function includes a chamfer distance loss function and a weighted binary cross-entropy loss function, wherein the formula of the chamfer distance loss function is as follows:

[0031]

[0032] Wherein, L CD represents the chamfer distance loss function, n gThe number of contour points representing the actual point cloud target, S1 represents a contour point set of the actual point cloud target, p i The coordinates of the i-th contour point in S1, S2 represents a contour point set of the predicted point cloud target, p j The coordinates of the i-th contour point in S2, n p The number of contour points representing the predicted point cloud target;

[0033] The weighted binary classification cross-entropy loss function is as follows:

[0034]

[0035] Wherein, L CE The weighted binary classification cross-entropy loss function, Q i The one-hot encoding of the contour point identification label of each point in the point cloud data, W i The loss calculation weight matrix of the i-th point in the point cloud data, W i ={1, n g1 / n p1}, n g1 The number of non-contour points in the actual point cloud data, n p1 The number of contour points in the actual point cloud data, Y i The calculation result set of the i-th point in the point cloud data,

[0036] The composite loss function is as follows:

[0037] L = aL CD + bL CE

[0038] Wherein, L represents the composite loss function, a and b represent the weights of L CD and L CE , a+b=1.

[0039] In a second aspect, the embodiments of the present application provide a target contour extraction device based on SCE-Net, comprising:

[0040] The data acquisition module is configured to acquire original point cloud data;

[0041] The network construction module is configured to construct the SCE-Net network, and the SCE-Net network comprises a point cloud target segmentation module and a point cloud target contour extraction module, and the point cloud target segmentation module and the point cloud target contour extraction module are connected in series, wherein the point cloud target segmentation module comprises a first multi-layer perception embedding layer, a first encoder, a first decoder and a segmentation classifier connected in sequence; the first encoder comprises N first sub-encoders, and the first decoder comprises N first sub-decoders; the point cloud target contour extraction module comprises a second multi-layer perception embedding layer, a second encoder, a second decoder and a contour classifier connected in sequence; the second encoder comprises N second sub-encoders, and the second decoder comprises N second sub-decoders.

[0042] The data processing module is configured to input the original point cloud data into the first multi-layer perception embedding layer for processing to obtain point cloud data features; input the point cloud data features into the first encoder for processing to obtain different-dimensional point cloud data features of different scale point cloud data; input the different-dimensional point cloud data features of the different scale point cloud data into the first decoder for processing, wherein each layer of the first sub-decoder in the first decoder takes the point cloud data features output by the previous layer of the first sub-decoder and the point cloud data features input by the corresponding level of the first sub-encoder as input to obtain point cloud data features of all points; input the point cloud data features of all points into the segmentation classifier for processing to obtain point cloud individual data of different categories; embed the point cloud individual data of different categories into a feature space through the second multi-layer perception embedding layer, process the point cloud individual data of different categories through the second encoder and the second decoder to obtain composite point cloud data features of each point in the point cloud individual data of different categories; and perform binary classification on all points in the point cloud individual data of different categories through the contour classifier by using the composite point cloud data features to obtain point cloud target contour extraction results of the entire scene.

[0043] In a third aspect, an SCE-Net-based target contour extraction system is provided, which comprises a processor and a memory, and the memory stores a computer program which is loaded and executed by the processor to implement the SCE-Net-based target contour extraction method provided in the first aspect.

[0044] In a fourth aspect, a computer readable storage medium is provided, which stores a computer program, and the computer program is executed by a processor to implement the SCE-Net-based target contour extraction method provided in the first aspect.

[0045] Compared with the prior art, the SCE-Net-based target contour extraction method provided in the embodiments of the present application has the following beneficial technical effects:

[0046] The embodiment of the application ensures that the starting point of analysis is real-world three-dimensional data, provides rich spatial information for subsequent processing, is suitable for point cloud data of various sources such as laser scanning, radar scanning or three-dimensional reconstruction, and enhances the universality and applicability of the method.

[0047] The embodiment of the application realizes a coarse-to-fine processing flow by connecting the point cloud target segmentation module and the point cloud target contour extraction module in series through the construction of the SCE-Net network, guarantees the accuracy of segmentation, improves the fineness of contour extraction, and directly outputs the segmentation and contour extraction results from the original point cloud data, reduces manual intervention, and improves the automation degree.

[0048] The point cloud target segmentation module of the embodiment of the application preliminarily extracts the global and local features of the point cloud data through the first multi-layer perception embedding layer, lays a foundation for subsequent processing, captures different scale features of the point cloud data through different levels of encoders, enhances the processing capability of the network for different scale targets, fuses the multi-scale features of the first encoder through the first decoder, helps to restore spatial information and improve segmentation accuracy, and determines the category of each point in the point cloud based on the extracted features through the segmentation classifier, to realize accurate target segmentation.

[0049] The point cloud target contour extraction module of the embodiment of the application further embeds the segmented point cloud data into a feature space more suitable for contour extraction through the second multi-layer perception embedding layer, optimizes the data representation for subsequent processing; performs a more fine coding-decoding process through the second encoder and the second decoder, extracts and fuses features of different levels to generate composite point cloud data features of each point, and further enhances the contour judgment; performs binary classification through the contour classifier, uses the composite point cloud data features for binary classification, judges whether each point belongs to the target contour, and thus accurately extracts the point cloud target contour of the entire scene. BRIEF DESCRIPTION OF DRAWINGS

[0050] Figure 1 A flowchart of a target contour extraction method based on SCE-Net according to an embodiment of the application is shown;

[0051] Figure 2 A structural block diagram of a target contour extraction device based on SCE-Net according to an embodiment of the application is shown;

[0052] Figure 3 A structural block diagram of a computer device according to an embodiment of the application is shown;

[0053] Figure 4 An SCE-Net network structure diagram of a target contour extraction method based on SCE-Net according to an embodiment of the application is shown;

[0054] Figure 5 A structural diagram of a GCN module of the SCE-Net-based target contour extraction method of the embodiment of the present application is shown.

[0055] Figure 6 A structural diagram of a GAF module of the SCE-Net-based target contour extraction method of the embodiment of the present application is shown. DETAILED DESCRIPTION

[0056] In order to make the above objectives, characteristics and advantages of the present application more obvious and easy to understand, the specific embodiments of the present application are described in detail below with reference to the drawings. It can be understood that the specific embodiments described herein are only used to explain the present application, but not to limit the present application. In addition, it should be noted that, for the convenience of description, only the parts related to the present application are shown in the drawings, but not all the structures. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0057] The terms "comprising" and "having" and any variations thereof in the present application are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device including a series of steps or units is not limited to the listed steps or units, but can optionally include steps or units not listed, or can optionally include other steps or units inherent to these processes, methods, products or devices.

[0058] In this document, the term "embodiment" means that the specific features, structures or characteristics described in connection with the embodiment can be included in at least one embodiment of the present application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it mutually exclusive or alternative to other embodiments. Those skilled in the art explicitly and implicitly understand that the embodiments described herein can be combined with other embodiments.

[0059] As Figure 1 The embodiment of the present application provides a SCE-Net-based target contour extraction method, which includes the following steps:

[0060] Step 101, obtaining original point cloud data.

[0061] Obtaining original point cloud data is a basic step in the field of image processing technology. The following is an extended description of the method of obtaining original point cloud data. Common methods of obtaining original point cloud data include laser radar, structured light scanning, time-of-flight sensor, stereo vision and ultrasonic sensor, wherein:

[0062] LiDAR (Laser Imaging Detection and Ranging) is a technology that measures the distance between an object and a sensor by emitting laser pulses from a vehicle-mounted LiDAR, a drone-mounted LiDAR, or a ground laser scanner and receiving reflected signals. The time required for the laser pulse to return from emission to return is calculated and converted into distance, thereby generating three-dimensional point cloud data.

[0063] Structured light scanning is a technology that projects light with a known pattern onto the surface of an object by a desktop 3D scanner or an industrial-grade structured light scanner, and captures the changes in the reflected light pattern by a camera. The three-dimensional shape of the object is calculated according to the deformation of the pattern.

[0064] Time-of-flight (ToF) sensors are a technology that calculates distance by emitting light pulses from a smartphone, AR / VR device, or robot navigation sensor and measuring the time required for the light to return from emission to return, generating point cloud data.

[0065] Stereo vision is a technology that uses two or more cameras to capture images of the same scene from different angles by a stereo camera or a drone photography system, and calculates the depth information of the object by comparing the parallax in the images, generating three-dimensional point clouds.

[0066] Ultrasonic sensors are a technology that measures distance by emitting high-frequency sound waves from an ultrasonic range finder and receiving sound waves reflected from an object, calculating the time of flight of the sound waves, generating point cloud data.

[0067] In step 102, an SCE-Net network is constructed, the SCE-Net network comprising a point cloud target segmentation module and a point cloud target contour extraction module, the point cloud target segmentation module and the point cloud target contour extraction module being connected in series; wherein the point cloud target segmentation module comprises a first multi-layer perception embedding layer, a first encoder, a first decoder, and a segmentation classifier connected in sequence; the first encoder comprises N layers of first sub-encoders, and the first decoder comprises N layers of first sub-decoders; the point cloud target contour extraction module comprises a second multi-layer perception embedding layer, a second encoder, a second decoder, and a contour classifier connected in sequence; the second encoder comprises N layers of second sub-encoders, and the second decoder comprises N layers of second sub-decoders.

[0068] The structure of the SCE-Net (Segmentation-Contour Extraction Network) network in the embodiments of the present application is as follows: Figure 4As shown, the network is composed of two modules: a point cloud target segmentation module and a point cloud target contour extraction module, both of which adopt an encoder-decoder structure. To develop an end-to-end network that directly completes the point cloud target segmentation and point cloud target contour extraction tasks, the embodiments of the present application directly input the output of the point cloud target segmentation module into the point cloud target contour extraction module through the SCE-Net network, thereby connecting the two modules in series.

[0069] In step 103, the original point cloud data is input into a first multi-layer perception embedding layer for processing to obtain point cloud data features; and the point cloud data features are input into a first encoder for processing to obtain different-dimensional point cloud data features of different scale point cloud data.

[0070] In the point cloud target segmentation module, it is found through research that the AF-GCN network adopts a network architecture combining point cloud Transformer and U-Net and the module proposed by the AF-GCN network for extracting point cloud data features has sufficient experimental evidence to prove that it has good performance in point cloud semantic segmentation and point cloud data feature extraction, so the point cloud target segmentation module mainly refers to the structure of the AF-GCN network to design the theoretical architecture of the module.

[0071] The entire network takes the scene original point cloud data as input. In the point cloud target segmentation module, first, a multi-layer perception embedding layer is used to map the feature values of the original point cloud data into a feature space and finally output point cloud data features.

[0072] The multi-layer perception embedding layer establishes a nonlinear mapping:

[0073] The formula is as follows:

[0074] Γ: R N×M → R N×C

[0075] Where N represents the number of points in the original point cloud data, M represents the dimension of the original point cloud data features, and C represents the feature dimension required for the point cloud data feature extraction by the first encoder.

[0076] In the first encoder, the point cloud data features are processed through downsampling. The first encoder includes N layers of first sub-encoders. The point cloud data feature dimension input into each layer of the first sub-encoder is C", and the point cloud data feature dimension output from each layer of the first sub-encoder is 2xC".

[0077] The point cloud data scale input into each layer of the first sub-encoder is N', and the point cloud data scale output from each layer of the first sub-encoder is N' / S, where S represents the point cloud data scale reduction ratio in the downsampling process. After downsampling, the point cloud data features are aggregated and different-dimensional point cloud data features of different scale point cloud data are extracted through graph convolution technology.

[0078] As shown in Figure 5 FIG. 1 is a structural diagram of a GCN (Graph Convolutional Networks, GCN) module in an encoder of an embodiment of the present application. The structure extracts point cloud data features according to the coordinates of points and the graph structure of the point cloud data, and fuses the two features through convolution and a multi-layer perceptron (MLP) to obtain the features of a graph composed of neighboring points of a certain point in the point cloud data, and finally aggregates through pooling to obtain higher-level and more abstract features of the point.

[0079] Then, the encoder of the point cloud target segmentation module extracts different dimensional point cloud data features of different scales of point cloud data from the point cloud data features by using down-sampling and graph convolution technology.

[0080] In step 104, the different dimensional point cloud data features of different scales of point cloud data are input into a first decoder for processing. Each first sub-decoder in the first decoder takes the point cloud data features output by the previous layer of first sub-decoder and the point cloud data features input by the corresponding level of first sub-encoder as input, and obtains the point cloud data features of all points.

[0081] The decoder of the point cloud target segmentation module aggregates the features of different dimensions of different scales of point cloud data as input. Each decoder takes the point cloud data features output by the previous layer of decoder and the point cloud data features input by the corresponding level of encoder as input, and the final decoder outputs the point cloud data features of all points in the scene. The features will be retained and applied as prior knowledge to the decoder of the point cloud target contour extraction module.

[0082] As shown in Figure 6 FIG. 2 is a structural diagram of a GAF (Graph Attention Filter, GAF) module of the decoder in an embodiment of the present application. The module has basically the same structure as the GCN module, except that the GAF module generates an attention weight matrix based on the spatial distance between the midpoints of the point cloud data when extracting the features of the graph structure, and suppresses the useless information provided by the features from the distant points while applying the graph convolution to extract the point cloud data features through the attention mechanism.

[0083] In addition, in the ordinary GCN and GAF modules, the BallQuery algorithm is used to find the neighboring points of each point in the point cloud data to build the graph structure. In the corresponding modules of up-sampling and down-sampling, the BallQuery algorithm is used to find the neighboring points of each point in the sampled point cloud data in the point cloud data before sampling to build the graph structure.

[0084] The first layer decoder of the module aggregates different point cloud data features with each other by taking the high-dimensional features extracted by the last layer encoder as input. The remaining decoders take the point cloud data features output by the last layer decoder and the input point cloud data features of the corresponding layer encoder as input. The dimension of the point cloud data features input by each decoder is C''', and the dimension of the point cloud data features output by each decoder is C''' / 2. The scale of the point cloud data input by each layer decoder is N'', and the scale of the point cloud data output by each layer decoder is N''xS, where S is the magnification of the point cloud data scale in the upsampling process. Finally, the decoder outputs the spatial features of each point in the scene, which will be retained and applied as prior knowledge to the decoder of the point cloud target contour extraction module. After upsampling, the different dimension point cloud data features of different scale point cloud data are aggregated and the point cloud data features of all points are extracted through the graph convolution technology.

[0085] In step 105, the point cloud data features of all points are input into the segmentation classifier for processing to obtain point cloud individual data of different categories.

[0086] For the point cloud target segmentation module, the multi-class cross-entropy is used as the loss function of the point cloud target segmentation module. In the process of calculating the loss function, the output of the point cloud target segmentation module is defined as:

[0087]

[0088] where n represents the number of points in the point cloud data, c represents the number of categories of the point individuals after the point cloud target segmentation, Y i represents the calculation set of different categories of the i-th point in the point cloud data, represents the calculation result corresponding to the m-th category of the i-th point of the point cloud target segmentation module.

[0089] The multi-class cross-entropy is used as the loss function of the point cloud target segmentation module. The loss function first normalizes the calculation result of the network by using the softmax function, then takes the logarithm operation for the normalized result, and finally calculates according to the following formula:

[0090]

[0091] where Q i represents the unique encoding of the segmentation category label of each point in the point cloud data.

[0092] Finally, the scene will be segmented into point cloud individuals of different categories by the segmentation classifier predicting semantic labels for each point in the scene. At the end of the point cloud target segmentation module, the segmentation classifier predicts semantic labels for each point in the scene according to the spatial features of each point in the scene, and the point cloud scene will be segmented into point cloud individuals of different categories.

[0093] In step 106, the point cloud individual data of different categories is embedded into a feature space by a second multi-layer perception embedding layer, and the point cloud individual data of different categories is processed by a second encoder and a second decoder to obtain a composite point cloud data feature of each point in the point cloud individual data of different categories.

[0094] In the point cloud target contour extraction module, a network architecture similar to that of the point cloud target segmentation module is adopted. However, compared with the point cloud target segmentation module, the point cloud target contour extraction module needs to extract more comprehensive and important point cloud data features. Therefore, in the point cloud target contour extraction module, in order to aggregate point cloud data features more finely and comprehensively, the point cloud target contour extraction module uses an attention pooling (AP) mechanism to complete the pooling of point cloud features. Therefore, the graph convolutional networks (GCN) and the graph attention filter (GAF) in this module are replaced by the attention pooling-graph convolutional networks (AP-GCN) and the attention pooling-graph attention filter (AP-GAF).

[0095] In the point cloud data used in the point cloud target contour extraction, the number of non-contour points often far exceeds that of contour points. This imbalance leads to the model easily deviating to the non-contour points with a larger number during training, thereby ignoring the learning of contour points. If the same as the point cloud target segmentation module, the training loss is directly calculated by the ordinary binary cross-entropy. Due to the large number of non-contour points, the model often pays excessive attention to the non-contour points, thereby overfitting the non-contour points. In order to pursue a higher accuracy, the model tends to identify all points in the scene point cloud as non-contour points.

[0096] To solve the above problems, a method can be used, which is to use weighted binary cross entropy as the loss function, and give higher weight to the loss of the contour points in the training process, so that the model pays more attention to the contour points in the learning process. However, through the experiment process of the embodiments of the present application, it is found that only using weighted binary cross entropy can alleviate the problem of imbalance between positive and negative samples to a certain extent, but cannot completely avoid the overfitting phenomenon in the training process.

[0097] In addition, due to the serious imbalance between positive and negative samples of the point cloud data used by the point cloud target contour extraction module, there is a problem that the model is prone to overfitting to the larger sample in the learning process. To solve this problem, the embodiments of the present application introduce a composite loss function into the point cloud target contour extraction module, which combines the chamfer distance and binary classification cross entropy in a weighted sum manner to control the parameter optimization direction of the model in the learning process and avoid overfitting in the training.

[0098] In order to better solve the overfitting problem caused by the imbalance between positive and negative samples of point cloud data and optimize the learning effect of the model, the embodiments of the present application use two kinds of loss functions to combine a relatively complex composite loss function to adjust the parameter optimization direction of the model. The composite loss function designed in this paper mainly includes two parts: chamfer distance and weighted binary classification cross entropy.

[0099] Chamfer distance is often used in the calculation of loss function of point cloud reconstruction. This method calculates the average value of the minimum distance from each point in one point cloud to all points in another point cloud. Therefore, this method can intuitively show the contour similarity between the predicted point cloud data of the model and the actual point cloud data, and can well measure the effect of point cloud target contour extraction from the final recognition result. In the process of calculating the chamfer distance, the set of contour points of the actual point cloud target is defined as:

[0100]

[0101] Where n g represents the number of contour points in the actual point cloud target, p i represents the coordinates of the i-th contour point in S1.

[0102] Similarly, the set of contour points of the predicted point cloud target is defined as:

[0103] Where n p represents the number of contour points in the predicted point cloud target, p j represents the coordinates of the j-th contour point in S2.

[0104] Specifically, first, based on a point p i, find a point p in S2 j So that p j to p i The distance is the smallest, and the minimum distance is defined as Then based on a point p in S2 j , find a point p in S1 i So that p i to p j The distance is the smallest, and the minimum distance is defined as Then find all the points in S1 The average value of all points in S2 The average value of is summed up as the loss value calculated based on the chamfer distance. The specific calculation process is as follows:

[0105]

[0106] Among them, L CD represents the chamfer distance loss function, n g Indicates the number of contour points of the actual point cloud target, S1 represents the contour point set of the actual point cloud target, p i represents the coordinates of the i-th contour point in S1, S2 represents the contour point set of the predicted point cloud target, and p j Represents the coordinates of the i-th contour point in S2, and np represents the number of contour points of the predicted point cloud target;

[0107] Binary classification cross entropy is widely used in binary classification problems as a method of calculating model training loss. The core concept of this method is to represent the difference between the probability distribution of different categories of samples predicted by the model and the true label value of the sample. If the predicted probability is closer to the label value, the value of the cross entropy loss function is smaller, and vice versa. The value of the cross entropy loss function is larger. Based on this principle, this method can be used to measure whether the binary classification prediction result is accurate. Since the point cloud data used by the point cloud target contour extraction module has the problem of sample imbalance, the embodiment of the present application sets weight values ​​for the binary cross entropy calculation of different categories, so that the model pays more attention to contour points when learning. When calculating the weighted binary classification cross entropy, the embodiment of the present application defines the output of the point cloud target contour extraction module and the weight of each category as follows:

[0108]

[0109] Among them, W i Represents the loss calculation weight matrix of the i-th point in the point cloud data, n g1 Indicates the number of non-contour points in the actual point cloud data, n p1 Indicates the number of contour points in the actual point cloud data, Y i Represents the calculation result set of the i-th point in the point cloud data,

[0110] The formula of the loss function based on the weighted binary classification cross entropy is as follows:

[0111]

[0112] wherein, L CE represents the weighted binary classification cross entropy loss function, Q i represents the one-hot encoding of the contour point identification label of each point in the point cloud data.

[0113] By comparison, the chamfer distance compares the point cloud data from the angle of the sample label predicted by the model itself, and the weighted binary classification cross entropy mainly calculates the training loss based on the predicted probability of different categories by the model. Both of them measure the model prediction result from different angles. The point cloud target contour point extraction module loss function of the embodiments of the present application combines the two loss functions to constrain the parameter optimization direction of the model from different dimensions. The composite loss function is mainly realized in the form of weighted summation of the two loss functions, and the formula is as follows:

[0114] L = aL CD + bL CE

[0115] wherein, L represents the composite loss function, a and b represent the weights of L CD and L CE , respectively, a+b = 1, a = 0.3 and b = 0.7 in the embodiments of the present application.

[0116] Since the point cloud target contour extraction module and the point cloud target segmentation module use similar module settings, the point cloud data features obtained by the decoder in the point cloud target segmentation module are used as prior knowledge in the embodiments of the present application, and the point cloud data features extracted by the two modules are fused in the decoder of the point cloud target contour extraction module. Through the feature sharing mode, the point cloud target segmentation and the point cloud target contour extraction module are more effectively connected to each other, and the point cloud data features extracted by the point cloud target segmentation module are fully utilized to provide more abundant point cloud data features for the point cloud target contour extraction.

[0117] In the process of extracting the point cloud target contour from the scene point cloud data, not only the curve of each individual surface needs to be extracted, but also the boundary between different individuals as part of the scene structure is important. At the same time, the point cloud target segmentation module accurately segments the scene point cloud data into several independent point cloud subsets belonging to different categories, which correspond to different objects or different components in the scene. In the segmentation process, the accurate segmentation of the scene point cloud data often means the accurate identification of the boundary between the point cloud individuals. Therefore, applying the point cloud data features extracted by the last decoder of the point cloud target segmentation module as prior knowledge to the last decoder of the point cloud target contour extraction module not only makes the point cloud target contour extraction module more sensitive to the boundary between the point cloud individuals, but also further optimizes the model training effect.

[0118] The embodiment of the present application reconstructs the last decoder of the point cloud target contour extraction module, and fuses the point cloud data features extracted by the point cloud target segmentation module with the point cloud data features extracted by the point cloud target contour extraction module.

[0119] In the decoder structure of the point cloud target contour extraction module without reconstruction, the features Featurel output by the previous level decoder and the input point cloud data features Feature2 of the corresponding level encoder are taken as inputs. Firstly, the graph structure built in the upsampling process is used to aggregate Featurel to generate the features of all points in the upsampled point cloud data, and Feature2 is further processed by a multi-layer perceptron to obtain a higher level representation of Feature2. Then, the two features are spliced and deeply fused by convolution operation. Finally, the fused features are taken as the output of the entire decoder for classification operation by the classifier.

[0120] In the reconstructed last decoder, the double-input head structure of the original last decoder is changed to a three-input head structure, and the point cloud features Feature3 extracted by the point cloud data segmentation module are added as a new input. Firstly, the newly added Feature3 is directly fused with Feature2 by element-wise addition after being input into the last encoder, which increases the feature information amount of Feature2. Then, the summed features and Featurel are fused by concatenation and convolution, and finally the fused point cloud data features are output for classification by the contour classifier.

[0121] By applying the point cloud data features extracted by the last layer encoder of the point cloud target segmentation module as prior knowledge to the last layer encoder of the point cloud target contour extraction module, the two modules can be connected together to make the network structure more coherent, and the amount of information of the point cloud data features extracted by the point cloud target contour extraction module is increased, and the final effect of the point cloud target contour extraction is improved. In addition, the prior knowledge is directly merged with Feature2 after being input into the decoder, and the input and output feature dimensions of the point cloud target extraction module are not changed. The point cloud target segmentation module and the point cloud target contour extraction module can not only be jointly trained, but also be separated for training, which increases the flexibility of the experiment and allows for the design of various training strategies as needed.

[0122] The point cloud target contour extraction module takes different categories of point cloud individuals as input. First, the multi-layer perception embedding layer is used to embed the point coordinates of different individuals into the feature space. Then, the encoder in the point cloud target contour extraction module further learns the features of different dimensions of point cloud data of different scales, and the decoder aggregates different point cloud data features. Except for the last layer decoder, each layer decoder still takes the features output by the previous layer decoder and the input point cloud data features of the corresponding level encoder. The last layer decoder takes the point cloud data features extracted by the point cloud target segmentation module, the input point cloud data features of the corresponding level encoder, and the point cloud data features extracted by the last layer decoder as input and fuses them together, and finally outputs the composite point cloud data features of each point in the individual.

[0123] In the SCE-Net network, after the graph convolution extracts the features of a point and its neighboring points in the point cloud based on the neighborhood of the point, it needs to aggregate the features of all points in the neighborhood through a pooling layer, and finally combine the aggregated features with the features of the point itself.

[0124] Among the existing pooling methods, the average pooling method or the maximum pooling method is often used to aggregate point cloud data features in recent point cloud data processing related work, but both methods have certain limitations. Using the maximum pooling method to aggregate features will select the maximum value of the features among the neighboring points as the representative feature of all points in the neighborhood. This method can intuitively identify the most important features, but at the same time it will ignore other point cloud data features except the maximum value, resulting in the loss of most point cloud data features. However, for the point cloud target contour extraction module, the maximum pooling method is not fine and comprehensive enough, and often ignores some point cloud data features that have a great contribution to the point cloud target contour extraction. By using the average pooling method to aggregate features, the average value of the features of a point and its neighboring points in the point cloud data is calculated to obtain a unified feature representation. Although this method takes into account all point cloud data features, it cannot distinguish between different point cloud data features according to their importance, and cannot highlight more important point cloud data features. In contrast, the weight-based pooling method can take into account the features of all points in the neighborhood by controlling the pooling weight, and can focus on learning important point cloud data features by controlling the weight value, making it more suitable for application in the point cloud target contour extraction module.

[0125] In the pooling layer, the present application embodiment dynamically allocates attention weights to the point cloud data features of the neighboring points of each point in the point cloud data, distinguishes different feature values of each point according to the importance, and strengthens the learning of important point cloud data features when aggregating features. In the design of the strategy for allocating pooling weights, this paper is inspired by previous work and integrates the attention mechanism into the pooling layer. Instead of using a fixed weight allocation strategy to allocate point cloud data feature weights and aggregate point cloud data features, the weight allocation strategy of the pooling layer is continuously learned and optimized during the training process, making the aggregation of point cloud data features more flexible.

[0126] The attention pooling layer takes the set of point cloud data features of the k neighboring points in the neighborhood of each point in the point cloud data as input. In order to identify important features in the point cloud data features, the importance of different point features needs to be evaluated first. The formula of the feature in the point cloud data is as follows:

[0127]

[0128] where F i represents the set of feature vectors corresponding to all neighboring points in the neighborhood of the i-th point in the point cloud data, represents the feature vector corresponding to the m-th neighboring point in the neighborhood of the point, and k represents the number of neighboring points in the neighborhood of the point.

[0129] The embodiments of the present application implement the evaluation of the importance of point cloud data features based on linear mapping. The feature vectors of all points in the neighborhood of a certain point are used to calculate the corresponding importance coefficients of the feature vectors of each neighboring point of the point using linear mapping, and the weight matrix of the linear mapping can be obtained and continuously optimized in the learning process, ensuring that more important features in the point cloud data are found more accurately. After completing the importance evaluation, the importance coefficients of the point cloud data features are normalized using the softmax function and the attention weight matrix is finally calculated, so that each feature value can be properly focused according to its importance in the subsequent weight mapping process. After the attention weight matrix is calculated, the point cloud data features are multiplied element by element with the attention weight matrix to obtain the point cloud data features after the attention weight is allocated. The formula of attention weight allocation is as follows:

[0130]

[0131] wherein is the point cloud data feature after weight mapping processing, represents the linear mapping used to calculate the attention matrix.

[0132] After completing the attention weight allocation, each feature value of all neighboring points needs to be summed according to the graph structure to complete the feature aggregation of all neighboring points. In addition, in order to fully learn the aggregated point cloud data features, MLP needs to be used to analyze the aggregated point cloud data features more deeply and extract point cloud data features with higher abstraction. The pooling formula based on the attention mechanism is as follows:

[0133]

[0134] wherein, represents the set of point cloud feature vectors obtained after feature aggregation.

[0135] In comparison, the point cloud data feature extraction module based on the graph attention filter used in SCE-Net mainly uses the spatial distance between each point in the point cloud data and each neighboring point in its neighborhood as the main basis for attention weight calculation, so that the entire network can suppress useless information from distant points when extracting features; the attention pooling mechanism mainly uses the importance of different dimensions of point cloud data features as the main basis for attention weight calculation, so that the entire network can focus on more important point cloud data features when aggregating features. Both of them adjust the focus of model learning from different dimensions. The attention pooling mechanism proposed in the embodiments of the present application is a supplement to the graph attention filter, which further strengthens the learning of important point cloud data features while aggregating the data features of all neighboring points.

[0136] In step 107, the composite point cloud data feature pair is classified by the contour classifier to perform binary classification on all points in the point cloud individual data of different categories, to obtain a point cloud target contour extraction result of the entire scene.

[0137] Finally, the contour classifier classifies the composite point cloud data feature pair of each point in the point cloud data to all points in the point cloud individual, and integrates the classification results in different individual point cloud data, to finally obtain a point cloud target contour extraction result of the entire scene.

[0138] To verify the effect of the method provided in the embodiment, experiments are performed on the training and testing of the SCE-Net on the S3DIS dataset, and the contour extraction effect of a room in the dataset is obtained after manual improvement and contour labeling of the S3DIS dataset, so that it can be concluded that the network has achieved good results in point cloud target contour extraction, and the SCE-Net provided in the embodiment basically has the ability to identify contour points in the point cloud target.

[0139] The experimental results of the model test of the SCE-Net on the S3DIS dataset are shown in the following table:

[0140] Table 1

[0141] Accuracy Precision Recall F1 Score 0.88 0.29 0.68 0.40

[0142] It can be concluded that the combination of point cloud target segmentation technology and point cloud target contour extraction technology has better recognition effect on contour points of point cloud data collected based on an actual scene.

[0143] As shown in Figure 2 The embodiment provides a target contour extraction device based on the SCE-Net, which comprises a data acquisition module 201, a network construction module 202 and a data processing module 203, wherein:

[0144] The data acquisition module 201 is configured to acquire original point cloud data.

[0145] The network construction module 202 is configured to construct the SCE-Net network, and the SCE-Net network comprises a point cloud target segmentation module and a point cloud target contour extraction module, and the point cloud target segmentation module and the point cloud target contour extraction module are connected in series, wherein the point cloud target segmentation module comprises a first multi-layer perception embedding layer, a first encoder, a first decoder and a segmentation classifier connected in sequence; the first encoder comprises N first sub-encoders, and the first decoder comprises N first sub-decoders; the point cloud target contour extraction module comprises a second multi-layer perception embedding layer, a second encoder, a second decoder and a contour classifier connected in sequence; the second encoder comprises N second sub-encoders, and the second decoder comprises N second sub-decoders.

[0146] The data processing module 203 is configured to input the original point cloud data into a first multi-layer perception embedding layer for processing to obtain point cloud data features; input the point cloud data features into a first encoder for processing to obtain different-dimensional point cloud data features of different scale point cloud data; input the different-dimensional point cloud data features of different scale point cloud data into a first decoder for processing, each layer of first sub-decoder in the first decoder taking the point cloud data features output by the previous layer of first sub-decoder and the point cloud data features input by the corresponding level of first sub-encoder as input to obtain point cloud data features of all points; input the point cloud data features of all points into a segmentation classifier for processing to obtain point cloud individual data of different categories; embed the point cloud individual data of different categories into a feature space through a second multi-layer perception embedding layer, process the point cloud individual data of different categories through a second encoder and a second decoder to obtain composite point cloud data features of each point in the point cloud individual data of different categories; perform binary classification on all points in the point cloud individual data of different categories through a contour classifier based on the composite point cloud data features to obtain a point cloud target contour extraction result of the entire scene.

[0147] The SCE-Net-based target contour extraction apparatus in the embodiments of the present application can be a computer device or a component in a computer device, such as an integrated circuit or a chip. The computer device can be a terminal or other devices other than a terminal. For example, the computer device can be a mobile phone, a tablet computer, a notebook computer, a palm computer, a vehicle-mounted computer device, a Mobile Internet Device (MID), an Ultra-Mobile Personal Computer (UMPC), a netbook, or a Personal Digital Assistant (PDA), and can also be a server, a Network Attached Storage (NAS), a Personal Computer (PC), and the like, and the embodiments of the present application are not limited in this regard.

[0148] The SCE-Net-based target contour extraction apparatus provided in the embodiments of the present application can realize each process of the SCE-Net-based target contour extraction method embodiment, and details are not repeated here to avoid repetition. Figure 1 The SCE-Net-based target contour extraction apparatus provided in the embodiments of the present application can realize each process of the SCE-Net-based target contour extraction method embodiment, and details are not repeated here to avoid repetition.

[0149] The embodiments of the present application further provide a computer device, such as a terminal, a server, a Network Attached Storage (NAS), a Personal Computer (PC), and the like. Figure 3As shown, the computer device includes a processor 301 and a memory 302, and the memory 302 stores a program or instruction executable on the processor 301, which, when executed by the processor 301, implements each step of the above-mentioned SCE-Net-based target contour extraction method and achieves the same technical effects. To avoid repetition, details are not described here.

[0150] It should be noted that the computer device in the embodiments of the present application includes the above-mentioned mobile computer device and non-mobile computer device.

[0151] The memory 302 can be used to store software programs and various data. The memory 302 can mainly include a first storage area storing programs or instructions and a second storage area storing data, wherein the first storage area can store an operating system, at least one application program or instruction required by a function (such as a sound playing function, an image playing function, etc.), etc. In addition, the memory 302 can include a volatile memory or a non-volatile memory, or the memory 302 can include both volatile and non-volatile memories. The non-volatile memory can be a Read-Only Memory (ROM), a Programmable ROM (PROM), an Erasable PROM (EPROM), an Electrically EPROM (EEPROM), or a flash memory. The volatile memory can be a Random Access Memory (RAM), a Static RAM (SRAM), a Dynamic RAM (DRAM), a Synchronous DRAM (SDRAM), a Double Data Rate SDRAM (DDR SDRAM), an Enhanced SDRAM (ESDRAM), a Synch link DRAM (SLDRAM), and a Direct Rambus RAM (DRRAM). The memory 302 in the embodiments of the present application includes but is not limited to these and any other suitable types of memories.

[0152] The processor 301 can include one or more processing units; optionally, the processor 301 integrates an application processor and a modem processor, wherein the application processor mainly processes operations related to an operating system, a user interface, and an application program, and the modem processor mainly processes a wireless communication signal, such as a baseband processor. It can be understood that the above-mentioned modem processor can also not be integrated into the processor 301.

[0153] The embodiment of the present application further provides a readable storage medium, and the readable storage medium stores a program or an instruction, the program or the instruction is executed by a processor to realize each process of the above-mentioned target contour extraction method based on the SCE-Net, and the same technical effects can be achieved, to avoid repetition, which will not be described here.

[0154] The embodiment of the present application further provides a chip, and the chip includes a processor and a communication interface, the communication interface is coupled with the processor, and the processor is used to run a program or an instruction to realize each process of the above-mentioned target contour extraction method based on the SCE-Net, and the same technical effects can be achieved, to avoid repetition, which will not be described here.

[0155] It should be understood that the chip mentioned in the embodiment of the present application can also be referred to as a system-level chip, a system chip, a chip system, or a system-on-chip, etc.

[0156] The embodiment of the present application further provides a computer program product, and the program product is stored in a storage medium, the program product is executed by at least one processor to realize each process of the above-mentioned target contour extraction method based on the SCE-Net, and the same technical effects can be achieved, to avoid repetition, which will not be described here.

[0157] It should be noted that in this document, the terms "comprising", "including", or any other variant thereof are intended to cover non-exclusive inclusion, so that processes, methods, articles, or devices including a series of elements not only include those elements, but also include other elements not explicitly listed, or include elements inherent to such processes, methods, articles, or devices. Without more limitations, the element defined by the statement "including a" does not exclude the presence of additional identical elements in the process, method, article, or device including the element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to the order of performing the functions as shown or discussed, and can also include performing the functions in a substantially simultaneous manner or in a reverse order, for example, the described method can be performed in an order different from the described order, and various steps can be added, omitted, or combined. In addition, the features described with reference to certain examples can be combined in other examples.

[0158] The embodiments of the present application are described above with reference to the accompanying drawings, but the present application is not limited to the specific embodiments described above, and the specific embodiments described above are merely illustrative, but not restrictive, and a person of ordinary skill in the art can make many forms under the inspiration of the present application without departing from the purpose of the present application and the scope protected by the claims.

Claims

1. A target contour extraction method based on SCE-Net, characterized in that: The method comprises: Acquire original point cloud data; construct an SCE-Net network, wherein the SCE-Net network includes a point cloud target segmentation module and a point cloud target contour extraction module, wherein the point cloud target segmentation module and the point cloud target contour extraction module are connected in series; The point cloud object segmentation module includes a first multi-layer perceptron embedding layer, a first encoder, a first decoder, and a segmentation classifier connected in sequence; the first encoder includes an N-layer first sub-encoder, and the first decoder includes an N-layer first sub-decoder; Inputting the original point cloud data into the first multi-layer perceptron embedding layer for processing to obtain point cloud data features; inputting the point cloud data features into the first encoder for processing to obtain point cloud data features of different dimensions of point cloud data of different scales; Inputting point cloud data features of different dimensions of the point cloud data of different scales into the first decoder for processing, the first sub-decoder of each layer in the first decoder takes the point cloud data features output by the first sub-decoder of the previous layer and the point cloud data features input by the first sub-encoder of the corresponding layer as input to obtain point cloud data features of all points; inputting the point cloud data features of all points into the segmentation classifier for processing to obtain individual point cloud data of different categories; The point cloud target contour extraction module includes a second multi-layer perceptron embedding layer, a second encoder, a second decoder and a contour classifier connected in sequence; the second encoder includes N layers of second sub-encoders, and the second decoder includes N layers of second sub-decoders. In the second sub-decoder of the point cloud target contour extraction module, except for the last layer of second sub-decoder, the second sub-decoders of each layer take the features output by the second sub-decoder of the previous layer and the input point cloud data features of the second sub-encoder of the corresponding layer as input, and the last layer decoder fuses the point cloud data features of all points obtained by the point cloud target segmentation module, the point cloud data features input by the second sub-encoder of the corresponding layer, and the features output by the second sub-decoder of the previous layer as input to obtain the composite point cloud data features of each point in the point cloud individual data of different categories; the point cloud individual data of different categories are embedded in the feature space through the second multi-layer perceptron embedding layer, and the point cloud individual data of different categories are processed by the second encoder and the second decoder to obtain the composite point cloud data features of each point in the point cloud individual data of different categories; The contour classifier uses the composite point cloud data features to perform binary classification on all points in the point cloud individual data of different categories, and obtains the point cloud target contour extraction result of the entire scene.

2. The target contour extraction method based on SCE-Net according to claim 1, characterized in that: The step of inputting the original point cloud data into the first multi-layer perceptron embedding layer for processing to obtain point cloud data features includes: A nonlinear mapping is established in the first multi-layer perceptron embedding layer, and the eigenvalues ​​of the original point cloud data are extracted through the first multi-layer perceptron embedding layer. The eigenvalues ​​of the original point cloud data are mapped to the feature space, and the point cloud data features are output. The formula is as follows: Where N represents the number of points in the original point cloud data, M represents the dimension of the original point cloud data features, and C represents the feature dimension required for input to the first encoder for point cloud data feature extraction.

3. The target contour extraction method based on SCE-Net according to claim 1, characterized in that: Inputting the point cloud data features into the first encoder for processing to obtain point cloud data features of different dimensions for point cloud data of different scales includes: The point cloud data features are processed by downsampling in the first encoder. The first encoder includes N layers of first sub-encoders. The feature dimension of the point cloud data input by each layer of the first sub-encoder is , the feature dimension of the point cloud data output by the first sub-encoder of each layer is ; The point cloud data scale of the first sub-encoder input of each layer is , the point cloud data scale output by the first sub-encoder of each layer is , where S represents the reduction ratio of the point cloud data scale in the downsampling process. After the downsampling is completed, the point cloud data features are aggregated through graph convolution technology and the point cloud data features of different dimensions of different scales are extracted.

4. The target contour extraction method based on SCE-Net according to claim 1, characterized in that: Inputting the point cloud data features of different dimensions of the point cloud data of different scales into the first decoder for processing, the first sub-decoder of each layer in the first decoder takes the point cloud data features output by the first sub-decoder of the previous layer and the point cloud data features input by the first sub-encoder of the corresponding layer as input, and obtains the point cloud data features of all points, including: In the first decoder, the point cloud data features of different dimensions of the point cloud data of different scales are processed by upsampling. The first decoder includes N layers of first sub-decoders. The feature dimension of the point cloud data input by each layer of the first sub-decoder is , the dimension of the point cloud data features output by the first sub-decoder of each layer is ; The point cloud data scale of the first sub-decoder input of each layer is , the point cloud data scale output by the first sub-decoder of each layer is , where S is the point cloud data scale magnification during the upsampling process. After the upsampling is completed, the graph convolution technology is used to aggregate the point cloud data features of different dimensions of point cloud data of different scales and extract the point cloud data features of all points.

5. The target contour extraction method based on SCE-Net according to claim 1, characterized in that: The point cloud data features of all points are input into the segmentation classifier for processing to obtain point cloud individual data of different categories, including: In the segmentation classifier, the semantic label of each point in the point cloud data features of all points is predicted, points with the same semantic label are divided into the same category, and points with different semantic labels are segmented to obtain individual point cloud data of different categories.

6. The target contour extraction method based on SCE-Net according to claim 1, characterized in that: A composite loss function is introduced into the point cloud target contour extraction module. The composite loss function includes a chamfer distance loss function and a weighted binary classification cross entropy loss function. The formula of the chamfer distance loss function is as follows: in, represents the chamfer distance loss function, Indicates the number of contour points of the actual point cloud target, Represents the contour point set of the actual point cloud target, express The coordinates of the i-th contour point in , Represents the set of contour points of the predicted point cloud target, express The coordinates of the i-th contour point in , Indicates the number of contour points of the predicted point cloud target; The weighted binary classification cross entropy loss function formula is as follows: in, represents the weighted binary classification cross entropy loss function, Represents the unique hot encoding of the contour point identification label of each point in the point cloud data, Represents the loss calculation weight matrix of the i-th point in the point cloud data, Indicates the number of non-contour points in the actual point cloud data, Indicates the number of contour points in the actual point cloud data, Represents the calculation result set of the i-th point in the point cloud data, ; The composite loss function formula is as follows: Among them, L represents the composite loss function, α and β represent and The weight of , α+β=1.

7. A target contour extraction device based on SCE-Net, characterized in that: The device comprises: A data acquisition module is configured to acquire raw point cloud data; A network construction module is configured to construct an SCE-Net network, wherein the SCE-Net network includes a point cloud target segmentation module and a point cloud target contour extraction module, wherein the point cloud target segmentation module and the point cloud target contour extraction module are connected in series, wherein the point cloud target segmentation module includes a first multi-layer perceptron embedding layer, a first encoder, a first decoder, and a segmentation classifier connected in sequence; the first encoder includes N layers of first sub-encoders, and the first decoder includes N layers of first sub-decoders; the point cloud target contour extraction module includes a second multi-layer perceptron embedding layer, a second encoder, a second decoder, and a contour classifier connected in sequence; The second encoder includes N layers of second sub-encoders, and the second decoder includes N layers of second sub-decoders. In the second sub-decoders of the point cloud object contour extraction module, except for the second sub-decoder of the last layer, the second sub-decoders of each layer take the features output by the second sub-decoder of the previous layer and the input point cloud data features of the second sub-encoder of the corresponding layer as input. The last layer decoder fuses the point cloud data features of all points obtained by the point cloud object segmentation module, the point cloud data features input by the second sub-encoder of the corresponding layer, and the features output by the second sub-decoder of the previous layer as input to obtain composite point cloud data features for each point in different categories of point cloud individual data; The data processing module is configured to input the original point cloud data into the first multi-layer perceptron embedding layer for processing to obtain point cloud data features; input the point cloud data features into the first encoder for processing to obtain point cloud data features of different dimensions of point cloud data of different scales; input the point cloud data features of different dimensions of point cloud data of different scales into the first decoder for processing, and the first sub-decoder of each layer in the first decoder uses the point cloud data features output by the first sub-decoder of the previous layer and the point cloud data features input by the first sub-encoder of the corresponding layer as input to obtain point cloud data features of all points; input the point cloud data features of all points into the segmentation classifier for processing to obtain point cloud individual data of different categories; embed the point cloud individual data of different categories into the feature space through the second multi-layer perceptron embedding layer, process the point cloud individual data of different categories through the second encoder and the second decoder to obtain composite point cloud data features for each point in the point cloud individual data of different categories; and perform binary classification on all points in the point cloud individual data of different categories using the composite point cloud data features through the contour classifier to obtain the point cloud target contour extraction result of the entire scene.

8. A target contour extraction system based on SCE-Net, the system comprising a processor and a memory, wherein the memory stores a computer program, characterized in that: The computer program is loaded and executed by the processor to implement the target contour extraction method based on SCE-Net according to any one of claims 1 to 7.

9. A computer-readable storage medium storing a computer program, wherein: When the computer program is executed by a processor, it is used to implement the target contour extraction method based on SCE-Net according to any one of claims 1 to 7.

Images