Color point cloud segmentation method and device, server and computer readable storage medium

Abstract

The invention is suitable for the technical field of point cloud segmentation, and provides a color point cloud segmentation method and device, a server and a computer readable storage medium, and the method comprises the steps: obtaining a to-be-processed point cloud, and carrying out the preprocessing of the to-be-processed point cloud; extracting color features and non-color features of the preprocessed to-be-processed point cloud, and performing feature fusion on the color features and the non-color features to obtain attribute vectors; inputting the attribute vector into a preset deep learning model to obtain a first point cloud segmentation result; in the point cloud segmentation process, the color features and the non-color features are fully combined to segment the point cloud, so that the accuracy of point cloud segmentation is improved.

Description

Technical Field

[0001] The present application belongs to the technical field of point cloud segmentation, and in particular, relates to a color point cloud segmentation method, device, server and computer-readable storage medium. Background Art

[0002] Point cloud data is a commonly used form of 3D spatial data. However, due to the limitations of acquisition equipment, the quality and quantity of point cloud data face great challenges. In the existing technology, the segmentation of color point clouds is simply based on the color features of the point clouds, resulting in poor accuracy of point cloud segmentation. Summary of the invention

[0003] The embodiments of the present application provide a color point cloud segmentation method, device, server and computer-readable storage medium, which can solve the problem of poor accuracy of point cloud segmentation in the prior art.

[0004] In a first aspect, an embodiment of the present application provides a color point cloud segmentation method, comprising: Acquire a point cloud to be processed and preprocess the point cloud to be processed; Extracting color features and non-color features of the preprocessed point cloud to be processed, and performing feature fusion on the color features and the non-color features to obtain an attribute vector; The attribute vector is input into a preset deep learning model to obtain a first point cloud segmentation result.

[0005] In a possible implementation manner of the first aspect, after inputting the attribute vector into a preset deep learning model to obtain a first point cloud segmentation result, the method further includes: Based on the first point cloud segmentation result, a region growing method is used to divide the point cloud to be processed into regions to obtain a second point cloud segmentation result.

[0006] In a possible implementation manner of the first aspect, based on the first point cloud segmentation result, a region growing method is used to perform region division on the point cloud to be processed to obtain a second point cloud segmentation result, further comprising: Based on the second point cloud segmentation result, construct a point cloud graph structure; wherein the point cloud graph structure includes the point clouds to be processed as nodes and the association relationships between the point clouds to be processed as edges; According to the point cloud graph structure, the second point cloud segmentation result is optimized based on a minimum spanning tree algorithm.

[0007] In a possible implementation manner of the first aspect, the non-color features include light features, geometric features, and position and posture information corresponding to a scanning device that collects the point cloud to be processed.

[0008] In a possible implementation manner of the first aspect, performing feature fusion on the color feature and the non-color feature to obtain an attribute vector includes: Performing vectorization processing on the color feature to obtain a first feature vector; Performing vectorization processing on the light feature to obtain a second feature vector; Performing vectorization processing on the geometric features to obtain a third feature vector; Performing vector processing on the position and posture information corresponding to the scanning device that collects the point cloud to be processed to obtain a fourth eigenvector; Based on the first eigenvector, the second eigenvector, the third eigenvector and the fourth eigenvector, an attribute vector is obtained according to the following formula: ; in, represents the attribute vector, represents the first eigenvector, represents the second eigenvector, represents the third eigenvector, represents the fourth eigenvector, represents the first weight value corresponding to the first eigenvector, represents the second weight value corresponding to the second eigenvector, represents the third weight value corresponding to the third eigenvector, Represents the fourth weight value corresponding to the fourth eigenvector.

[0009] In a possible implementation manner of the first aspect, the first point cloud segmentation result is a pixel classification probability corresponding to the point cloud to be processed; The attribute vector is input into a preset deep learning model to obtain a first point cloud segmentation result, including: Obtaining a coordinate vector corresponding to the point cloud to be processed; Concatenate the coordinate vector corresponding to the point cloud to be processed and the attribute vector according to a preset input layer to obtain an input vector; Perform feature aggregation on the input vector according to a preset feature extraction layer to obtain a comprehensive vector; Performing feature transformation on the comprehensive vector according to a preset feature aggregation layer to obtain a local vector; Performing global feature determination on the local vector according to a preset feature determination layer to obtain a global vector; The global vector is feature mapped according to a preset output layer, and the pixel classification probability corresponding to the point cloud to be processed is output.

[0010] In a possible implementation manner of the first aspect, performing feature mapping on the global vector according to a preset output layer and outputting a pixel classification probability corresponding to the point cloud to be processed includes: The pixel classification probability corresponding to the point cloud to be processed is calculated according to the following formula: ; ; ; ; in, Represents the pixel classification probability corresponding to the point cloud to be processed, represents the classification function, represents the convolution operation, represents the global vector, Indicates The global pooling operation corresponding to the layer, Indicates The global vector corresponding to the layer, Indicates The local pooling operation corresponding to the layer, Indicates Layer local vectors, Indicates the The multi-layer perceptron corresponding to the layer, Indicates attribute vector, Indicates that according to A comprehensive vector.

[0011] In a second aspect, an embodiment of the present application provides a color point cloud segmentation device, comprising: An acquisition module, used for acquiring a point cloud to be processed and preprocessing the point cloud to be processed; An extraction module, used for extracting the color features and non-color features of the pre-processed point cloud to be processed, and performing feature fusion on the color features and the non-color features to obtain an attribute vector; The segmentation module is used to input the attribute vector into a preset deep learning model to obtain a first point cloud segmentation result.

[0012] In a third aspect, an embodiment of the present application provides a server, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method described in the first aspect when executing the computer program.

[0013] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method described in the first aspect above is implemented.

[0014] Compared with the prior art, the embodiments of the present invention have the following beneficial effects: In the embodiment of the present application, a point cloud to be processed is obtained and preprocessed; the color features and non-color features of the preprocessed point cloud to be processed are extracted, and the color features and non-color features are feature-fused to obtain an attribute vector; the attribute vector is input into a pre-set deep learning model to obtain a first point cloud segmentation result. It can be seen that the embodiment of the present application fully combines color features and non-color features to segment the point cloud during the point cloud segmentation process, thereby improving the accuracy of point cloud segmentation. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0016] Figure 1 is a schematic flow chart of a color point cloud method provided in an embodiment of the present application; Figure 2 It is a structural block diagram of a color point cloud segmentation device provided in an embodiment of the present application; Figure 3 It is a schematic diagram of the structure of the server provided in the embodiment of the present application. DETAILED DESCRIPTION

[0017] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0018] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or combinations thereof.

[0019] It should also be understood that the term “and / or” used in the specification and appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0020] As used in the specification and appended claims of this application, the term "if" can be interpreted as "when" or "uponce" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "uponce it is determined" or "in response to determining" or "uponce [described condition or event] is detected" or "in response to detecting [described condition or event]", depending on the context.

[0021] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0022] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0023] The technical solution of the embodiment of the present application is introduced below.

[0024] See also Figure 1 , is a schematic flow chart of a color point cloud method provided in an embodiment of the present application. As an example but not a limitation, the method can be applied to a server. The method includes the following steps: Step S101, obtaining a point cloud to be processed and preprocessing the point cloud to be processed.

[0025] It can be understood that the point cloud to be processed in the embodiment of the present application may be a color point cloud collected in advance by a scanning device (such as a depth camera).

[0026] Exemplarily, preprocessing the point cloud to be processed may be: preprocessing the point cloud to be processed through steps such as downsampling, noise removal and normalization to reduce data redundancy and noise interference.

[0027] Step S102 , extracting the color features and non-color features of the pre-processed point cloud to be processed, and performing feature fusion on the color features and the non-color features to obtain an attribute vector.

[0028] The non-color features include light features, geometric features, and position and posture information corresponding to a scanning device that collects the point cloud to be processed.

[0029] In a specific application, the color feature and the non-color feature are fused to obtain an attribute vector, including: Performing vectorization processing on the color feature to obtain a first feature vector; Performing vectorization processing on the light feature to obtain a second feature vector; Performing vectorization processing on the geometric features to obtain a third feature vector; Performing vector processing on the position and posture information corresponding to the scanning device that collects the point cloud to be processed to obtain a fourth eigenvector; Based on the first eigenvector, the second eigenvector, the third eigenvector and the fourth eigenvector, an attribute vector is obtained according to the following formula: ; in, represents the attribute vector, represents the first eigenvector, represents the second eigenvector, represents the third eigenvector, represents the fourth eigenvector, represents the first weight value corresponding to the first eigenvector, represents the second weight value corresponding to the second eigenvector, represents the third weight value corresponding to the third eigenvector, Represents the fourth weight value corresponding to the fourth eigenvector.

[0030] Exemplarily, vectorizing the color feature to obtain a first feature vector includes: encoding the color feature through an RGB color space to obtain the first feature vector.

[0031] Exemplarily, the light characteristics include light intensity and lighting direction, and the light characteristics are vectorized to obtain a second feature vector, including: obtaining the light intensity feature vector using the exposure value corresponding to the light sensor or the scanning device that collects the point cloud to be processed; using a spherical coordinate system or a Cartesian coordinate system as a basis, calculating the angle between each point cloud to be processed and the direction of the light source as a lighting direction feature vector; normalizing or standardizing the light intensity feature vector and the lighting direction feature vector to obtain a second feature vector.

[0032] Exemplarily, the geometric features include normals and curvatures, and the geometric features are vectorized to obtain a third eigenvector including: for each point cloud to be processed, a plane can be fitted within its neighborhood, and a normal vector is obtained by calculating based on the covariance matrix; for each point cloud to be processed, a surface can be fitted within its neighborhood, and a curvature vector is obtained by calculating the least squares method; the normal vector and the curvature vector are normalized or standardized to obtain the third eigenvector.

[0033] Exemplarily, the position and attitude information corresponding to the scanning device that collects the point cloud to be processed includes position information and attitude information, and the position and attitude information corresponding to the scanning device that collects the point cloud to be processed is vectorized to obtain a fourth eigenvector, including: during the scanning process, obtaining position information (such as three-dimensional coordinates expressed in absolute coordinates) and attitude information (such as a three-dimensional rotation matrix expressed in Euler angles) based on an inertial measurement unit (IMU), a global positioning system (GPS) or a vision-based tracking algorithm, and normalizing or standardizing the position information and attitude information to obtain a fourth eigenvector.

[0034] Step S103: input the attribute vector into a preset deep learning model to obtain a first point cloud segmentation result.

[0035] Among them, the first point cloud segmentation result is the pixel classification probability corresponding to the point cloud to be processed.

[0036] The preset deep learning model refers to a neural network structure composed of multiple layers of neurons, and the preset deep learning model includes a preset input layer, a preset feature extraction layer, a preset feature aggregation layer, a preset feature determination layer, and a preset output layer. The preset input layer is used to perform a splicing operation on the attribute vector and coordinate vector of each point cloud to be processed; the preset feature extraction layer is used to process and extract features from the input vector of each point cloud to be processed through a series of fully connected layers or convolutional layers; the preset feature aggregation layer is used to aggregate and combine the comprehensive vector of each point cloud to be processed through a local pooling operation to capture the relationship between local parts; the preset feature determination layer is used to perform global feature extraction on the local vector of each point cloud to be processed through a global pooling operation to obtain a global feature vector; the preset output layer is used to map the global feature vector to a specific pixel-level classification task through a classification function to achieve semantic segmentation of the point cloud and obtain the pixel classification probability corresponding to the point cloud to be processed.

[0037] In a specific application, the attribute vector is input into a preset deep learning model to obtain a first point cloud segmentation result, including: Obtaining a coordinate vector corresponding to the point cloud to be processed; Concatenate the coordinate vector and the attribute vector corresponding to the point cloud to be processed according to a preset input layer to obtain an input vector; Perform feature aggregation on the input vector according to a preset feature extraction layer to obtain a comprehensive vector; Performing feature transformation on the comprehensive vector according to a preset feature aggregation layer to obtain a local vector; Performing global feature determination on the local vector according to a preset feature determination layer to obtain a global vector; The global vector is feature mapped according to a preset output layer, and the pixel classification probability corresponding to the point cloud to be processed is output.

[0038] Exemplarily, feature mapping is performed on the global vector according to a preset output layer, and the pixel classification probability corresponding to the point cloud to be processed is output, including: The pixel classification probability corresponding to the point cloud to be processed is calculated according to the following formula: ; ; ; ; in, Represents the pixel classification probability corresponding to the point cloud to be processed, represents the classification function, represents the convolution operation, represents the global vector, Indicates The global pooling operation corresponding to the layer, Indicates The global vector corresponding to the layer, Indicates The local pooling operation corresponding to the layer, Indicates Layer local vectors, Indicates the The multi-layer perceptron corresponding to the layer, Indicates attribute vector, Indicates that according to The comprehensive vector may be obtained by aggregating the coordinate vectors and the attribute vectors of all the point clouds to be processed (such as maximum pooling, average pooling, etc.).

[0039] It can be understood that the coordinate vector corresponding to the point cloud to be processed represents the three-dimensional coordinates of the point cloud to be processed in the real world, and the coordinate vector corresponding to the point cloud to be processed can be obtained from a scanning device that collects the point cloud to be processed.

[0040] In the embodiment of the present application, by mapping point cloud data to pixel-level classification probabilities, each point can be fine-grainedly classified to achieve semantic segmentation of point cloud data, and different objects or categories in the point cloud can be more accurately identified and distinguished, thereby providing more detailed semantic understanding; combining global pooling and local pooling operations, the overall and local features of the point cloud data are simultaneously considered in the feature extraction process. The global pooling operation can extract the global features of the point cloud data and capture the overall information of the point cloud data, while the local pooling operation can capture the local relationship between points in the point cloud data and extract local features, which can better utilize the spatial structure information of the point cloud data and improve the feature characterization capability; the multi-layer perceptron layer can effectively fuse and learn local features and attribute features. Through the multiple hidden layers of the multi-layer perceptron, the features can be nonlinearly transformed and combined to further extract and express higher-level semantic information.

[0041] In a possible implementation, after inputting the attribute vector into a preset deep learning model to obtain a first point cloud segmentation result, the method further includes: Based on the first point cloud segmentation result, a region growing method is used to divide the point cloud to be processed into regions to obtain a second point cloud segmentation result.

[0042] The second point cloud segmentation result is a point cloud set merged into the same area.

[0043] It can be understood that the region growing method is used to merge adjacent point cloud sets to be processed into a larger point cloud region to obtain a more complete object segmentation result.

[0044] Exemplarily, based on the first point cloud segmentation result, the region growing method is used to divide the point cloud to be processed into regions to obtain a second point cloud segmentation result, including: taking the initial point cloud to be processed as the starting point, traversing each point cloud to be processed, screening out the point clouds to be processed whose pixel classification probability is greater than a preset threshold, checking the distance or similarity value of the attribute characteristics between the point cloud to be processed whose pixel classification probability is greater than the preset threshold and the neighboring points, merging the point clouds to be processed whose similarity values ​​are greater than the similarity threshold into the same region, and continuing to expand until they can no longer be merged, to obtain a set of point clouds merged into the same region.

[0045] In a possible implementation, based on the first point cloud segmentation result, a region growing method is used to perform region division on the point cloud to be processed, and after obtaining the second point cloud segmentation result, the method further includes: Based on the second point cloud segmentation result, construct a point cloud graph structure; wherein the point cloud graph structure includes the point clouds to be processed as nodes and the association relationships between the point clouds to be processed as edges; According to the point cloud graph structure, the second point cloud segmentation result is optimized based on a minimum spanning tree algorithm.

[0046] The minimum spanning tree algorithm may be a Prim algorithm or a Kruskal algorithm.

[0047] It can be understood that based on the segmentation results of region growing, a graph structure representation of the point cloud is constructed, and the relationship between the point cloud is converted into nodes and edges of the graph. The minimum spanning tree algorithm can be used to effectively manage the connections between regions, and the segmentation results can be optimized by minimizing mis-segmentation through connectivity and maximizing correct segmentation through connectivity.

[0048] In the embodiment of the present application, a point cloud to be processed is obtained and preprocessed; the color features and non-color features of the preprocessed point cloud to be processed are extracted, and the color features and non-color features are feature-fused to obtain an attribute vector; the attribute vector is input into a pre-set deep learning model to obtain a first point cloud segmentation result. It can be seen that the embodiment of the present application fully combines color features and non-color features to segment the point cloud during the point cloud segmentation process, thereby improving the accuracy of point cloud segmentation.

[0049] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0050] Corresponding to the color point cloud segmentation method described in the above embodiment, Figure 2 A structural block diagram of a color point cloud segmentation device provided in an embodiment of the present application is shown. For ease of explanation, only the parts related to the embodiment of the present application are shown.

[0051] Reference Figure 2 , the device comprises: An acquisition module 21 is used to acquire a point cloud to be processed and pre-process the point cloud to be processed; An extraction module 22 is used to extract the color features and non-color features of the pre-processed point cloud to be processed, and perform feature fusion on the color features and the non-color features to obtain an attribute vector; The first segmentation module 23 is used to input the attribute vector into a preset deep learning model to obtain a first point cloud segmentation result.

[0052] In a possible implementation, the device further includes: The second segmentation module is used to divide the point cloud to be processed into regions based on the first point cloud segmentation result by using a region growing method to obtain a second point cloud segmentation result. In a possible implementation, the device further includes: A construction module, configured to construct a point cloud graph structure based on the second point cloud segmentation result; wherein the point cloud graph structure includes the point clouds to be processed as nodes and the association relationships between the point clouds to be processed as edges; An optimization module is used to optimize the second point cloud segmentation result based on the point cloud graph structure and a minimum spanning tree algorithm.

[0053] In a possible implementation, the non-color features include light features, geometric features, and position and posture information corresponding to a scanning device that collects the point cloud to be processed.

[0054] Extraction modules, including: A first vectorization submodule, used for performing vectorization processing on the color feature to obtain a first feature vector; A second vectorization submodule, used for performing vectorization processing on the light feature to obtain a second feature vector; A third vectorization submodule performs vectorization processing on the geometric features to obtain a third feature vector; A fourth vectorization submodule performs vectorization processing on the position and posture information corresponding to the scanning device that collects the point cloud to be processed to obtain a fourth feature vector; The weighting submodule is used to obtain an attribute vector according to the following formula based on the first eigenvector, the second eigenvector, the third eigenvector and the fourth eigenvector: ; in, represents the attribute vector, represents the first eigenvector, represents the second eigenvector, represents the third eigenvector, represents the fourth eigenvector, represents the first weight value corresponding to the first eigenvector, represents the second weight value corresponding to the second eigenvector, represents the third weight value corresponding to the third eigenvector, Represents the fourth weight value corresponding to the fourth eigenvector.

[0055] In a possible implementation, the first point cloud segmentation result is a pixel classification probability corresponding to the point cloud to be processed; The first segmentation module comprises: An acquisition submodule, used to acquire a coordinate vector corresponding to the point cloud to be processed; An input submodule, used for concatenating the coordinate vector corresponding to the point cloud to be processed and the attribute vector according to a preset input layer to obtain an input vector; A convergence submodule, used for performing feature convergence on the input vector according to a preset feature extraction layer to obtain a comprehensive vector; A transformation submodule, used for performing feature transformation on the comprehensive vector according to a preset feature aggregation layer to obtain a local vector; A determination submodule, used for performing global feature determination on the local vector according to a preset feature determination layer to obtain a global vector; The classification submodule is used to perform feature mapping on the global vector according to a preset output layer, and output the pixel classification probability corresponding to the point cloud to be processed.

[0056] In a possible implementation, the classification submodule includes: The classification unit is used to calculate the pixel classification probability corresponding to the point cloud to be processed according to the following formula: ; ; ; ; in, Represents the pixel classification probability corresponding to the point cloud to be processed, represents the classification function, represents the convolution operation, represents the global vector, Indicates The global pooling operation corresponding to the layer, Indicates The global vector corresponding to the layer, Indicates The local pooling operation corresponding to the layer, Indicates Layer local vectors, Indicates the The multi-layer perceptron corresponding to the layer, Indicates attribute vector, Indicates that according to A comprehensive vector.

[0057] It should be noted that the information interaction, execution process, etc. between the above-mentioned devices / units are based on the same concept as the method embodiment of the present application. Their specific functions and technical effects can be found in the method embodiment part and will not be repeated here.

[0058] Figure 3This is a schematic diagram of the structure of the server provided in the embodiment of the present application. Figure 3 As shown, the server 3 of this embodiment includes: at least one processor 30, a memory 31, and a computer program 32 stored in the memory 31 and executable on the at least one processor 30, and when the processor 30 executes the computer program 32, the steps in any of the above-mentioned method embodiments are implemented.

[0059] The server 3 may be a computing device such as a cloud server. The server may include, but is not limited to, a processor 30 and a memory 31. Those skilled in the art will appreciate that Figure 3 It is only an example of server 3 and does not constitute a limitation on server 3. Server 3 may include more or fewer components than shown in the figure, or a combination of certain components, or different components, for example, it may also include input and output devices, network access devices, etc.

[0060] The processor 30 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0061] In some embodiments, the memory 31 may be an internal storage unit of the server 3, such as a hard disk or memory of the server 3. In other embodiments, the memory 31 may also be an external storage device of the server 3, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the server 3. Further, the memory 31 may also include both an internal storage unit of the server 3 and an external storage device. The memory 31 is used to store an operating system, an application program, a boot loader (BootLoader), data, and other programs, such as the program code of the computer program. The memory 31 may also be used to temporarily store data that has been output or is to be output.

[0062] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0063] An embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in the above-mentioned method embodiments can be implemented.

[0064] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, which can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor, the steps of the above-mentioned various method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may at least include: any entity or device capable of carrying the computer program code to the server, a recording medium, a computer memory, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), an electric carrier signal, a telecommunication signal, and a software distribution medium. For example, a USB flash drive, a mobile hard disk, a magnetic disk or an optical disk. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electric carrier signals and telecommunication signals.

[0065] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0066] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0067] In the embodiments provided in the present application, it should be understood that the disclosed devices / network equipment and methods can be implemented in other ways. For example, the device / network equipment embodiments described above are merely schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0068] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0069] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A color point cloud segmentation method, characterized in that: include: Acquire a point cloud to be processed and preprocess the point cloud to be processed; Extracting color features and non-color features of the preprocessed point cloud to be processed, and performing feature fusion on the color features and the non-color features to obtain an attribute vector; The attribute vector is input into a preset deep learning model to obtain a first point cloud segmentation result.

2. The color point cloud segmentation method according to claim 1, characterized in that: After inputting the attribute vector into a preset deep learning model to obtain a first point cloud segmentation result, the method further includes: Based on the first point cloud segmentation result, a region growing method is used to divide the point cloud to be processed into regions to obtain a second point cloud segmentation result.

3. The color point cloud segmentation method according to claim 2, characterized in that: Based on the first point cloud segmentation result, the region growing method is used to divide the point cloud to be processed into regions to obtain the second point cloud segmentation result, and the method further includes: Based on the second point cloud segmentation result, construct a point cloud graph structure; wherein the point cloud graph structure includes the point clouds to be processed as nodes and the association relationships between the point clouds to be processed as edges; According to the point cloud graph structure, the second point cloud segmentation result is optimized based on a minimum spanning tree algorithm.

4. The color point cloud segmentation method according to claim 1, characterized in that: The non-color features include light features, geometric features, and position and posture information corresponding to a scanning device that collects the point cloud to be processed.

5. The color point cloud segmentation method according to claim 4, characterized in that: The color feature and the non-color feature are subjected to feature fusion to obtain an attribute vector, including: Performing vectorization processing on the color feature to obtain a first feature vector; Performing vectorization processing on the light feature to obtain a second feature vector; Performing vectorization processing on the geometric features to obtain a third feature vector; Performing vector processing on the position and posture information corresponding to the scanning device that collects the point cloud to be processed to obtain a fourth eigenvector; Based on the first eigenvector, the second eigenvector, the third eigenvector and the fourth eigenvector, an attribute vector is obtained according to the following formula: ; in, represents the attribute vector, represents the first eigenvector, represents the second eigenvector, represents the third eigenvector, represents the fourth eigenvector, represents the first weight value corresponding to the first eigenvector, represents the second weight value corresponding to the second eigenvector, represents the third weight value corresponding to the third eigenvector, Represents the fourth weight value corresponding to the fourth eigenvector.

6. The color point cloud segmentation method according to claim 1, characterized in that: The first point cloud segmentation result is a pixel classification probability corresponding to the point cloud to be processed; The attribute vector is input into a preset deep learning model to obtain a first point cloud segmentation result, including: Obtaining a coordinate vector corresponding to the point cloud to be processed; Concatenate the coordinate vector corresponding to the point cloud to be processed and the attribute vector according to a preset input layer to obtain an input vector; Perform feature aggregation on the input vector according to a preset feature extraction layer to obtain a comprehensive vector; Performing feature transformation on the comprehensive vector according to a preset feature aggregation layer to obtain a local vector; Performing global feature determination on the local vector according to a preset feature determination layer to obtain a global vector; The global vector is feature mapped according to a preset output layer, and the pixel classification probability corresponding to the point cloud to be processed is output.

7. The color point cloud segmentation method as claimed in claim 6, characterized in that: Performing feature mapping on the global vector according to a preset output layer and outputting pixel classification probabilities corresponding to the point cloud to be processed includes: The pixel classification probability corresponding to the point cloud to be processed is calculated according to the following formula: ； ； ； ； in, Represents the pixel classification probability corresponding to the point cloud to be processed, represents the classification function, represents the convolution operation, represents the global vector, Indicates The global pooling operation corresponding to the layer, Indicates The global vector corresponding to the layer, Indicates The local pooling operation corresponding to the layer, Indicates Layer local vectors, Indicates the The multi-layer perceptron corresponding to the layer, Indicates attribute vector, Indicates that according to A comprehensive vector.

8. A color point cloud segmentation device, characterized in that: include: An acquisition module, used for acquiring a point cloud to be processed and preprocessing the point cloud to be processed; An extraction module, used for extracting the color features and non-color features of the pre-processed point cloud to be processed, and performing feature fusion on the color features and the non-color features to obtain an attribute vector; The first segmentation module is used to input the attribute vector into a preset deep learning model to obtain a first point cloud segmentation result.

9. A server comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images