Urban scene point cloud automatic semantic segmentation method and system based on multi-channel attention mechanism, terminal and medium

Abstract

The invention discloses an urban scene point cloud automatic semantic segmentation method and system based on a multi-channel attention mechanism, a terminal and a medium, and relates to the technical field of artificial intelligence, and the method comprises the steps: inputting urban scene point cloud data, carrying out the semantic-driven point cloud adaptive resampling of the urban scene point cloud data, and carrying out the semantic-driven point cloud adaptive resampling of the urban scene point cloud data; obtaining resampled point cloud data; performing serialized feature coding on the resampled point cloud data, randomly selecting a mapping starting point during each feature aggregation, and converting the resampled point cloud data from a three-dimensional point cloud to a one-dimensional feature sequence; and performing feature learning on the one-dimensional feature sequence based on a multi-channel attention mechanism neural network, fusing the learned features based on a feature fusion module, and outputting semantic annotations to realize semantic segmentation. According to the method, the automatic semantic annotation practicability of the large-scale city scene point cloud is comprehensively improved from the aspects of improving semantic analysis performance, improving spatial information analysis capability, improving robustness and precision and the like.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a method, system, terminal and medium for automatic semantic segmentation of urban scene point clouds based on a multi-channel attention mechanism. Background Art

[0002] Semantic parsing of city-scale point clouds is a critical function for smart city applications. When dealing with large scene areas, scanned point clouds cover a wide range and contain semantic objects of complex types, with large scale and geometric differences. Furthermore, they are interfered with by low-quality data, making semantic parsing of scene data extremely difficult. To build a city digital twin and meet the various application requirements of smart city management, it is necessary to rely on remote sensing satellites, vehicle-mounted radars, drones, and other equipment to complete the acquisition of scene-oriented three-dimensional real-world data. Then, based on the acquired three-dimensional real-world data, spatial computing services are implemented to achieve accurate semantic perception and parsing of scene objects. However, current mainstream semantic parsing technologies suffer from problems such as low feature training efficiency, weak spatial scale perception, sensitivity to metadata quality, and loss of small-scale semantic objects.

[0003] Therefore, the prior art still has defects. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to address the above-mentioned defects of the prior art and provide a method, system, terminal and medium for automatic semantic segmentation of urban scene point clouds based on a multi-channel attention mechanism. The technical solution adopted by the present invention is as follows: In a first aspect, the present invention provides a method for automatic semantic segmentation of urban scene point clouds based on a multi-channel attention mechanism, wherein the method comprises: inputting urban scene point cloud data, and performing semantic-driven point cloud adaptive resampling on the urban scene point cloud data to obtain resampled point cloud data; The resampled point cloud data is serialized and feature encoded. At each feature aggregation, a mapping starting point is randomly selected to convert the resampled point cloud data from a three-dimensional point cloud into a one-dimensional feature sequence. The one-dimensional feature sequence is learned based on a multi-channel attention mechanism neural network, and the learned features are fused based on a feature fusion module to output semantic annotations to achieve semantic segmentation.

[0005] In one implementation, semantically driven point cloud adaptive resampling is performed on the urban scene point cloud data to obtain resampled point cloud data, including: Performing an initialization semantic feature analysis on the urban scene point cloud data to obtain a semantic feature display representation of the urban scene point cloud; Based on the semantic feature display representation of the urban scene point cloud, semantic-driven point cloud adaptive resampling is performed to enhance the geometric semantic features of the urban scene point cloud data to obtain resampled point cloud data.

[0006] In one implementation, performing initialization semantic feature analysis on the urban scene point cloud data to obtain a display representation of the urban scene point cloud semantic features includes: Elevation information embedding and an adaptive attention module are introduced into the point cloud processing neural network. Initial semantic parsing of urban scene point cloud data is performed based on the point cloud processing neural network to obtain a semantic feature display representation of the urban scene point cloud. The elevation information embedding is used to fuse the spatial information of the urban scene point cloud data, and the adaptive attention module is used to guide the variable-length feature encoding perception range.

[0007] In one implementation, the elevation information is embedded as:

[0008] in, The point cloud blocks in the urban scene point cloud data are divided into blocks. and are the points in the point cloud blocks, and Represents the index numbers of two points, 、 、 as well as They are the ReLU activation functions in the MLP multi-layer perceptron, is the convolutional layer, is the elevation information value, is the adaptive attention module, Represents the transposed matrix.

[0009] In one implementation, the adaptive attention module is represented as: , in, for point The corresponding mask threshold, , The coordinate information of the point.

[0010] In one implementation, semantically driven point cloud adaptive resampling is performed based on the semantic feature display representation of the urban scene point cloud to enhance the geometric semantic features of the urban scene point cloud data to obtain resampled point cloud data, including: Determining semantic feature annotations of the urban scene point cloud data based on a semantic feature display representation of the urban scene point cloud; Obtaining a point cloud set in which the semantic feature is labeled as a small-scale semantic object, randomly selecting a number of points from the point cloud set, and obtaining a median of the maximum value of elevation information in the Z-axis direction based on the selected points; The urban scene point cloud data is voxelized using the median as a scale, and each voxel is downsampled to obtain resampled point cloud data.

[0011] In one implementation, feature learning is performed on the one-dimensional feature sequence based on a multi-channel attention mechanism neural network, and the learned features are fused based on a feature fusion module to output semantic annotations to achieve semantic segmentation, including: Training multiple independent attention mechanism neural networks, using the multiple independent attention mechanism neural networks to perform feature learning on the one-dimensional feature sequence, and outputting a probability vector of the semantic annotation of each point cloud; For each point cloud, the feature fusion module is used to fuse the probability vectors of the semantic annotations output by each attention mechanism neural network, and the semantic annotation with the largest probability vector is output to achieve semantic segmentation.

[0012] In a second aspect, an embodiment of the present invention further provides a system for automatic semantic segmentation of urban scene point clouds based on a multi-channel attention mechanism, wherein the system is used to implement the steps of the method for automatic semantic segmentation of urban scene point clouds based on a multi-channel attention mechanism described in the above scheme, and the system includes: a point cloud adaptive resampling module, configured to input urban scene point cloud data, perform semantic-driven point cloud adaptive resampling on the urban scene point cloud data, and obtain resampled point cloud data; The serialization feature encoding module is used to perform serialization feature encoding on the resampled point cloud data. During each feature aggregation, the mapping starting point is randomly selected to convert the resampled point cloud data from a three-dimensional point cloud into a one-dimensional feature sequence. The multi-channel feature learning module is used to perform feature learning on the one-dimensional feature sequence based on a multi-channel attention mechanism neural network, and to fuse the learned features based on the feature fusion module, output semantic annotations, and realize semantic segmentation.

[0013] In a third aspect, an embodiment of the present invention further provides a terminal, wherein the terminal includes a memory, a processor, and an automatic semantic segmentation program for urban scene point clouds based on a multi-channel attention mechanism stored in the memory and runnable on the processor. When the processor executes the automatic semantic segmentation program for urban scene point clouds based on a multi-channel attention mechanism, the steps of the automatic semantic segmentation method for urban scene point clouds based on a multi-channel attention mechanism of any one of the above-mentioned schemes are implemented.

[0014] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium, wherein a program for automatic semantic segmentation of urban scene point clouds based on a multi-channel attention mechanism is stored on the computer-readable storage medium, and the program for automatic semantic segmentation of urban scene point clouds based on a multi-channel attention mechanism implements the steps of the method for automatic semantic segmentation of urban scene point clouds based on a multi-channel attention mechanism described in any one of the above-mentioned schemes on the computer-readable storage medium.

[0015] Beneficial effects: Compared with the prior art, the present invention provides an automatic semantic segmentation method for urban scene point clouds based on a multi-channel attention mechanism. The present invention first inputs urban scene point cloud data, performs semantic-driven point cloud adaptive resampling on the urban scene point cloud data, and obtains resampled point cloud data. Then, the resampled point cloud data is serialized and feature encoded. At each feature aggregation, a mapping starting point is randomly selected to convert the resampled point cloud data from a three-dimensional point cloud to a one-dimensional feature sequence. Finally, feature learning is performed on the one-dimensional feature sequence based on a multi-channel attention mechanism neural network, and the learned features are fused based on a feature fusion module to output semantic annotations to achieve semantic segmentation. The present invention comprehensively improves the practicality of automated semantic annotation of large-scale urban scene point clouds from several aspects, such as improving semantic parsing performance, improving spatial information parsing capabilities, and improving robustness and accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 A flowchart of a preferred embodiment of the method for automatic semantic segmentation of urban scene point clouds based on a multi-channel attention mechanism provided in an embodiment of the present invention.

[0017] Figure 2 Schematic diagram of the technical route of the method for automatic semantic segmentation of urban scene point clouds based on a multi-channel attention mechanism provided in an embodiment of the present invention.

[0018] Figure 3 A schematic diagram is provided for comparing the space-filling curve search method according to an embodiment of the present invention with the space-filling curve search method according to the prior art.

[0019] Figure 4 Schematic diagram of the architecture of a multi-channel attention mechanism neural network provided by an embodiment of the present invention.

[0020] Figure 5 This is a block diagram of the principle of the automatic semantic segmentation system for urban scene point clouds based on the multi-channel attention mechanism provided in an embodiment of the present invention.

[0021] Figure 6 This is a functional block diagram of a terminal provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0022] In order to make the purpose, technical solution and effect of the present invention clearer and more specific, the present invention is further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0023] The flowcharts shown in the accompanying drawings are for illustrative purposes only and do not necessarily include all contents, operations, or steps, nor must they be executed in the order described. For example, some operations or steps may be decomposed, combined, or partially merged, so the actual execution order may vary depending on the actual situation.

[0024] It should be understood that the terms used in this specification are only for the purpose of describing particular embodiments and are not intended to limit the present invention. As used in the specification and appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms unless the context clearly indicates otherwise. It should be understood that, to facilitate a clear description of the technical solutions of the embodiments of the present invention, in the embodiments of the present invention, terms such as "first" and "second" are used to distinguish between identical or similar items with substantially the same functions and effects. For example, the first control information and the second control information are merely used to distinguish different control information and do not limit their order. Those skilled in the art can understand that words such as "first" and "second" do not limit the quantity and execution order, and words such as "first" and "second" do not necessarily limit them to be different. It should be further understood that the term "and / or" used in the present description and the appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0025] Current mainstream semantic parsing technologies suffer from limitations, including low feature training efficiency, weak spatial scale perception, sensitivity to metadata quality, and loss of small-scale semantic objects. The data size of ultra-large-scale urban scene point clouds is orders of magnitude larger than that of general point cloud data. Traditional convolutional neural networks, graph networks, or attention mechanism architectures have significant processing efficiency issues for such large-scale point cloud data. Objects in urban scene point clouds possess significant spatial correlation information. However, spatial consistency between urban point clouds with different coverage areas is often not guaranteed. This requires a certain level of spatial perception for semantic parsing of urban scene point clouds, and current solutions do not meet these requirements. Limited by the performance of data scanning equipment and complex scene conditions, large-scale urban scene scanned point clouds often contain low-quality data, such as noise, non-uniform distribution, and missing areas. This low-quality data can affect the accuracy of feature analysis. Furthermore, due to the large scale variation of semantic objects in urban point clouds, establishing a universal semantic feature analysis method will inevitably miss features of some small-scale semantic objects (such as cars and trees), increasing the false recognition rate. Therefore, a new technical route is needed that has high feature encoding efficiency, spatial perception ability, strong robustness and compatibility with small-scale semantic objects.

[0026] Based on this, this embodiment provides a method for automatic semantic segmentation of urban scene point clouds based on a multi-channel attention mechanism. In specific applications, this embodiment first inputs urban scene point cloud data and performs semantically driven point cloud adaptive resampling on the urban scene point cloud data to obtain resampled point cloud data. The resampled point cloud data is then serialized and feature encoded. During each feature aggregation, a mapping starting point is randomly selected to convert the resampled point cloud data from a three-dimensional point cloud into a one-dimensional feature sequence. Finally, feature learning is performed on the one-dimensional feature sequence based on a multi-channel attention mechanism neural network. The learned features are then fused using a feature fusion module, and semantic annotations are output to achieve semantic segmentation.

[0027] The automatic semantic segmentation method of urban scene point cloud based on multi-channel attention mechanism of this embodiment can be applied to terminals, which can be intelligent terminal products such as computers, mobile phones or smart TVs. Figure 1 As shown, the automatic semantic segmentation method of urban scene point cloud based on the multi-channel attention mechanism of this embodiment includes the following steps: Step S100: input urban scene point cloud data, perform semantic-driven point cloud adaptive resampling on the urban scene point cloud data, and obtain resampled point cloud data.

[0028] Specifically, this embodiment first collects urban scene point cloud data, and the collection methods include but are not limited to remote sensing satellites, vehicle-mounted radars, drones and other equipment. Figure 2As shown in , after collecting a city scene point cloud, this embodiment can perform initialization semantic feature analysis on the city scene point cloud data to obtain a semantic feature display representation of the city scene point cloud. In this embodiment, the initialization semantic feature analysis step specifically includes two parts: initialization semantic feature analysis and semantic-driven adaptive resampling.

[0029] Initialized semantic feature analysis utilizes a point cloud processing neural network, such as the PTV3 network model, to achieve initial semantic parsing of the point cloud. The PTV3 network model is an advanced neural network for three-dimensional point cloud processing that excels in a variety of 3D perception tasks. Specifically, this embodiment incorporates elevation information embedding and an adaptive attention module into the point cloud processing neural network. This then performs initial semantic parsing of the urban scene point cloud data based on the point cloud processing neural network to obtain a semantic feature display representation of the urban scene point cloud. The elevation information embedding is intended to assist in the feature encoding stage, integrating the spatial information of the urban scene point cloud data, and the adaptive attention module is used to guide the variable-length feature encoding perception range.

[0030] The elevation information embedding is expressed as:

[0031] in, The point cloud blocks in the urban scene point cloud data are divided into blocks. and are the points in the point cloud blocks, and Represents the index numbers of two points, 、 、 as well as They are the ReLU activation functions in the MLP multi-layer perceptron, is the convolutional layer, is the elevation information value, and The spatial coordinates of are bound as the original input, represents the transposed matrix, The adaptive attention module introduces a learnable range-aware mask mechanism to guide a variable-length feature encoding perception range. Expressed as: , in, for point The corresponding mask threshold, , The coordinate information of the point.

[0032] Depend on Export The feature encoding perception range can adaptively control the perception domain of the feature encoding of the attention mechanism of the point cloud processing neural network. Its advantage is that it can effectively improve the perception ability of small-scale semantic objects, and also provide the subsequent serialized feature encoding steps with clear semantic annotations and more sufficient input data for small-scale object details, avoiding the encoding noise caused by uneven density of the original point cloud.

[0033] Through the above-mentioned initialization semantic feature analysis, a rough semantic feature display representation of the urban scene point cloud can be obtained. On this basis, this embodiment proposes semantic-driven adaptive resampling to achieve more accurate and semantically consistent distribution density optimization. Specifically, based on the semantic feature display representation of the urban scene point cloud, the semantic feature annotation of the urban scene point cloud data is determined. Then, a point cloud set with the semantic feature annotation as a small-scale semantic object, such as a car or a ship, is obtained, and a number of points are randomly selected from the point cloud set, such as 1,000 points, and the median of the maximum value of the elevation information in the Z-axis direction (such as the height of the top of the car and the ship) is obtained based on the selected points; the urban scene point cloud data is voxelized with the median as the scale, and each voxel is downsampled to obtain the resampled point cloud data. Specifically, this embodiment employs the AIVS (Approximate Intrinsic Voxel Structure) method to downsample each voxel. The sampling rate is differentiated based on the voxel's internal semantic features. Specifically, the voxel sampling rate for small-scale semantic objects is maintained at a 3:1 ratio compared to that for general semantic objects. This results in resampled point cloud data, ensuring that small-scale objects retain more detail points to enhance geometric semantic features. This semantically driven adaptive point cloud resampling method improves density optimization for point cloud data and enhances semantic object recognition capabilities in complex environments.

[0034] Step S200: Serialize feature encoding for the resampled point cloud data. During each feature aggregation, randomly select a mapping starting point to convert the resampled point cloud data from a three-dimensional point cloud into a one-dimensional feature sequence.

[0035] Generally, when it comes to feature encoding of large-scale urban point cloud data, the pain point is that feature aggregation requires a large amount of time. Taking attention feature aggregation as an example, it is necessary to aggregate each point layer by layer according to the KNN (k-nearest neighbor, a distance-based classification algorithm) nearest neighbor to the global feature, and then decode the feature back to each point, which is inefficient. The serialized feature encoding in this embodiment provides a simple and efficient method. Its principle is to search according to the space-filling curve and combine hash mapping to construct a one-dimensional feature sequence from the three-dimensional point cloud. In this way, when performing feature aggregation, there is no need to perform KNN nearest neighbor search. Instead, the corresponding field can be directly extracted on the one-dimensional feature sequence for feature aggregation, which significantly improves the efficiency of feature encoding and decoding.

[0036] Specifically, this embodiment proposes a new space-filling curve search method for urban scene point clouds based on the serialized feature encoding of the PTV3 network model, which is called the snake curve. Figure 3 As shown in , compared to the classic existing Z-Order (Z-shaped regular) curve and Hilbert curve, the search path of the snake curve increases randomness and reduces the neighborhood information redundancy generated by local symmetric search. At the same time, the snake curve avoids the large-span path problem of the Z-Order curve and inherits the regular search advantage of the path search of the Hilbert filling curve. This embodiment is based on the space-filling curve search method of the snake curve. The random selection strategy is sampled at each feature aggregation, and the mapping starting point is randomly selected to drive the mapping of the three-dimensional point cloud to the one-dimensional feature sequence, thereby improving the spatial coverage of the neighborhood points.

[0037] Step S300: perform feature learning on the one-dimensional feature sequence based on a multi-channel attention mechanism neural network, fuse the learned features based on a feature fusion module, output semantic annotations, and implement semantic segmentation.

[0038] For large-scale urban point cloud semantic annotation tasks, a difficult problem that cannot be ignored is to solve the impact of low-quality information such as noise, random displacement and data loss. Single-channel attention structure network models often cannot solve this problem. This embodiment proposes a multi-channel attention mechanism neural network architecture, the architecture is as follows Figure 4 As shown in the figure, by adding noise and random sampling to the original point cloud, providing data augmentation copies, and training multiple independent attention mechanism neural networks, compatibility with low-quality data is achieved.

[0039] Specifically, in this embodiment, the original urban scene point cloud data is used to train a single-channel attention mechanism neural network, and the single-channel attention mechanism neural network adopts the backbone network of the PTV3 network model. The attention mechanism neural networks of the additional two channels use the same backbone network. In this embodiment, by adding noise and random sampling to the original urban scene point cloud data, a data enhancement copy can be obtained, and the data enhancement copy is used as input data to train the attention mechanism neural networks of the additional two channels, thereby forming a multi-channel attention mechanism neural network. After the training is completed, the present embodiment uses multiple independent attention mechanism neural networks to perform feature learning on the obtained one-dimensional feature sequence, and outputs the probability vector of the semantic annotation of each point cloud. A feature fusion module is introduced into the multi-channel attention mechanism neural network of this embodiment, and the feature fusion module is equivalent to a maximum pooling module. Therefore, for each point cloud, the feature fusion module can be used to fuse the probability vectors of the semantic annotations output by each attention mechanism neural network, and output the semantic annotation with the largest probability vector, thereby achieving semantic segmentation.

[0040] The automatic semantic segmentation method of urban scene point cloud based on the multi-channel attention mechanism of the present invention includes at least the following advantages: first, the semantic-driven adaptive resampling of point cloud can improve the quality of density optimization of large-scale urban point cloud, and combined with elevation information, it can improve the utilization rate of spatial information and improve the accuracy of subsequent feature learning. In addition, by combining the adaptive attention module, the perception ability of small-scale objects within the scene is significantly improved. Serialized feature coding is enhanced by introducing a new space filling search method of the snake curve, combined with the Z-order curve and the Hilbert curve, which can effectively improve the neighborhood probability and improve the quality of feature coding while maintaining the high efficiency advantage of the original serialized feature coding. The multi-channel attention mechanism neural network architecture improves the semantic parsing ability of low-quality data in the scene, and combined with a feature fusion module, it can significantly improve the accuracy and robustness of semantic parsing, and comprehensively improve the practicality of automated semantic annotation of large-scale urban scene point clouds.

[0041] Based on the above embodiment, the present invention also provides a system for automatic semantic segmentation of urban scene point clouds based on a multi-channel attention mechanism, which is used to implement the steps of the method for automatic semantic segmentation of urban scene point clouds based on a multi-channel attention mechanism in the above method embodiment. Specifically, Figure 5As shown in , the system includes: a point cloud adaptive resampling module 10, a serialized feature encoding module 20 and a multi-channel feature learning module 30. Specifically, the point cloud adaptive resampling module 10 is used to input urban scene point cloud data, perform semantic-driven point cloud adaptive resampling on the urban scene point cloud data, and obtain resampled point cloud data. The serialized feature encoding module 20 is used to perform serialized feature encoding on the resampled point cloud data, and randomly select a mapping starting point at each feature aggregation to convert the resampled point cloud data from a three-dimensional point cloud to a one-dimensional feature sequence. The multi-channel feature learning module 30 is used to perform feature learning on the one-dimensional feature sequence based on a multi-channel attention mechanism neural network, and fuse the learned features based on the feature fusion module, output semantic annotations, and realize semantic segmentation.

[0042] The working principles of each module in the urban scene point cloud automatic semantic segmentation system based on the multi-channel attention mechanism in this embodiment are the same as the principles of each step in the above method embodiment, and will not be repeated here.

[0043] Each module in the aforementioned multi-channel attention-based automatic semantic segmentation system for urban scene point clouds can be implemented in whole or in part through software, hardware, or a combination thereof. Each module can be embedded in or independent of the processor in the terminal in hardware form, or stored in the terminal's memory in software form, allowing the processor to call and execute the corresponding operations of each module.

[0044] Based on the above embodiment, the present invention further provides a terminal, the principle block diagram of the terminal can be as follows: Figure 6 The terminal may include one or more processors 100 ( Figure 6 Only one is shown in the figure), memory 101, and computer program 102 stored in memory 101 and executable on one or more processors 100. For example, a program for automatic semantic segmentation of urban scene point clouds based on a multi-channel attention mechanism. When one or more processors 100 execute computer program 102, each step in an embodiment of a method for automatic semantic segmentation of urban scene point clouds based on a multi-channel attention mechanism can be implemented. Alternatively, when one or more processors 100 execute computer program 102, the functions of each module / unit in an embodiment of a system for automatic semantic segmentation of urban scene point clouds based on a multi-channel attention mechanism can be implemented, without limitation herein.

[0045] In one embodiment, the processor 100 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0046] In one embodiment, memory 101 may be an internal storage unit of an electronic device, such as a hard drive or memory. Memory 101 may also be an external storage device of the electronic device, such as a plug-in hard drive, smart media card (SMC), secure digital (SD) card, flash memory card, etc. Furthermore, memory 101 may include both an internal storage unit of the electronic device and an external storage device. Memory 101 is used to store computer programs and other programs and data required by the terminal. Memory 101 may also be used to temporarily store data that has been output or is about to be output.

[0047] Those skilled in the art will understand that Figure 6 The principle block diagram shown in the figure is only a block diagram of a partial structure related to the solution of the present invention, and does not constitute a limitation on the terminal to which the solution of the present invention is applied. The specific terminal may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0048] Those skilled in the art will understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, operation database or other media used in the embodiments provided by the present invention may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct RAM bus dynamic RAM (DRDRAM), and RAM bus dynamic RAM (RDRAM), etc.

[0049] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A method for automatic semantic segmentation of urban scene point clouds based on a multi-channel attention mechanism, characterized by: The method comprises: inputting urban scene point cloud data, and performing semantic-driven point cloud adaptive resampling on the urban scene point cloud data to obtain resampled point cloud data; The resampled point cloud data is serialized and feature encoded. At each feature aggregation, a mapping starting point is randomly selected to convert the resampled point cloud data from a three-dimensional point cloud into a one-dimensional feature sequence. The one-dimensional feature sequence is learned based on a multi-channel attention mechanism neural network, and the learned features are fused based on a feature fusion module to output semantic annotations to achieve semantic segmentation.

2. The method for automatic semantic segmentation of urban scene point clouds based on a multi-channel attention mechanism according to claim 1 is characterized in that: Performing semantic-driven point cloud adaptive resampling on the urban scene point cloud data to obtain resampled point cloud data, including: Performing an initialization semantic feature analysis on the urban scene point cloud data to obtain a semantic feature display representation of the urban scene point cloud; Based on the semantic feature display representation of the urban scene point cloud, semantic-driven point cloud adaptive resampling is performed to enhance the geometric semantic features of the urban scene point cloud data to obtain resampled point cloud data.

3. The method for automatic semantic segmentation of urban scene point clouds based on a multi-channel attention mechanism according to claim 2 is characterized in that: Performing an initialization semantic feature analysis on the urban scene point cloud data to obtain a semantic feature display representation of the urban scene point cloud, including: Elevation information embedding and an adaptive attention module are introduced into the point cloud processing neural network. Initial semantic parsing of urban scene point cloud data is performed based on the point cloud processing neural network to obtain a semantic feature display representation of the urban scene point cloud. The elevation information embedding is used to fuse the spatial information of the urban scene point cloud data, and the adaptive attention module is used to guide the variable-length feature encoding perception range.

4. The method for automatic semantic segmentation of urban scene point clouds based on a multi-channel attention mechanism according to claim 3 is characterized in that: The elevation information embedding is expressed as: in, The point cloud blocks in the urban scene point cloud data are divided into blocks. and are the points in the point cloud blocks, and Represents the index numbers of two points, 、 、 as well as They are the ReLU activation functions in the MLP multi-layer perceptron, is the convolutional layer, is the elevation information value, is the adaptive attention module, Represents the transposed matrix.

5. The method for automatic semantic segmentation of urban scene point clouds based on a multi-channel attention mechanism according to claim 4 is characterized in that: The adaptive attention module is expressed as: ， in, for point The corresponding mask threshold, , The coordinate information of the point.

6. The method for automatic semantic segmentation of urban scene point clouds based on a multi-channel attention mechanism according to claim 2, characterized in that: Based on the semantic feature display representation of the urban scene point cloud, semantic-driven point cloud adaptive resampling is performed to enhance the geometric semantic features of the urban scene point cloud data to obtain resampled point cloud data, including: Determining semantic feature annotations of the urban scene point cloud data based on a semantic feature display representation of the urban scene point cloud; Obtaining a point cloud set in which the semantic feature is labeled as a small-scale semantic object, randomly selecting a number of points from the point cloud set, and obtaining a median of the maximum value of elevation information in the Z-axis direction based on the selected points; The urban scene point cloud data is voxelized using the median as a scale, and each voxel is downsampled to obtain resampled point cloud data.

7. The method for automatic semantic segmentation of urban scene point clouds based on a multi-channel attention mechanism according to claim 1, characterized in that: The one-dimensional feature sequence is learned based on a multi-channel attention mechanism neural network, and the learned features are fused based on a feature fusion module to output semantic annotations to achieve semantic segmentation, including: Training multiple independent attention mechanism neural networks, using the multiple independent attention mechanism neural networks to perform feature learning on the one-dimensional feature sequence, and outputting a probability vector of the semantic annotation of each point cloud; For each point cloud, the feature fusion module is used to fuse the probability vectors of the semantic annotations output by each attention mechanism neural network, and the semantic annotation with the largest probability vector is output to achieve semantic segmentation.

8. An automatic semantic segmentation system for urban scene point clouds based on a multi-channel attention mechanism, characterized by: The system is used to implement the steps of the method for automatic semantic segmentation of urban scene point clouds based on a multi-channel attention mechanism according to any one of claims 1 to 7, and the system includes: a point cloud adaptive resampling module, configured to input urban scene point cloud data, perform semantic-driven point cloud adaptive resampling on the urban scene point cloud data, and obtain resampled point cloud data; The serialization feature encoding module is used to perform serialization feature encoding on the resampled point cloud data. During each feature aggregation, the mapping starting point is randomly selected to convert the resampled point cloud data from a three-dimensional point cloud into a one-dimensional feature sequence. The multi-channel feature learning module is used to perform feature learning on the one-dimensional feature sequence based on a multi-channel attention mechanism neural network, and to fuse the learned features based on the feature fusion module, output semantic annotations, and realize semantic segmentation.

9. A terminal, characterized in that: The terminal includes a memory, a processor, and an automatic semantic segmentation program for urban scene point clouds based on a multi-channel attention mechanism stored in the memory and runnable on the processor. When the processor executes the automatic semantic segmentation program for urban scene point clouds based on a multi-channel attention mechanism, the steps of the method for automatic semantic segmentation of urban scene point clouds based on a multi-channel attention mechanism as described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a program for automatic semantic segmentation of urban scene point clouds based on a multi-channel attention mechanism, and the program for automatic semantic segmentation of urban scene point clouds based on a multi-channel attention mechanism implements the steps of the method for automatic semantic segmentation of urban scene point clouds based on a multi-channel attention mechanism as described in any one of claims 1 to 7 on the computer-readable storage medium.

Images