A method, system, terminal, and medium for automatic semantic segmentation of urban scene point clouds based on a multi-channel attention mechanism.

Abstract

This invention discloses an automatic semantic segmentation method, system, terminal, and medium for urban scene point clouds based on a multi-channel attention mechanism, relating to the field of artificial intelligence technology. The method includes: inputting urban scene point cloud data; performing semantically driven adaptive resampling of the urban scene point cloud data to obtain resampled point cloud data; performing serialized feature encoding on the resampled point cloud data; randomly selecting a mapping starting point during each feature aggregation to convert the resampled point cloud data from a 3D point cloud into a 1D feature sequence; performing feature learning on the 1D feature sequence based on a multi-channel attention mechanism neural network; and fusing the learned features based on a feature fusion module to output semantic annotations, thereby achieving semantic segmentation. This invention comprehensively improves the practicality of automated semantic annotation for large-scale urban scene point clouds from several aspects, including improving semantic parsing performance, enhancing spatial information parsing capabilities, and improving robustness and accuracy.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to an automatic semantic segmentation method, system, terminal, and medium for point clouds in urban scenes based on a multi-channel attention mechanism. Background Technology

[0002] Semantic parsing of city-scale point clouds is a crucial function for smart city applications. When dealing with large scene areas, the scanned point cloud covers a wide area, containing complex semantic objects with significant scale and geometric differences. Furthermore, it is susceptible to interference from low-quality data, making semantic parsing of scene data extremely difficult. To construct a digital twin of the city and meet the various application needs of smart city management, it is necessary to rely on equipment such as remote sensing satellites, vehicle-mounted radar, and drones to acquire 3D reality data of the scene. Then, based on the acquired 3D reality data, spatial computing services are implemented to achieve accurate semantic perception and parsing of scene objects. However, current mainstream semantic parsing technologies suffer from low feature training efficiency, weak spatial scale perception capabilities, sensitivity to metadata quality, and loss of small-scale semantic objects.

[0003] Therefore, existing technologies still have shortcomings. Summary of the Invention

[0004] The technical problem to be solved by this invention is to provide an automatic semantic segmentation method, system, terminal, and medium for urban scene point clouds based on a multi-channel attention mechanism, addressing the aforementioned deficiencies of existing technologies. The technical solution adopted by this invention is as follows:

[0005] In a first aspect, the present invention provides an automatic semantic segmentation method for point clouds in urban scenes based on a multi-channel attention mechanism, wherein the method includes:

[0006] Input urban scene point cloud data, and perform semantic-driven adaptive resampling of the urban scene point cloud data to obtain resampled point cloud data;

[0007] The resampled point cloud data is serialized and encoded. During each feature aggregation, a mapping starting point is randomly selected to convert the resampled point cloud data from a three-dimensional point cloud into a one-dimensional feature sequence.

[0008] The one-dimensional feature sequence is learned by a neural network based on a multi-channel attention mechanism, and the learned features are fused by a feature fusion module to output semantic annotations and achieve semantic segmentation.

[0009] In one implementation, semantically driven adaptive resampling of the urban scene point cloud data is performed to obtain resampled point cloud data, including:

[0010] Initial semantic feature analysis is performed on the urban scene point cloud data to obtain a semantic feature display representation of the urban scene point cloud;

[0011] Based on the semantic feature display representation of urban scene point cloud, semantic-driven adaptive resampling of point cloud is performed to enhance the geometric semantic features of the urban scene point cloud data, resulting in resampled point cloud data.

[0012] In one implementation, the urban scene point cloud data is subjected to initial semantic feature analysis to obtain a semantic feature display representation of the urban scene point cloud, including:

[0013] Elevation information embedding and an adaptive attention module are introduced into a point cloud processing neural network. Based on the point cloud processing neural network, initial semantic parsing of urban scene point cloud data is performed to obtain a semantic feature display representation of urban scene point cloud. The elevation information embedding is used to fuse the spatial information of the urban scene point cloud data, and the adaptive attention module is used to guide the perception range of variable-length feature encoding.

[0014] In one implementation, the embedded elevation information is represented as follows:

[0015]

[0016] in, The point cloud data of the urban scene is divided into blocks. and These are points in a point cloud block. and Indicates the index number of two points. , , as well as These are the ReLU activation functions in the MLP (Multilayer Perceptron). It is a convolutional layer. Elevation information value, For adaptive attention modules, This represents the transpose of the matrix.

[0017] In one implementation, the adaptive attention module is represented as:

[0018] ,

[0019] in, For point The corresponding mask threshold, , This refers to the coordinates of a point.

[0020] In one implementation, based on the semantic feature display representation of urban scene point clouds, semantically driven adaptive resampling of the point clouds is performed to enhance the geometric semantic features of the urban scene point cloud data, resulting in resampled point cloud data, including:

[0021] Based on the semantic feature display representation of urban scene point cloud, the semantic feature annotation of the urban scene point cloud data is determined;

[0022] Obtain a point cloud set labeled with the semantic features as small-scale semantic objects, randomly select several points from the point cloud set, and obtain the median of the maximum elevation information in the Z-axis direction based on the selected points;

[0023] Using the median as a scale, the point cloud data of the urban scene is voxelized, and each voxel is downsampled to obtain resampled point cloud data.

[0024] In one implementation, a multi-channel attention mechanism neural network is used to learn features from the one-dimensional feature sequence, and a feature fusion module is used to fuse the learned features to output semantic annotations, thereby achieving semantic segmentation, including:

[0025] Train multiple independent attention mechanism neural networks, use multiple independent attention mechanism neural networks to perform feature learning on the one-dimensional feature sequence, and output the probability vector of semantic annotation for each point cloud.

[0026] For each point cloud, the probability vectors of semantic annotations output by each attention mechanism neural network are fused using the feature fusion module, and the semantic annotation with the largest probability vector is output to achieve semantic segmentation.

[0027] Secondly, embodiments of the present invention also provide an automatic semantic segmentation system for urban scene point clouds based on a multi-channel attention mechanism, wherein the system is used to implement the steps of the automatic semantic segmentation method for urban scene point clouds based on a multi-channel attention mechanism described in the above-mentioned scheme, and the system includes:

[0028] The point cloud adaptive resampling module is used to input urban scene point cloud data, perform semantic-driven adaptive resampling on the urban scene point cloud data, and obtain resampled point cloud data.

[0029] The serialization feature encoding module is used to perform serialization feature encoding on the resampled point cloud data. During each feature aggregation, a mapping starting point is randomly selected to convert the resampled point cloud data from a three-dimensional point cloud into a one-dimensional feature sequence.

[0030] The multi-channel feature learning module is used to learn features from the one-dimensional feature sequence based on a multi-channel attention mechanism neural network, and to fuse the learned features based on the feature fusion module to output semantic annotations and achieve semantic segmentation.

[0031] Thirdly, embodiments of the present invention also provide a terminal, wherein the terminal includes a memory, a processor, and an automatic semantic segmentation program for urban scene point clouds based on a multi-channel attention mechanism stored in the memory and executable on the processor. When the processor executes the automatic semantic segmentation program for urban scene point clouds based on a multi-channel attention mechanism, it implements the steps of the automatic semantic segmentation method for urban scene point clouds based on a multi-channel attention mechanism of any of the above-mentioned schemes.

[0032] Fourthly, embodiments of the present invention also provide a computer-readable storage medium, wherein the computer-readable storage medium stores an automatic semantic segmentation program for urban scene point clouds based on a multi-channel attention mechanism, the automatic semantic segmentation program for urban scene point clouds based on a multi-channel attention mechanism implementing the steps of the automatic semantic segmentation method for urban scene point clouds based on a multi-channel attention mechanism as described in any of the above schemes on the computer-readable storage medium.

[0033] Beneficial Effects: Compared with existing technologies, this invention provides an automatic semantic segmentation method for urban scene point clouds based on a multi-channel attention mechanism. First, urban scene point cloud data is input, and semantically driven adaptive resampling is performed to obtain resampled point cloud data. Then, serialized feature encoding is performed on the resampled point cloud data. During each feature aggregation, a mapping starting point is randomly selected, converting the resampled point cloud data from a 3D point cloud into a 1D feature sequence. Finally, feature learning is performed on the 1D feature sequence based on a multi-channel attention mechanism neural network, and the learned features are fused using a feature fusion module to output semantic annotations, achieving semantic segmentation. This invention comprehensively improves the practicality of automated semantic annotation for large-scale urban scene point clouds from several aspects, including enhancing semantic parsing performance, improving spatial information parsing capabilities, and improving robustness and accuracy. Attached Figure Description

[0034] Figure 1 This is a flowchart of a preferred embodiment of the automatic semantic segmentation method for urban scene point clouds based on a multi-channel attention mechanism provided in this invention.

[0035] Figure 2 This is a schematic diagram illustrating the technical route of the automatic semantic segmentation method for urban scene point clouds based on a multi-channel attention mechanism provided in this embodiment of the invention.

[0036] Figure 3This diagram illustrates a comparison between the space-filling curve search method provided in this embodiment of the invention and existing space-filling curve search methods.

[0037] Figure 4 This is a schematic diagram of the architecture of a multi-channel attention mechanism neural network provided in an embodiment of the present invention.

[0038] Figure 5 This is a block diagram illustrating the principle of an automatic semantic segmentation system for urban scene point clouds based on a multi-channel attention mechanism, as provided in an embodiment of the present invention.

[0039] Figure 6 A schematic diagram of a terminal provided in an embodiment of the present invention. Detailed Implementation

[0040] To make the objectives, technical solutions, and effects of this invention clearer and more explicit, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0041] The flowchart shown in the attached diagram is for illustrative purposes only and does not necessarily include all content, operations, or steps, nor does it require execution in the described order. For example, some operations or steps can be broken down, combined, or partially merged, so the actual execution order may change depending on the actual situation.

[0042] It should be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the invention. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0043] It should be understood that, in order to clearly describe the technical solutions of the embodiments of the present invention, the terms "first" and "second" are used in the embodiments of the present invention to distinguish identical or similar items with essentially the same function and effect. For example, the first control information and the second control information are only used to distinguish different control information and do not limit their order.

[0044] Those skilled in the art will understand that the words "first" and "second" do not limit the quantity or the order of execution, and that the words "first" and "second" do not necessarily imply that they are different.

[0045] It should also be understood that the term "and / or" as used in this specification and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0046] Current mainstream semantic parsing techniques suffer from several limitations, including low feature training efficiency, weak spatial scale awareness, sensitivity to metadata quality, and loss of small-scale semantic objects. The scale of ultra-large-scale urban scene point clouds differs by orders of magnitude from that of general point cloud data. Traditional convolutional neural networks, graph networks, or attention mechanisms exhibit significant shortcomings in processing such large-scale point cloud data. While urban scene point cloud objects possess significant spatial correlation information, spatial consistency is often not guaranteed for urban point clouds with varying coverage areas. This necessitates a certain level of spatial awareness for point cloud semantic parsing of urban scenes, a requirement that current solutions cannot fully meet. Limited by the performance of data scanning equipment and complex scene conditions, large-scale urban scene point cloud scanning carries low-quality data, such as noise, non-uniform distribution, and missing regions. This low-quality data affects the accuracy of feature analysis. Furthermore, due to the large scale differences among semantic objects in urban point clouds, establishing a universal semantic feature analysis method inevitably leads to the loss of features of some small-scale semantic objects (cars, trees, etc.), increasing the false recognition rate. Therefore, a new technological approach is needed that features high feature encoding efficiency, spatial awareness, robustness, and compatibility with small-scale semantic objects.

[0047] Based on this, this embodiment provides an automatic semantic segmentation method for urban scene point clouds based on a multi-channel attention mechanism. In specific application, this embodiment first inputs urban scene point cloud data and performs semantically driven adaptive resampling of the point cloud data to obtain resampled point cloud data. Then, the resampled point cloud data undergoes serialized feature encoding. During each feature aggregation, a mapping starting point is randomly selected to convert the resampled point cloud data from a 3D point cloud into a 1D feature sequence. Finally, a multi-channel attention mechanism neural network is used to learn features from the 1D feature sequence, and the learned features are fused using a feature fusion module to output semantic annotations, thus achieving semantic segmentation.

[0048] The automatic semantic segmentation method for urban scene point clouds based on a multi-channel attention mechanism in this embodiment can be applied to terminals, such as computers, mobile phones, or smart TVs. Specifically, for example... Figure 1 As shown, the automatic semantic segmentation method for urban scene point clouds based on a multi-channel attention mechanism in this embodiment includes the following steps:

[0049] Step S100: Input urban scene point cloud data, and perform semantic-driven adaptive resampling of the urban scene point cloud data to obtain resampled point cloud data.

[0050] Specifically, this embodiment first collects point cloud data of urban scenes, and the collection methods include, but are not limited to, remote sensing satellites, vehicle-mounted radar, and drones. Combined with... Figure 2 As shown, after acquiring the urban scene point cloud, this embodiment can perform initial semantic feature analysis on the urban scene point cloud data to obtain a semantic feature display representation of the urban scene point cloud. In this embodiment, the initial semantic feature analysis step specifically includes two parts: initial semantic feature analysis and semantic-driven adaptive resampling.

[0051] Initial semantic feature analysis utilizes a point cloud processing neural network to perform initial semantic parsing of the point cloud, such as the PTV3 network model. The PTV3 network model is an advanced neural network for 3D point cloud processing, exhibiting excellent performance in various 3D perception tasks. Specifically, this embodiment introduces elevation information embedding and an adaptive attention module into the point cloud processing neural network. Then, based on the point cloud processing neural network, initial semantic parsing of urban scene point cloud data is performed to obtain a semantic feature display representation of the urban scene point cloud. The elevation information embedding aims to assist the feature encoding stage by fusing the spatial information of the urban scene point cloud data, and the adaptive attention module is used to guide the variable-length feature encoding perception range.

[0052] The embedded elevation information is represented as follows:

[0053]

[0054] in, The point cloud data of the urban scene is divided into blocks. and These are points in a point cloud block. and Indicates the index number of two points. , , as well as These are the ReLU activation functions in the MLP (Multilayer Perceptron). It is a convolutional layer. Elevation information value, and Spatial coordinate binding is used as the original input. Represents the transpose matrix. The adaptive attention module introduces a learnable range-aware masking mechanism to guide a variable-length feature encoding perception range. Represented as:

[0055] ,

[0056] in, For point The corresponding mask threshold, , This refers to the coordinates of a point.

[0057] Depend on Export The feature encoding perception range can adaptively control the perceptual domain of the attention mechanism feature encoding in the point cloud processing neural network. Its advantage lies in its ability to effectively improve the perception of small-scale semantic objects, and also provides input data with clear semantic annotation and more sufficient preservation of small-scale object details for subsequent serialization feature encoding steps, avoiding encoding noise caused by uneven density in the original point cloud.

[0058] Through the above initial semantic feature analysis, a coarse semantic feature display representation of urban scene point cloud can be obtained. Based on this, this embodiment proposes semantic-driven adaptive resampling to achieve more accurate and semantically consistent distribution density optimization. Specifically, based on the semantic feature display representation of urban scene point cloud, the semantic feature annotation of the urban scene point cloud data is determined. Then, a point cloud set of semantic feature annotations labeled as small-scale semantic objects, such as cars or ships, is obtained. Several points are randomly selected from the point cloud set, for example, 1000 points, and the median of the maximum elevation information in the Z-axis direction (such as the height of the top of the car or ship) is obtained based on the selected points. Using the median as the scale, the urban scene point cloud data is voxelized, and each voxel is downsampled to obtain resampled point cloud data. Specifically, this embodiment employs the AIVS (Approximate intrinsic voxel structure) method to downsample each voxel. The sampling rate is differentiated according to the internal semantic features of the voxel; that is, the sampling rate ratio of small-scale semantic objects to that of general semantic objects is 3:1. This results in resampled point cloud data, ensuring that small-scale objects retain more detail points to enhance geometric semantic features. By adopting the semantic-driven adaptive resampling of point clouds in this embodiment, density optimization for point cloud data can be improved, enhancing the ability to recognize semantic objects in complex environments.

[0059] Step S200: Perform serialization feature encoding on the resampled point cloud data. During each feature aggregation, randomly select the mapping starting point to convert the resampled point cloud data from a three-dimensional point cloud into a one-dimensional feature sequence.

[0060] Typically, feature encoding for large-scale urban point cloud data is hampered by the significant time overhead of feature aggregation. Taking attention-based feature aggregation as an example, it requires layer-by-layer aggregation from each point using KNN (k-nearest neighbor, a distance-based classification algorithm) to reach global features, followed by feature decoding back to each point – a process that is inefficient. The serialized feature encoding method in this embodiment offers a simple and efficient approach. Its principle is to construct a one-dimensional feature sequence from the 3D point cloud by searching the spatial filling curve and combining it with hash mapping. This eliminates the need for KNN nearest neighbor search during feature aggregation; instead, the corresponding neighborhood can be directly extracted from the one-dimensional feature sequence for feature aggregation, significantly improving the efficiency of feature encoding and decoding.

[0061] Specifically, this embodiment proposes a novel spatial filling curve search method for point clouds in urban scenes, known as the "snake curve," based on the serialized feature encoding of the PTV3 network model. For example... Figure 3 As shown, compared to the classic Z-Order curves and Hilbert curves, the snake curve increases the randomness of its search path, reducing the redundancy of neighborhood information caused by local symmetric searches. Simultaneously, the snake curve avoids the large-span path problem of the Z-Order curve and inherits the rule-based search advantages of the Hilbert filling curve. This embodiment, based on the snake curve spatial filling curve search method, samples a random selection strategy during each feature aggregation, randomly selecting the mapping starting point to drive the mapping from the 3D point cloud to the 1D feature sequence, thereby improving the spatial coverage of neighborhood points.

[0062] Step S300: Based on the multi-channel attention mechanism neural network, feature learning is performed on the one-dimensional feature sequence, and the learned features are fused based on the feature fusion module to output semantic annotations and achieve semantic segmentation.

[0063] For large-scale urban point cloud semantic annotation tasks, a significant challenge is addressing the impact of low-quality information such as noise, random displacement, and data loss. Single-channel attention-based network models often fail to solve this problem. This embodiment proposes a multi-channel attention mechanism neural network architecture, as shown below. Figure 4 As shown, by adding noise and random sampling to the original point cloud, a data augmentation copy is provided, and multiple independent attention mechanism neural networks are trained, thereby achieving compatibility with low-quality data.

[0064] Specifically, in this embodiment, the original urban scene point cloud data is used to train a single-channel attention mechanism neural network, which adopts the backbone network of the PTV3 network model. The additional two channels of attention mechanism neural networks use the same backbone network. This embodiment obtains a data-enhanced copy by adding noise and random sampling to the original urban scene point cloud data. This data-enhanced copy is used as input data to train the additional two channels of attention mechanism neural networks, thus forming a multi-channel attention mechanism neural network. After training, this embodiment uses multiple independent attention mechanism neural networks to perform feature learning on the obtained one-dimensional feature sequence, outputting a probability vector of semantic annotation for each point cloud. This embodiment introduces a feature fusion module into the multi-channel attention mechanism neural network, which is equivalent to a max-pooling module. Therefore, for each point cloud, the feature fusion module can fuse the probability vectors of semantic annotations output by each attention mechanism neural network, and output the semantic annotation with the largest probability vector, thereby achieving semantic segmentation.

[0065] The automatic semantic segmentation method for urban scene point clouds based on a multi-channel attention mechanism of the present invention has at least the following advantages: First, semantically driven adaptive resampling of point clouds can improve the quality of density optimization for large-scale urban point clouds. Combined with elevation information, it can improve the utilization rate of spatial information and improve the accuracy of subsequent feature learning. In addition, by combining an adaptive attention module, the perception ability of small-scale objects within the scene is significantly improved. Enhanced serialized feature encoding, by introducing a novel snake curve space-filling search method, combined with Z-order curves and Hilbert curves, can effectively improve neighborhood probability, improving the quality of feature encoding while maintaining the high efficiency advantage of the original serialized feature encoding. The multi-channel attention mechanism neural network architecture improves the semantic parsing ability of low-quality data within the scene, and combined with a feature fusion module, it can significantly improve the accuracy and robustness of semantic parsing, comprehensively improving the practicality of automated semantic annotation for large-scale urban scene point clouds.

[0066] Based on the above embodiments, the present invention also provides an automatic semantic segmentation system for urban scene point clouds based on a multi-channel attention mechanism. This system is used to implement the steps of the automatic semantic segmentation method for urban scene point clouds based on a multi-channel attention mechanism in the above method embodiments. Specifically, as... Figure 5As shown, the system includes: a point cloud adaptive resampling module 10, a serialized feature encoding module 20, and a multi-channel feature learning module 30. Specifically, the point cloud adaptive resampling module 10 is used to input urban scene point cloud data, perform semantically driven adaptive resampling on the urban scene point cloud data, and obtain resampled point cloud data. The serialized feature encoding module 20 is used to perform serialized feature encoding on the resampled point cloud data. During each feature aggregation, a mapping starting point is randomly selected to convert the resampled point cloud data from a three-dimensional point cloud into a one-dimensional feature sequence. The multi-channel feature learning module 30 is used to learn features from the one-dimensional feature sequence based on a multi-channel attention mechanism neural network, and fuse the learned features based on a feature fusion module to output semantic annotations and achieve semantic segmentation.

[0067] The working principle of each module in the automatic semantic segmentation system for urban scene point clouds based on the multi-channel attention mechanism in this embodiment is the same as that of each step in the above method embodiment, and will not be repeated here.

[0068] The modules in the aforementioned automatic semantic segmentation system for urban scene point clouds based on a multi-channel attention mechanism can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of the terminal in hardware form or independent of it, or stored in the terminal's memory in software form, so that the processor can call and execute the corresponding operations of each module.

[0069] Based on the above embodiments, the present invention also provides a terminal, the principle block diagram of which can be as follows: Figure 6 As shown. The terminal may include one or more processors 100 ( Figure 6 (Only one is shown in the image), memory 101, and computer program 102 stored in memory 101 and executable on one or more processors 100. For example, an automatic semantic segmentation program for urban scene point clouds based on a multi-channel attention mechanism. When one or more processors 100 execute computer program 102, they can implement the various steps in the embodiments of the automatic semantic segmentation method for urban scene point clouds based on a multi-channel attention mechanism. Alternatively, when one or more processors 100 execute computer program 102, they can implement the functions of various modules / units in the embodiments of the automatic semantic segmentation system for urban scene point clouds based on a multi-channel attention mechanism, which is not limited here.

[0070] In one embodiment, the processor 100 may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.

[0071] In one embodiment, memory 101 may be an internal storage unit of an electronic device, such as a hard drive or RAM. Memory 101 may also be an external storage device of the electronic device, such as a plug-in hard drive, smart media card (SMC), secure digital (SD) card, flash card, etc. Furthermore, memory 101 may include both internal and external storage units. Memory 101 is used to store computer programs and other programs and data required by the terminal. Memory 101 can also be used to temporarily store data that has been output or will be output.

[0072] Those skilled in the art will understand that Figure 6 The block diagram shown is merely a partial structural diagram related to the present invention and does not constitute a limitation on the terminal to which the present invention is applied. A specific terminal may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0073] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, operational databases, or other media used in the embodiments provided by this invention can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual operating data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0074] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An automatic semantic segmentation method for urban scene point clouds based on a multi-channel attention mechanism, characterized in that, The method includes: Input urban scene point cloud data, and perform semantic-driven adaptive resampling of the urban scene point cloud data to obtain resampled point cloud data; The resampled point cloud data is serialized and encoded. During each feature aggregation, a mapping starting point is randomly selected to convert the resampled point cloud data from a three-dimensional point cloud into a one-dimensional feature sequence. The one-dimensional feature sequence is learned by a neural network based on a multi-channel attention mechanism, and the learned features are fused by a feature fusion module to output semantic annotations and achieve semantic segmentation. Semantic-driven adaptive resampling of the urban scene point cloud data is performed to obtain resampled point cloud data, including: Initial semantic feature analysis is performed on the urban scene point cloud data to obtain a semantic feature display representation of the urban scene point cloud; Based on the semantic feature display representation of urban scene point cloud, semantic-driven adaptive resampling of point cloud is performed to enhance the geometric semantic features of the urban scene point cloud data, resulting in resampled point cloud data. Initial semantic feature analysis is performed on the urban scene point cloud data to obtain a semantic feature display representation of the urban scene point cloud, including: Elevation information embedding and an adaptive attention module are introduced into a point cloud processing neural network. Based on the point cloud processing neural network, initial semantic parsing of urban scene point cloud data is performed to obtain a semantic feature display representation of urban scene point cloud. The elevation information embedding is used to fuse the spatial information of the urban scene point cloud data, and the adaptive attention module is used to guide the perception range of variable-length feature encoding. The embedded elevation information is represented as follows: in, The point cloud data of the urban scene is divided into blocks. and These are points in a point cloud block. and Indicates the index number of two points. , , as well as These are the ReLU activation functions in the MLP (Multilayer Perceptron). It is a convolutional layer. Elevation information value, For adaptive attention modules, Represents the transpose matrix; The adaptive attention module is represented as follows: ， in, For point The corresponding mask threshold, , This refers to the coordinate information of a point.

2. The automatic semantic segmentation method for urban scene point clouds based on a multi-channel attention mechanism according to claim 1, characterized in that, Based on the semantic feature display representation of urban scene point clouds, semantic-driven adaptive resampling of point clouds is performed to enhance the geometric semantic features of the urban scene point cloud data, resulting in resampled point cloud data, including: Based on the semantic feature display representation of urban scene point cloud, the semantic feature annotation of the urban scene point cloud data is determined; Obtain a point cloud set labeled with the semantic features as small-scale semantic objects, randomly select several points from the point cloud set, and obtain the median of the maximum elevation information in the Z-axis direction based on the selected points; Using the median as a scale, the point cloud data of the urban scene is voxelized, and each voxel is downsampled to obtain resampled point cloud data.

3. The automatic semantic segmentation method for urban scene point clouds based on a multi-channel attention mechanism according to claim 1, characterized in that, The one-dimensional feature sequence is learned using a multi-channel attention mechanism neural network, and the learned features are fused using a feature fusion module to output semantic annotations, thereby achieving semantic segmentation, including: Train multiple independent attention mechanism neural networks, use multiple independent attention mechanism neural networks to perform feature learning on the one-dimensional feature sequence, and output the probability vector of semantic annotation for each point cloud. For each point cloud, the probability vectors of semantic annotations output by each attention mechanism neural network are fused using the feature fusion module, and the semantic annotation with the largest probability vector is output to achieve semantic segmentation.

4. An automatic semantic segmentation system for urban scene point clouds based on a multi-channel attention mechanism, characterized in that, The system is used to implement the steps of the automatic semantic segmentation method for urban scene point clouds based on a multi-channel attention mechanism as described in any one of claims 1-3, the system comprising: The point cloud adaptive resampling module is used to input urban scene point cloud data, perform semantic-driven adaptive resampling on the urban scene point cloud data, and obtain resampled point cloud data. The serialization feature encoding module is used to perform serialization feature encoding on the resampled point cloud data. During each feature aggregation, a mapping starting point is randomly selected to convert the resampled point cloud data from a three-dimensional point cloud into a one-dimensional feature sequence. The multi-channel feature learning module is used to learn features from the one-dimensional feature sequence based on a multi-channel attention mechanism neural network, and to fuse the learned features based on the feature fusion module to output semantic annotations and achieve semantic segmentation.

5. A terminal, characterized in that, The terminal includes a memory, a processor, and an automatic semantic segmentation program for urban scene point clouds based on a multi-channel attention mechanism stored in the memory and executable on the processor. When the processor executes the automatic semantic segmentation program for urban scene point clouds based on a multi-channel attention mechanism, it implements the steps of the automatic semantic segmentation method for urban scene point clouds based on a multi-channel attention mechanism as described in any one of claims 1-3.

6. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores an automatic semantic segmentation program for urban scene point clouds based on a multi-channel attention mechanism. The automatic semantic segmentation program for urban scene point clouds based on a multi-channel attention mechanism implements the steps of the automatic semantic segmentation method for urban scene point clouds based on a multi-channel attention mechanism as described in any one of claims 1-3 on the computer-readable storage medium.

Images