Coal mine tunnel point cloud segmentation method based on interactive grouping linear attention mechanism

Abstract

The invention provides a coal mine roadway point cloud segmentation method based on an interactive grouping linear attention mechanism, and relates to the technical field of computer vision. Constructing a local coordinate system to describe three angle features between the normal vector of the query point and the normal vector of the reference point, and distance features between the query point and the reference point; carrying out high-dimensional feature coding to construct a high-dimensional feature vector; introducing an interactive grouping linear vector attention mechanism to divide channels of value vectors mapped by the high-dimensional feature vectors into a plurality of groups, and applying linear change in each group and interaction of high-dimensional features among the groups to obtain a target feature vector; and performing semantic segmentation by using a multi-layer perceptron to obtain a segmentation category of the point cloud data of the coal mine tunnel. Therefore, efficient and accurate segmentation of the point cloud data in the complex scene of the coal mine tunnel is realized without increasing equipment and computing power in the coal mine tunnel.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and particularly to a coal mine roadway point cloud segmentation method based on an interactive grouped linear attention mechanism. Background Art

[0002] The application of point cloud segmentation in coal mine development and management has gradually attracted attention. Coal mine point cloud segmentation methods can identify different targets inside the mine. In related technologies, point cloud segmentation methods for underground tunnel scenes often rely on rich attribute information, such as spatial distribution, color, normal vector, intensity, and density. However, there is currently a lack of coal mine roadway semantic datasets, and the coal mine roadway environment is complex and variable, containing objects and scenes with different shapes, sizes, and materials, such as dim light and messy pipelines, which makes it difficult to guarantee the accuracy of coal mine roadway point cloud segmentation methods. Summary of the Invention

[0003] The present invention aims to solve at least one of the technical problems in the related technologies to some extent.

[0004] To this end, the first object of the present invention is to propose a coal mine roadway point cloud segmentation method based on an interactive grouped linear attention mechanism, which can effectively achieve efficient and accurate segmentation of point cloud data in complex coal mine roadway scenes without increasing the equipment and computing power in the coal mine roadway.

[0005] The second object of the present invention is to propose a coal mine roadway point cloud segmentation device based on an interactive grouped linear attention mechanism.

[0006] The third object of the present invention is to propose an electronic device.

[0007] The fourth object of the present invention is to propose a non-transitory computer-readable storage medium storing computer instructions.

[0008] To achieve the above object, the first aspect embodiment of the present invention proposes a coal mine roadway point cloud segmentation method based on an interactive grouped linear attention mechanism, and the method includes: Construct a local coordinate system according to an arbitrarily selected query point and a reference point within the neighborhood of the query point in the coal mine roadway point cloud data to describe the four-element feature between the query point and the reference point, and the four-element feature includes three angular features between the normal vector of the query point and the normal vector of the reference point 、 、 , and the distance feature between the query point and the reference point ; For the four-element feature ( , , , )(0) Perform high-dimensional feature encoding to construct high-dimensional feature vectors; Introduce an interactive grouped linear vector attention mechanism to divide the channels that map the high-dimensional feature vectors into value vectors into multiple groups, apply linear transformations within each group, and perform interaction of high-dimensional features between groups through global attention weights to obtain target feature vectors with the interaction of linear transformations and high-dimensional features, where the linear transformation weights within each group are shared when applying linear transformations within each group; Use a multi-layer perceptron to perform semantic segmentation on the target feature vectors to obtain the segmentation categories after segmenting the coal mine roadway point cloud data, where the segmentation categories include the roof, floor, rib surface, ventilation duct, and pipeline.

[0009] To achieve the above object, the second aspect embodiment of the present invention proposes a coal mine roadway point cloud segmentation device based on an interactive grouped linear attention mechanism. The device includes: A construction module for constructing a local coordinate system based on a query point arbitrarily selected in the coal mine roadway point cloud data and a reference point within the neighborhood of the query point to describe the quaternion features between the query point and the reference point. The quaternion features include three angular features between the normal vector of the query point and the normal vector of the reference point 、 、 , and the distance feature between the query point and the reference point ; An encoding module for performing high-dimensional feature encoding on the quaternion features ( , , , ) to construct high-dimensional feature vectors; An interaction module for introducing an interactive grouped linear vector attention mechanism to divide the channels that map the high-dimensional feature vectors into value vectors into multiple groups, apply linear transformations within each group, and perform interaction of high-dimensional features between groups through global attention weights to obtain target feature vectors with the interaction of linear transformations and high-dimensional features, where the linear transformation weights within each group are shared when applying linear transformations within each group; A semantic segmentation module for using a multi-layer perceptron to perform semantic segmentation on the target feature vectors to obtain the segmentation categories after segmenting the coal mine roadway point cloud data, where the segmentation categories include the roof, floor, rib surface, ventilation duct, and pipeline.

[0010] The method, device, electronic device and storage medium for coal mine roadway point cloud segmentation based on an interactive grouped linear attention mechanism according to the embodiments of the present invention construct a local coordinate system to describe three angular features between the normal vector of a query point and the normal vector of a reference point within the neighborhood of the query point arbitrarily selected from the coal mine roadway point cloud data, and the distance feature between the query point and the reference point; and perform high-dimensional feature encoding to construct a high-dimensional feature vector; introduce an interactive grouped linear vector attention mechanism to divide the channels for mapping the high-dimensional feature vector into value vectors into multiple groups, and apply linear transformation within each group and interaction of high-dimensional features between groups to obtain a target feature vector; and use a multi-layer perceptron for semantic segmentation to obtain the segmentation categories of the coal mine roadway point cloud data. Thereby, without increasing the equipment and computing power in the coal mine roadway, the efficient and accurate segmentation of the point cloud data in the complex scene of the coal mine roadway can be effectively realized.

[0011] To achieve the above object, an embodiment of the third aspect of the present invention provides an electronic device, including: at least one processor; and a memory communicatively connected to the at least one processor; wherein, the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method described in the first aspect.

[0012] To achieve the above object, an embodiment of the fourth aspect of the present invention provides a non-transitory computer-readable storage medium storing computer instructions for causing a computer to execute the method described in the first aspect.

[0013] Additional aspects and advantages of the present invention will be given in part in the following description, become apparent in part from the following description, or be learned through the practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] The above and / or additional aspects and advantages of the present invention will become apparent and be readily understood from the following description of the embodiments in conjunction with the drawings, wherein: Figure 1 is a schematic flow chart of a method for coal mine roadway point cloud segmentation based on an interactive grouped linear attention mechanism provided by an embodiment of the present invention; Figure 2 is a schematic diagram of the segmentation categories of the coal mine roadway point cloud data provided by an embodiment of the present invention; Figure 3 is a technical flow chart of a method for coal mine roadway point cloud segmentation based on an interactive grouped linear attention mechanism provided by an embodiment of the present invention.

[0015] Figure 4 is a schematic structural diagram of a device for coal mine roadway point cloud segmentation based on an interactive grouped linear attention mechanism provided by an embodiment of the present invention. Detailed implementation manners

[0016] The embodiments of the present invention will be described in detail below. Examples of the embodiments are shown in the accompanying drawings, where the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below by referring to the accompanying drawings are exemplary and are intended to explain the present invention, and should not be construed as a limitation of the present invention.

[0017] It should be noted that, in the technical solution of the present invention, the acquisition, storage, use, processing, etc. of data all comply with the relevant regulations of relevant laws and regulations.

[0018] The coal mine roadway point cloud segmentation method and device based on the interactive grouped linear attention mechanism according to the embodiments of the present invention will be described below with reference to the accompanying drawings.

[0019] Figure 1 It is a schematic flow chart of a coal mine roadway point cloud segmentation method provided by an embodiment of the present invention.

[0020] As Figure 1 shown, the method includes the following steps: Step 101, construct a local coordinate system according to an arbitrarily selected query point and a reference point within the neighborhood of the query point in the coal mine roadway point cloud data to describe the four-element feature between the query point and the reference point. The four-element feature includes three angular features between the normal vector of the query point and the normal vector of the reference point , , , and the distance feature between the query point and the reference point .

[0021] In some possible implementation manners, constructing a local coordinate system according to an arbitrarily selected query point and a reference point within the neighborhood of the query point in the coal mine roadway point cloud data to describe the four-element feature between the query point and the reference point includes: arbitrarily selecting a query point in the coal mine roadway point cloud data and a reference point within the neighborhood of the query point , and according to and and their respective normal vectors and construct a local coordinate system; under the local coordinate system, calculate the relative deviation between the normal vector of and the normal vector of , that is, the three angular features between the normal vector of and the normal vector of ​ , , , and the distance feature between the query point and the reference point , so as to describe the query point and the reference point as the quaternary feature between them ( , , , ); Among them, ; .

[0022] Among them, the coal mine roadway point cloud data is collected from a section of the underground roadway. For example, there are a total of 27,077,467 points. The coal mine roadway point cloud data can be obtained using a scanner. To ensure the credibility of segmentation training and evaluation, the coal mine roadway point cloud data is divided into a training set and a test set, and the random segmentation ratio is 8:2, but not limited to this.

[0023] Step 102, perform high-dimensional feature encoding on the quaternary feature ( , , , ) to construct a high-dimensional feature vector.

[0024] In some possible implementation manners, performing high-dimensional feature encoding on the quaternary feature ( , , , ) to construct a high-dimensional feature vector includes: representing the quaternary features of all query points in the coal mine roadway point cloud data as , where i = 1, 2,..., C represents the number of query points, and R represents the real number space; selecting a random projection B that follows a normal distribution , ; where m represents the dimension of the high-dimensional feature vector, and s represents the hyperparameter that controls the range of random number values; the high-dimensional feature vector after high-dimensional feature encoding of the query point quaternary feature is represented as , where .

[0025] Step 103, introduce an interactive grouped linear vector attention mechanism to divide the channels of the high-dimensional feature vector into multiple groups, apply linear transformation within each group, and perform interaction of high-dimensional features between groups through global attention weights, so as to obtain an object feature vector of the interaction between linear transformation and high-dimensional features, where the linear transformation weights within each group are shared when applying linear transformation within each group.

[0026] In some possible embodiments, the channels that introduce the interactive grouped linear vector attention mechanism to map high-dimensional feature vectors into value vectors are divided into multiple groups, and linear transformations are applied within each group, and the high-dimensional features between groups are interacted through global attention weights to obtain the target feature vectors of the interaction between the linear transformation and the high-dimensional features, including: dividing the channels that map high-dimensional feature vectors into value vectors by the interactive grouped linear vector attention mechanism into multiple groups, and applying linear transformations within the groups to obtain the value vector matrix after the linear transformation; using a banded matrix to learn the context information within each group of channels and the information between channels, construct global attention weights, and perform the interaction of high-dimensional features between groups on the value vector matrix to obtain the target feature vectors; where involves parameters, k is different groups, and c is the number of channels of the value vector, .

[0027] Specifically, set the initial point cloud set formed by the high-dimensional feature vectors and the initial position information of the corresponding query points through the interactive grouped linear vector attention mechanism, and use the grid strategy to divide the space into several non-overlapping partitions, and then perform max pooling on the in different partitions, and perform average pooling on the initial position information for downsampling to obtain the point cloud set .

[0028] According to the point cloud set , calculate the vector attention value of the high-dimensional feature vectors and use it as the value vector, , where and respectively represent the high-dimensional feature vectors of the i-th query point and the j-th query point in the point cloud set , represents the weighted feature of the j-th query point, represents the Hadamard product, is a relational function (such as subtraction), is a sigmoid activation function, represents the position encoding, which is the initial position information .

[0029] Divide the number of channels c of the value vector into g groups, and the number of channels in each group is . Within the same (attention) group, apply linear transformations within each group to obtain the value vector matrix after the linear transformation , the channels of the value vector share the same linear transformation weights (attention weight parameters) from grouped attention. This way not only maintains the computational efficiency but also captures the interaction relationships between features within the group. , relating to parameters, where .

[0030] Using a banded matrix to learn the context information within each group of channels and the information between channels, construct global attention weights, and perform the interaction of high-dimensional features between groups on the value vector matrix to obtain the target feature vector.

[0031] where relates to parameters, k is different groups, and c is the number of channels of the value vector.

[0032] At this time, the weight calculation of the global attention weights only needs to consider the interaction between any one group and the surrounding k neighbor groups, and its global attention weight parameter is , where represents the set of k adjacent groups of the banded matrix that learns the context information within each group of channels and the information between channels.

[0033] In some other possible implementation manners, the coverage rate of the high-dimensional feature interaction between groups can also be adjusted through cross-validation. Specifically, the coverage rate can be set to 6, achieving the best performance of the segmentation. It not only reduces the computational complexity of the channels but also effectively captures the group interaction between local channels, ensuring both efficiency and effect.

[0034] Step 104, using a multi-layer perceptron to perform semantic segmentation on the target feature vector to obtain the segmentation categories after the segmentation of the coal mine roadway point cloud data, where the segmentation categories include the roof, the floor, the sidewall, the ventilation duct, and the pipeline.

[0035] In some possible implementation manners, using a multi-layer perceptron to perform semantic segmentation on the target feature vector to obtain the segmentation categories after the segmentation of the coal mine roadway point cloud data, where the segmentation categories include the roof, the floor, the sidewall, the ventilation duct, and the pipeline, includes: extracting the point features of the target feature vector stored during the downsampling of the multi-layer perceptron for semantic segmentation of the target feature vector, and mapping the point features to all points in the same set of settings to obtain the segmentation categories after the segmentation of the coal mine roadway point cloud data, where the segmentation categories include the roof (roof), the floor (floor), the sidewall (sidewall), the ventilation duct (air duct), and the pipeline (pipeline + cable).

[0036] Further, as Figure 2 shown, Figure 2 This is a schematic diagram of the segmentation categories of coal mine roadway point cloud data provided by an embodiment of the present invention. Figure 2 In it, a is the unsegmented coal mine roadway point cloud data, Figure 2 In it, b is the segmentation category, which are respectively the roof, the floor, the rib surface, the air duct (ventilation duct), and the pipe + cable (pipeline), realizing the precise segmentation of the coal mine roadway point cloud data.

[0037] The coal mine roadway point cloud segmentation method based on the interactive grouped linear attention mechanism according to the embodiment of the present invention constructs a local coordinate system to describe three angular features between the normal vector of a query point and the normal vector of a reference point within the neighborhood of the query point arbitrarily selected in the coal mine roadway point cloud data, and the distance feature between the query point and the reference point; and performs high-dimensional feature encoding to construct a high-dimensional feature vector; introduces an interactive grouped linear vector attention mechanism to divide the channels mapping the high-dimensional feature vector into value vectors into multiple groups, and applies a linear transformation within each group, as well as the interaction of high-dimensional features between groups, to obtain a target feature vector; uses a multi-layer perceptron for semantic segmentation to obtain the segmentation categories of the coal mine roadway point cloud data. Thus, without increasing the equipment and computing power in the coal mine roadway, it effectively realizes the efficient and precise segmentation of the point cloud data in the complex scene of the coal mine roadway.

[0038] In some embodiments, the present invention can use the overall accuracy (OA), the mean intersection over union (mIoU) between classes, and the mean accuracy within each class (mAcc) as evaluation metrics for the segmentation categories. In particular, mIoU presents the most reasonable evaluation for class-imbalanced datasets, so it can be used as the priority metric. The calculation methods of the evaluation metrics are as follows:

[0039] Where K is the total number of classes, N is the total number of points, respectively represent the predicted values of true positives, false positives, and false negatives.

[0040] Optionally, the coal mine roadway point cloud segmentation method based on the interactive grouped linear attention mechanism of the present invention can run on a high-performance computing cluster, using a graphics processing unit (48GB), a deep learning development environment, and various deep learning frameworks and parallel computing platforms and programming models supported by different programming languages. To mitigate overfitting, an optimizer (AdamW) is used, and the cross-entropy loss is used as the loss function.

[0041] Thus, through the coal mine roadway point cloud segmentation method based on the interactive grouped linear attention mechanism of the present invention, the coal mine roadway point cloud data set within the cycle is tested, and the overall accuracy (OA) reaches 99.1%, the mean intersection over union (mIoU) between unions is 97.5%, and the mean accuracy (mAcc) is 97.5%. In the specific five categories (roof, floor, rib surface, ventilation duct, and pipeline), the method of the present invention obtains the highest Intersection over Union (IoU) score.

[0042] In addition, for a better understanding of the present invention, in some possible implementation manners, the embodiment of the present invention further provides a technical flowchart of a coal mine roadway point cloud segmentation method based on the interactive grouped linear attention mechanism, as Figure 3 shown. The coal mine roadway point cloud segmentation method based on the interactive grouped linear attention mechanism of the present invention mainly constructs a coal mine roadway point cloud segmentation network model based on the interactive grouped linear attention mechanism. The core of the coal mine roadway point cloud segmentation network model is to construct an angle and distance feature encoding module and an interactive grouped linear attention module for representing local spatial relationships. Specifically, after the angle and distance feature encoding module obtains the input coal mine roadway point cloud data (such as Figure 2 part a in it), a local coordinate system is constructed to describe the quaternion features between an arbitrarily selected query point and its neighborhood points in the coal mine roadway point cloud data, and the calculated quaternion features are encoded into high-dimensional features to construct a high-dimensional feature vector of the query point; the interactive grouped linear attention module combines downsampling of the high-dimensional feature vector, point cloud partitioning, feature extraction, and interactive grouped linear attention calculation to output a target feature vector; using a multi-layer perceptron to perform semantic segmentation on the target feature vector to obtain the segmentation categories after the segmentation of the coal mine roadway point cloud data, that is, output the segmentation categories after the segmentation of the coal mine roadway point cloud data (such as Figure 2 part b in it). The coal mine roadway point cloud segmentation network model has robustness for segmenting complex shapes and noisy point clouds, and can provide strong support for subsequent digital modeling, safety monitoring, and intelligent inspection of coal mine roadway structures.

[0043] Among them, the downsampling and point cloud partitioning include: using a grid strategy to divide the space into several non-overlapping partitions, and then performing max pooling on the high-dimensional feature vectors of different partitions, and performing average pooling on the initial position information of the high-dimensional feature vector corresponding to the query point for downsampling to obtain a point cloud set; the feature extraction includes calculating the vector attention value of the high-dimensional feature vector as the value vector according to the point cloud set, dividing the number of channels c of the value vector into g groups, and applying a linear transformation within each group to obtain a matrix of value vectors after the linear transformation; the interactive grouped linear attention calculation includes: using a band matrix Learn the context information within each group of channels and the information between channels, construct global attention weights, and perform cross-group high-dimensional feature interaction on the value vector matrix to obtain the target feature vector.

[0044] In summary, the coal mine roadway point cloud segmentation method based on the interactive grouped linear attention mechanism of the present invention realizes the efficient and accurate segmentation of point cloud data in complex mine environments by comprehensively using local coordinate system construction, high-dimensional feature encoding, grouped linear vector attention mechanism, and multi-layer perceptron technical means. This method has robustness to complex shapes and noises and can provide strong support for subsequent coal mine geological analysis and resource assessment.

[0045] To implement the above embodiments, the present invention also proposes a coal mine roadway point cloud segmentation device based on the interactive grouped linear attention mechanism.

[0046] Figure 4 It is a schematic structural diagram of a coal mine roadway point cloud segmentation device based on the interactive grouped linear attention mechanism provided by an embodiment of the present invention.

[0047] As Figure 4 shown, the coal mine roadway point cloud segmentation device 40 based on the interactive grouped linear attention mechanism includes: a construction module 41, an encoding module 42, an interaction module 43, and a semantic segmentation module 44.

[0048] The construction module 41 is used to construct a local coordinate system according to a query point arbitrarily selected in the coal mine roadway point cloud data and a reference point within the neighborhood of the query point to describe the four-element features between the query point and the reference point, and the four-element features include three angular features between the normal vector of the query point and the normal vector of the reference point , , , as well as the distance feature between the query point and the reference point ; The encoding module 42 is used to perform high-dimensional feature encoding on the four-element features ( , , , ) to construct high-dimensional feature vectors; The interaction module 43 is used to introduce an interactive grouped linear vector attention mechanism to divide the channels that map the high-dimensional feature vectors into value vectors into multiple groups, apply linear transformations within each group, and perform cross-group high-dimensional feature interaction through global attention weights to obtain the target feature vector of the interaction between the linear transformation and the high-dimensional feature, where the linear transformation weights within each group are shared when applying linear transformations within each group; The semantic segmentation module 44 is used to perform semantic segmentation on the target feature vector by using a multi-layer perceptron to obtain the segmentation categories after the segmentation of the coal mine roadway point cloud data, where the segmentation categories include the roof, the floor, the rib surface, the ventilation duct, and the pipeline.

[0049] Further, in a possible implementation manner of the embodiment of the present invention, the construction module 41 is specifically used for: Arbitrarily select a query point in the coal mine roadway point cloud data and a reference point within the neighborhood of the query point , and according to the and and their respective corresponding normal vectors and construct a local coordinate system; Under the local coordinate system, calculate the normal vector of and the normal vector of the relative deviation between them, that is, the normal vector of and the normal vector of the three angular features , , , as well as the distance feature between the query point and the reference point, to be used as the four-element feature ( and the reference point ); , , , ); wherein, ; .

[0050] Further, in a possible implementation manner of the embodiment of the present invention, the encoding module 42 is specifically used for: Represent the four-element features of all query points in the coal mine roadway point cloud data as , where i = 1, 2,..., C represents the number of query points, and R represents the real number space; Select a random projection B that follows a normal distribution , ; where m represents the dimension of the high-dimensional feature vector, and s represents the hyperparameter that controls the range of random number values; The high-dimensional feature vector after high-dimensional feature encoding of the query point four-element feature is represented as , where, 。

[0051] Further, in a possible implementation manner of the embodiment of the present invention, the interaction module 43 is specifically configured to: Divide the channels that map the high-dimensional feature vector into value vectors through an interactive grouped linear vector attention mechanism into multiple groups, and apply a linear transformation within the groups to obtain a value vector matrix after linear transformation; Adopt a banded matrix Learn the context information within each group of channels and the information between channels, construct global attention weights, and perform cross-group high-dimensional feature interaction on the value vector matrix to obtain a target feature vector; Wherein, involves parameters, k is different groups, c is the number of channels of the value vector, 。

[0052] Further, in a possible implementation manner of the embodiment of the present invention, the device further includes: An adjustment module, configured to adjust the coverage rate of cross-group high-dimensional feature interaction through cross-validation.

[0053] Further, in a possible implementation manner of the embodiment of the present invention, the semantic segmentation module 44 is specifically configured to: Extract the point features of the target feature vector stored during the downsampling of the multi-layer perceptron for semantic segmentation of the target feature vector, and map the point features to all points in the same set subset to obtain the segmentation categories after the segmentation of the coal mine roadway point cloud data, where the segmentation categories include the roof, floor, rib surface, ventilation duct, and pipeline.

[0054] The coal mine roadway point cloud segmentation device based on the interactive grouped linear attention mechanism in the embodiment of the present invention constructs three angular features between the normal vector of the query point and the normal vector of the reference point, and the distance feature between the query point and the reference point according to a query point arbitrarily selected in the coal mine roadway point cloud data and a reference point within its neighborhood; and performs high-dimensional feature encoding to construct a high-dimensional feature vector; introduces an interactive grouped linear vector attention mechanism to divide the channels that map the high-dimensional feature vector into value vectors into multiple groups, and applies a linear transformation within each group, as well as cross-group high-dimensional feature interaction, to obtain a target feature vector; and uses a multi-layer perceptron for semantic segmentation to obtain the segmentation categories of the coal mine roadway point cloud data. Thus, without increasing the equipment and computing power in the coal mine roadway, it effectively realizes the efficient and accurate segmentation of the point cloud data in the complex scene of the coal mine roadway.

[0055] To implement the above embodiments, the present invention also proposes an electronic device, including: At least one processor; and a memory communicatively connected to the at least one processor; wherein, the memory stores instructions executable by the at least one processor, and when executed by the at least one processor, enable the at least one processor to perform the foregoing method.

[0056] To implement the above embodiments, the present invention further provides a non-transitory computer-readable storage medium storing computer instructions for causing a computer to perform the foregoing method.

[0057] In the description of this specification, the descriptions with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples", etc. mean that the specific features, structures, materials, or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms are not necessarily directed to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, without contradiction, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of different embodiments or examples.

[0058] In addition, the terms "first" and "second" are used for descriptive purposes only and cannot be construed as indicating or implying relative importance or implicitly indicating the quantity of the indicated technical features. Thus, the features defined with "first" and "second" may explicitly or implicitly include at least one of the features. In the description of the present invention, "a plurality" means at least two, such as two, three, etc., unless otherwise specifically defined.

[0059] Any process or method description in the flowchart or described in other ways herein may be understood to represent a module, segment, or portion of code including one or more executable instructions for implementing a customized logic function or process, and the scope of the preferred embodiments of the present invention includes additional implementations, where the functions may be performed in a substantially simultaneous manner or in a reverse order according to the functions involved, rather than in the order shown or discussed, which should be understood by those skilled in the art to which the embodiments of the present invention pertain.

[0060] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a definite sequence list of executable instructions for implementing logical functions, and can be specifically implemented in any computer-readable medium for use by an instruction execution system, apparatus, or device (such as a computer-based system, a system including a processor, or other systems that can fetch and execute instructions from the instruction execution system, apparatus, or device), or in combination with these instruction execution systems, apparatus, or devices. For the purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport a program for use by or in combination with an instruction execution system, apparatus, or device. More specific examples (non-exhaustive list) of computer-readable media include the following: an electrical connection portion having one or more wirings (electronic device), a portable computer diskette (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disc read-only memory (CDROM). Additionally, the computer-readable medium can even be paper or other suitable media on which the program can be printed, because the program can be obtained electronically, for example, by optically scanning the paper or other media, followed by editing, interpretation, or other suitable processing as necessary, and then storing it in a computer memory.

[0061] It should be understood that various parts of the present invention can be implemented by hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, any one or a combination of the following techniques well known in the art can be used: discrete logic circuits having logic gate circuits for implementing logical functions on data signals, application-specific integrated circuits having appropriate combinational logic gate circuits, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0062] Those of ordinary skill in the art of this technology can understand that all or part of the steps carried by the method of implementing the above embodiments can be completed by a program instructing relevant hardware, and the program can be stored in a computer-readable storage medium. When the program is executed, it includes one or a combination of the steps of the method embodiments.

[0063] In addition, each functional unit in various embodiments of the present invention may be integrated into a processing module, or each unit may exist physically alone, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. When the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.

[0064] The above-mentioned storage medium may be a read-only memory, a magnetic disk, an optical disc, etc. Although the embodiments of the present invention have been shown and described above, it can be understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those of ordinary skill in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of the present invention.

Claims

1. A coal mine tunnel point cloud segmentation method based on interactive grouping linear attention mechanism, characterized in that: The method comprises: According to the query point and a reference point in the neighborhood of the query point selected arbitrarily in the coal mine tunnel point cloud data, a local coordinate system is constructed to describe the four-element feature between the query point and the reference point, and the four-element feature includes three angle features between the normal vector of the query point and the normal vector of the reference point , , , and the distance feature between the query point and the reference point ; For the four-element features ( , , , ) Perform high-dimensional feature encoding to construct a high-dimensional feature vector; An interactive grouped linear vector attention mechanism is introduced to map the high-dimensional feature vector into channels of value vectors into multiple groups, and linear changes are applied within each group, and high-dimensional features are interacted between groups through global attention weights to obtain a target feature vector of the interaction of linear changes and high-dimensional features, wherein the linear transformation weights within each group are shared when linear changes are applied within each group; A multi-layer perceptron is used to perform semantic segmentation on the target feature vector to obtain the segmentation categories after the coal mine tunnel point cloud data is segmented. The segmentation categories include roof, bottom plate, side surface, ventilation duct and pipeline.

2. The method according to claim 1, characterized in that The method of constructing a local coordinate system to describe the quaternary features between the query point and the reference point according to a query point arbitrarily selected in the coal mine tunnel point cloud data and a reference point in the neighborhood of the query point includes: Select any query point in the coal mine tunnel point cloud data and a reference point in the neighborhood of the query point , and according to the and and their corresponding normal vectors and Construct a local coordinate system; In the local coordinate system, calculate The normal vector and The normal vector The relative deviation between The normal vector and The normal vector Three angle features between , , , and the distance feature between the query point and the reference point , as a description query point With reference point The four-element features between , , , ); in, ; 。 3. The method according to claim 1, characterized in that The four-element feature ( , , , ) performs high-dimensional feature encoding to construct a high-dimensional feature vector, including: The quaternion features of all query points in the coal mine tunnel point cloud data are expressed as , where i=1,2,..., C represents the number of query points, and R represents the real number space; Choose a normal distribution The random projection B, ; Where m represents the dimension of the high-dimensional feature vector, and s represents the hyperparameter that controls the range of random number values; The high-dimensional feature vector after high-dimensional feature encoding of the query point quadruple feature is expressed as ,in, .

4. The method according to claim 1, characterized in that The interactive grouped linear vector attention mechanism is introduced to map the high-dimensional feature vector into channels of value vectors into multiple groups, and linear changes are applied within each group, and the high-dimensional features between groups are interacted through global attention weights to obtain a target feature vector of the interaction of linear changes and high-dimensional features, including: The channels of the high-dimensional feature vector mapped to the value vector are divided into multiple groups through an interactive grouped linear vector attention mechanism, and a linear transformation is applied within the group to obtain a linearly changed value vector matrix; Banded Matrix Learn the context information within each group of channels and the information between channels, construct a global attention weight, and interact the high-dimensional features between groups of value vector matrices to obtain the target feature vector; in, Involved parameters, k is the different groups, c is the number of channels of the value vector, 。 5. The method according to claim 4, characterized in that The method further includes: adjusting the coverage of high-dimensional feature interactions between groups through cross-validation.

6. The method according to claim 1, characterized in that The target feature vector is semantically segmented by using a multi-layer perceptron to obtain segmentation categories after segmentation of the coal mine tunnel point cloud data, wherein the segmentation categories include roof, floor, side, ventilation duct and pipeline, including: The point features of the target feature vector stored when the multilayer perceptron performs semantic segmentation on the target feature vector are extracted, and the point features are mapped to all points in the same set subset to obtain the segmentation categories after the segmentation of the coal mine tunnel point cloud data, wherein the segmentation categories include the roof, bottom plate, side surface, ventilation duct and pipeline.

7. A coal mine tunnel point cloud segmentation device based on interactive grouping linear attention mechanism, characterized in that: The device comprises: A construction module is used to construct a local coordinate system based on a query point arbitrarily selected in the coal mine tunnel point cloud data and a reference point in the neighborhood of the query point to describe the four-element feature between the query point and the reference point, wherein the four-element feature includes three angle features between the normal vector of the query point and the normal vector of the reference point , , , and the distance feature between the query point and the reference point ; The encoding module is used to encode the four-element features ( , , , ) Perform high-dimensional feature encoding to construct a high-dimensional feature vector; An interaction module, used for introducing an interactive grouped linear vector attention mechanism to map the high-dimensional feature vector into a channel of a value vector into a plurality of groups, and applying a linear change within each group, and performing an interaction of high-dimensional features between groups through a global attention weight, so as to obtain a target feature vector of the interaction of the linear change and the high-dimensional features, wherein the linear transformation weights within each group are shared when applying a linear change within each group; The semantic segmentation module is used to perform semantic segmentation on the target feature vector using a multi-layer perceptron to obtain segmentation categories after segmentation of the coal mine tunnel point cloud data, where the segmentation categories include roof, bottom plate, side surface, ventilation ducts and pipelines.

8. The device according to claim 7, characterized in that The building blocks are specifically used for: Select any query point in the coal mine tunnel point cloud data and a reference point in the neighborhood of the query point , and according to the and and their corresponding normal vectors and Construct a local coordinate system; In the local coordinate system, calculate The normal vector and The normal vector The relative deviation between The normal vector and The normal vector Three angle features between , , , and the distance feature between the query point and the reference point , as a description query point With reference point The four-element features between , , , ); in, ; 。 9. The device according to claim 7, characterized in that The encoding module is specifically used for: The quaternion features of all query points in the coal mine tunnel point cloud data are expressed as , where i=1,2,..., C represents the number of query points, and R represents the real number space; Choose a normal distribution The random projection B, ; Where m represents the dimension of the high-dimensional feature vector, and s represents the hyperparameter that controls the range of random number values; The high-dimensional feature vector after high-dimensional feature encoding of the query point quadruple feature is expressed as ,in, .

10. The device according to claim 7, characterized in that The interaction module is specifically used for: The channels of the high-dimensional feature vector mapped to the value vector are divided into multiple groups through an interactive grouped linear vector attention mechanism, and a linear transformation is applied within the group to obtain a linearly changed value vector matrix; Banded Matrix Learn the context information within each group of channels and the information between channels, construct a global attention weight, and interact the high-dimensional features between groups of value vector matrices to obtain the target feature vector; in, Involved parameters, k is the different groups, c is the number of channels of the value vector, 。 11. The device according to claim 10, characterized in that The device further comprises: An adjustment module is used to adjust the coverage of high-dimensional feature interactions between groups through cross-validation.

12. The device according to claim 7, characterized in that The semantic segmentation module is specifically used for: The point features of the target feature vector stored when the multilayer perceptron performs semantic segmentation on the target feature vector are extracted, and the point features are mapped to all points in the same set subset to obtain the segmentation categories after the segmentation of the coal mine tunnel point cloud data, wherein the segmentation categories include the roof, bottom plate, side surface, ventilation duct and pipeline.

13. An electronic device, characterized in that: include: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 6.

14. A non-transitory computer-readable storage medium storing computer instructions, characterized in that: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-6.

Images