Residue soil point cloud segmentation processing method, system and platform based on point cloud convolution

Abstract

The invention discloses a muck point cloud segmentation processing method, system and platform based on point cloud convolution, and the method comprises the steps: generating and obtaining first data and second data corresponding to the point cloud segmentation processing of muck, creating a first geometric relation between the first data and the second data, and generating corresponding third data based on the first corresponding relation; according to the third data, dividing the input feature data into a plurality of attention heads in combination with a multi-head self-attention mechanism, dynamically adjusting and processing weight data corresponding to the feature data, and generating and integrating fourth data corresponding to the third data; based on the fourth data, performing segmentation processing on point cloud data corresponding to the residue soil, and generating fifth data corresponding to the first data; according to the muck point cloud segmentation method and system, local convolution feature extraction, a global multi-head self-attention mechanism and an effective feature up-sampling strategy are innovatively combined, the point cloud data processing capacity is remarkably improved, and especially excellent performance is shown in muck point cloud segmentation tasks.

Description

Technical Field

[0001] The present invention belongs to the technical field of soil point cloud segmentation processing, and specifically relates to a soil point cloud segmentation processing method, system and platform based on point cloud convolution. Background Art

[0002] The construction of shield tunnels plays a vital role in modern urban infrastructure, with demand growing in particular for underground projects such as subways and sewers. As highly efficient underground excavation equipment, shield machines generate large amounts of excavated soil during excavation, which is transported to the surface via a conveyor belt system. Effectively monitoring the excavated soil during shield tunneling, thereby improving construction efficiency and safety, has become a pressing issue in the engineering field. Point cloud technology offers a potential solution to this problem. By acquiring and processing three-dimensional excavated soil data through sensors, precise measurement of the soil volume and status monitoring can be achieved. However, due to the complex accumulation of excavated soil and the harsh construction environment, existing point cloud processing technologies still face numerous challenges. Accurate extraction and segmentation of excavated soil point clouds have become an important research direction for intelligent construction.

[0003] During shield tunneling, the excavated soil transported by the conveyor belt forms complex three-dimensional geometric structures during transportation and accumulation. The shape of these excavated soil is not only affected by the geological environment but also changes dynamically as tunneling progresses. Traditional methods of excavation monitoring, such as manual observation or simple volume measurement, are unable to accurately describe the accumulation state and volume changes of the excavated soil. They are also inefficient, prone to human error, and unable to meet the needs of modern intelligent construction. Collecting three-dimensional excavated soil data through point cloud technology, combined with automated segmentation and analysis techniques, not only improves excavated soil handling efficiency but also provides reliable data support for decision-making and process optimization during construction. Therefore, researching and developing point cloud technology that can accurately extract and segment excavated soil from shield tunnel conveyor belts is of great practical significance.

[0004] Against this backdrop, research on the extraction and segmentation of excavation point clouds is particularly necessary. First, point cloud technology can provide high-precision three-dimensional data for excavation monitoring during shield tunneling, enabling timely acquisition and analysis of excavation volume and morphology. This is crucial for safety monitoring and quality management during construction. Excessive excavation accumulation can cause conveyor belt blockages, leading to construction stalls and even equipment damage. Real-time monitoring of excavation accumulation using point cloud technology can promptly identify potential risks and avoid safety hazards caused by improper excavation handling. Second, precise point cloud segmentation technology can assist in construction decision-making and optimize excavation handling plans. By analyzing point cloud data, construction managers can dynamically adjust excavation handling methods and transportation plans, reducing resource waste and improving construction efficiency. Finally, automated point cloud processing can reduce manual labor, minimize human error, enhance construction automation, and promote the development of intelligent construction.

[0005] Currently, technologies for extracting and segmenting three-dimensional point clouds have been applied in multiple engineering fields. Common methods for acquiring point clouds include laser radar (LiDAR) and structured light scanning. LiDAR obtains point cloud data on the surface of an object by measuring the time difference between laser returns, while structured light scanning calculates the three-dimensional structure through the deformation of light. In industries such as construction and mining, these technologies can provide accurate three-dimensional data for complex engineering environments. However, the slag used in shield tunnel construction has special characteristics. Its surface morphology is irregular and, affected by geological conditions and the construction environment, its morphology changes dynamically with the progress of construction. This results in point cloud data with a lot of noise, which poses challenges to subsequent extraction and segmentation.

[0006] To address these challenges, existing point cloud segmentation methods include methods based on geometric features, clustering methods, and deep learning methods. Methods based on geometric features divide regions by analyzing the local normal vectors, curvature, and other geometric properties of point clouds. They work well for regular objects, but are difficult to deal with slag with complex shapes and fuzzy boundaries. Clustering methods such as K-means and DBSCAN segment based on the spatial distribution of point clouds, but they mainly rely on the density or distance of the data, do not fully utilize the geometric information of the point cloud, and perform poorly when processing irregular objects. In recent years, with the development of deep learning, convolutional neural networks (CNNs) have been introduced into point cloud processing tasks and have made significant progress. Structures such as PointNet and PointNet++ have performed well in point cloud classification and segmentation tasks. However, these methods are mostly based on regular convolution operations and still have limitations for irregular point cloud data.

[0007] KPConv (kernel point convolution), as a convolution operation that can directly act on irregular point cloud data, solves the limitations of traditional convolutional networks on regular grids and is particularly suitable for sparse and irregular point cloud data. In the task of extracting and segmenting slag point clouds in shield tunnel construction, KPConv can effectively capture the local geometric features of slag by defining kernel points in space and using kernel point weights for convolution, avoiding the feature loss and waste of computing resources caused by point cloud regularization in traditional methods. However, KPConv also has limitations, especially when processing complex and dynamically changing three-dimensional point cloud data such as shield slag, it has shown some shortcomings.

[0008] First, KPConv has advantages in extracting local geometric features, but because it is essentially a local convolution operation, it is insufficient in capturing long-distance dependencies and processing global information. During shield construction, the accumulation morphology of slag undergoes large-scale dynamic changes due to changes in geological conditions or construction progress. It is difficult to effectively model the overall morphology of the slag by relying solely on local convolution. Secondly, KPConv has limited ability to capture geometric features at different scales. The accumulation morphology of slag is complex and has diverse scales. It may manifest as an overall accumulation trend at a large scale, while showing detailed structures at a small scale. KPConv's standard convolution operation lacks a mechanism to integrate information at different scales and cannot fully capture these features, resulting in an incomplete description of the slag morphology.

[0009] Therefore, in view of the above technical problems and defects, it is urgent to design and develop a soil point cloud segmentation processing method, system and platform based on point cloud convolution. Summary of the Invention

[0010] In order to overcome the shortcomings and difficulties of the above-mentioned existing technologies, the purpose of the present invention is to provide a method, system and platform for slag point cloud segmentation processing based on point cloud convolution. By innovatively combining local convolution feature extraction, global multi-head self-attention mechanism and effective feature upsampling strategy, the ability of point cloud data processing is significantly improved, especially showing superior performance in slag point cloud segmentation tasks.

[0011] The first purpose of the present invention is to provide a method for slag point cloud segmentation and processing based on point cloud convolution; the second purpose of the present invention is to provide a slag point cloud segmentation and processing system based on point cloud convolution; the third purpose of the present invention is to provide a slag point cloud segmentation and processing platform based on point cloud convolution.

[0012] The first object of the present invention is achieved in that the method comprises the steps of:

[0013] Generate and obtain first data and second data corresponding to the muck to be segmented, establish a first geometric relationship between the first data and the second data, and generate corresponding third data based on the first corresponding relationship; wherein the first data is three-dimensional point cloud data corresponding to the muck; the second data is core point data corresponding to the muck; and the third data is local convolution feature data of the point cloud;

[0014] Based on the third data, the input feature data is divided into multiple attention heads in combination with a multi-head self-attention mechanism, weight data corresponding to the feature data is dynamically adjusted, and fourth data corresponding to the third data is generated and integrated; wherein the fourth data is the output feature data of each attention head;

[0015] Based on the fourth data, the point cloud data corresponding to the muck is segmented and processed, and fifth data corresponding to the first data is generated; wherein the fifth data is category prediction data corresponding to the muck point cloud.

[0016] Furthermore, the generating and acquiring of first data and second data corresponding to the soil point cloud to be segmented, establishing a first geometric relationship between the first data and the second data, and generating corresponding third data based on the first corresponding relationship, further includes:

[0017] Combined with the weight function, the sixth data between the first data and the second data is calculated and generated; wherein the sixth data is distance weighted data; the calculation formula is as follows:

[0018]

[0019] In the formula, h(p j ,k j ) is the kernel function; p j is a point in the neighborhood; k j is the kernel point; σ is the standard deviation of the Gaussian function, which represents the scale parameter of weight decay.

[0020] Based on the rigid kernel, the shift corresponding to each convolution position is learned and processed to generate the corresponding third data. The learning process calculation formula is as follows:

[0021]

[0022] Where g deform is the deformable convolution kernel function; Δ(p i ) is at point p i The offset learned at h(p j ,k j +Δ j (p i )) represents point p i and the kernel point k after the shift j +Δ j (p i ) between the distance weight; Δ j (p i ) is the core point k j In the convolution position p i The offset is obtained through convolutional network learning; W j is the learnable weight of kernel point j.

[0023] Furthermore, the generating and acquiring of first data and second data corresponding to the soil to be segmented and processed by the point cloud, establishing a first geometric relationship between the first data and the second data, and generating corresponding third data based on the first corresponding relationship, further includes:

[0024] Based on the third data, feature fusion processing is performed in combination with residual connection, and feature data is captured by residual convolution with a step size.

[0025] Furthermore, the method further includes dividing the input feature data into multiple attention heads based on the third data and combining a multi-head self-attention mechanism, dynamically adjusting and processing weight data corresponding to the feature data, and generating and integrating fourth data corresponding to the third data:

[0026] Based on the third data, dividing and processing at least one subspace corresponding to the feature space, and combining different linear transformations to create a first matrix, a second matrix, and a third matrix, respectively; wherein the first matrix is a query matrix; the second matrix is a key matrix; and the third matrix is a value matrix;

[0027] Calculate, generate, and obtain dot product data between the first matrix and the second matrix, and generate corresponding seventh data based on the dot product data; wherein the seventh data is correlation data between points; the calculation formula is as follows:

[0028]

[0029] Where A h is the weight matrix, Q h is the query matrix; K h is the bond matrix; d k is the dimension of query and key vectors;

[0030] According to the weight matrix and in combination with the third matrix, fourth data corresponding to the attention head is generated and integrated.

[0031] Furthermore, the segmenting and processing of the point cloud data corresponding to the slag based on the fourth data and generating fifth data corresponding to the first data further includes:

[0032] Sampling and acquiring eighth data, and processing the sparse and irregularly distributed point cloud data based on the eighth data and in combination with a nearest neighbor interpolation method; wherein the eighth data is fused feature data;

[0033] Based on the fully connected layer, fifth data corresponding to the eighth data is generated, and the fifth data is mapped to the output space of each category.

[0034] The second object of the present invention is achieved as follows: the system is applied to the soil point cloud segmentation processing method based on point cloud convolution, and the system includes:

[0035] A first data generating unit is configured to generate and obtain first data and second data corresponding to the muck to be segmented, establish a first geometric relationship between the first data and the second data, and generate corresponding third data based on the first corresponding relationship; wherein the first data is three-dimensional point cloud data corresponding to the muck; the second data is core point data corresponding to the muck; and the third data is local convolution feature data of the point cloud;

[0036] a second data generating unit, configured to divide the input feature data into a plurality of attention heads based on the third data and in combination with a multi-head self-attention mechanism, dynamically adjust weight data corresponding to the feature data, and generate and integrate fourth data corresponding to the third data; wherein the fourth data is output feature data of each attention head;

[0037] The third data generating unit is used to segment and process the point cloud data corresponding to the slag based on the fourth data, and generate fifth data corresponding to the first data; wherein the fifth data is category prediction data corresponding to the slag point cloud.

[0038] Furthermore, the first data generating unit further includes:

[0039] The first generating module is configured to calculate and generate sixth data between the first data and the second data by combining a weight function; wherein the sixth data is distance weight data; and the calculation formula is as follows:

[0040]

[0041] In the formula, h(p j ,k j ) is the kernel function; p j is a point in the neighborhood; k j is the kernel point; σ is the standard deviation of the Gaussian function, which represents the scale parameter of weight attenuation;

[0042] The second generation module is used to learn and process the shift corresponding to each convolution position based on the rigid kernel, and generate corresponding third data; wherein the learning and processing calculation formula is as follows:

[0043]

[0044] Where g deform is the deformable convolution kernel function; Δ(p i ) is at point p i The offset learned at h(pj ,k j +Δ j (p i )) represents point p i and the kernel point k after the shift j +Δ j (p i ) between the distance weight; Δ j (p i ) is the core point k j In the convolution position p i The offset is obtained through convolutional network learning; W j is the learnable weight of kernel point j;

[0045] And / or, the second data generating unit further includes:

[0046] A first creation module is configured to divide and process at least one subspace corresponding to the feature space based on the third data, and create a first matrix, a second matrix, and a third matrix respectively by combining different linear transformations; wherein the first matrix is a query matrix; the second matrix is a key matrix; and the third matrix is a value matrix;

[0047] The third generating module is configured to calculate, generate, and obtain dot product data between the first matrix and the second matrix, and generate corresponding seventh data based on the dot product data; wherein the seventh data is correlation data between points; and the calculation formula is as follows:

[0048]

[0049] Where A h is the weight matrix, Q h is the query matrix; K h is the bond matrix; d k is the dimension of query and key vectors;

[0050] a fourth generating module, configured to generate and integrate fourth data corresponding to the attention head according to the weight matrix and in combination with the third matrix;

[0051] And / or, the third data generating unit further includes:

[0052] a first processing module, configured to sample and obtain eighth data, and process the sparse and irregularly distributed point cloud data based on the eighth data and in combination with a nearest neighbor interpolation method; wherein the eighth data is fused feature data;

[0053] The second processing module is used to generate fifth data corresponding to the eighth data based on the fully connected layer, and map the fifth data to the output space of each category.

[0054] Furthermore, the first data generating unit further includes:

[0055] The third processing module is used to perform feature fusion processing based on the third data in combination with residual connection, and to capture feature data using residual convolution with a step size.

[0056] The third purpose of the present invention is achieved as follows: the platform includes a processor, a memory and a slag point cloud segmentation processing platform control program based on point cloud convolution; wherein, the slag point cloud segmentation processing platform control program based on point cloud convolution is executed by the processor, the slag point cloud segmentation processing platform control program based on point cloud convolution is stored in the memory, and the slag point cloud segmentation processing platform control program based on point cloud convolution implements the slag point cloud segmentation processing method based on point cloud convolution.

[0057] The present invention generates and obtains first data and second data corresponding to the muck to be segmented into a point cloud, establishes a first geometric relationship between the first data and the second data, and generates corresponding third data based on the first corresponding relationship; wherein the first data is three-dimensional point cloud data corresponding to the muck; the second data is core point data corresponding to the muck; and the third data is local convolution feature data of the point cloud;

[0058] Based on the third data, the input feature data is divided into multiple attention heads in combination with a multi-head self-attention mechanism, weight data corresponding to the feature data is dynamically adjusted, and fourth data corresponding to the third data is generated and integrated; wherein the fourth data is the output feature data of each attention head;

[0059] Based on the fourth data, the point cloud data corresponding to the slag is segmented and processed, and fifth data corresponding to the first data is generated; wherein, the fifth data is the category prediction data corresponding to the slag point cloud, and the system and platform corresponding to the method, through the innovative combination of local convolution feature extraction, global multi-head self-attention mechanism and effective feature upsampling strategy, significantly improve the ability of point cloud data processing, especially showing superior performance in the slag point cloud segmentation task.

[0060] In other words, the KPConvAttentionNet network proposed by the present invention significantly improves the ability to process point cloud data by innovatively combining local convolution feature extraction, global multi-head self-attention mechanism and effective feature upsampling strategy, especially showing superior performance in the task of slag point cloud segmentation. The local convolution module (KPConv) uses deformable convolution kernels to adapt to complex geometric structures, ensuring the flexibility and accuracy of feature extraction. At the same time, the multi-head self-attention mechanism effectively enhances the model's ability to model long-distance dependencies and promotes the perception of global structure. Through the design of upsampling and fully connected layers, we successfully achieved high-resolution feature recovery and accurate category prediction. Experimental results show that KPConvAttentionNet has good robustness and accuracy in the task of slag point cloud segmentation, providing an important reference and reference for future research in the field of point cloud processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0062] Figure 1 This is a schematic flow chart of the steps of a soil point cloud segmentation processing method based on point cloud convolution according to the present invention;

[0063] Figure 2 This is a schematic diagram of a network technology route for an embodiment of a soil point cloud segmentation processing method based on point cloud convolution of the present invention;

[0064] Figure 3 Schematic diagram of the KPConv local convolution feature extraction module of an embodiment of a soil point cloud segmentation processing method based on point cloud convolution of the present invention;

[0065] Figure 4 This is a schematic diagram of a multi-head global attention mechanism in an embodiment of a soil point cloud segmentation processing method based on point cloud convolution of the present invention;

[0066] Figure 5 This is a schematic diagram of the architecture of a soil point cloud segmentation and processing system based on point cloud convolution invented by the present invention.

[0067] Figure 6 This is a structural schematic diagram of a slag point cloud segmentation processing platform based on point cloud convolution in the present invention. DETAILED DESCRIPTION

[0068] In order to better understand the purpose, technical solutions and advantages of the present invention, the present invention is further described below with reference to the accompanying drawings and specific embodiments. Those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification.

[0069] The present invention may also be implemented or applied through other different specific examples, and the details in this specification may also be modified and changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention.

[0070] It should be noted that if the embodiments of the present invention involve directional indications (such as up, down, left, right, front, back, etc.), the directional indications are only used to explain the relative position relationship, movement status, etc. between the various components under a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indications will also change accordingly.

[0071] In addition, if there are descriptions involving "first", "second", etc. in the embodiments of the present invention, the descriptions of "first", "second", etc. are only for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of such features. Secondly, the technical solutions between the various embodiments can be combined with each other, but this must be based on the fact that ordinary technicians in this field can implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such a combination of technical solutions does not exist and is not within the scope of protection required by the present invention.

[0072] Preferably, the soil point cloud segmentation processing method based on point cloud convolution of the present invention is applied to one or more terminals or servers. The terminal is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to a microprocessor, an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), a digital signal processor (DSP), an embedded device, etc.

[0073] The terminal can be a computing device such as a desktop computer, notebook, PDA, cloud server, etc. The terminal can interact with the client through a keyboard, mouse, remote control, touchpad, or voice control device.

[0074] The present invention provides a method, system, platform and storage medium for soil point cloud segmentation based on point cloud convolution.

[0075] like Figure 1 , which is a flow chart of a method for segmenting and processing slag point clouds based on point cloud convolution provided by an embodiment of the present invention.

[0076] In this embodiment, the soil point cloud segmentation processing method based on point cloud convolution can be applied to a terminal with a display function or a fixed terminal. The terminal is not limited to a personal computer, a smart phone, a tablet computer, a desktop computer or an all-in-one computer equipped with a camera, etc.

[0077] The soil point cloud segmentation processing method based on point cloud convolution can also be applied in a hardware environment consisting of a terminal and a server connected to the terminal via a network. The network includes, but is not limited to, a wide area network, a metropolitan area network, or a local area network. The soil point cloud segmentation processing method based on point cloud convolution according to the embodiments of the present invention can be executed by a server, a terminal, or both.

[0078] For example, for a terminal that needs to perform soil point cloud segmentation processing based on point cloud convolution, the soil point cloud segmentation processing function based on point cloud convolution provided by the method of the present invention can be directly integrated on the terminal, or a client for implementing the method of the present invention can be installed. For another example, the method provided by the present invention can also be run on a server or other device in the form of a software development kit (SDK), and an interface for the soil point cloud segmentation processing function based on point cloud convolution is provided in the form of the SDK. The terminal or other device can implement the soil point cloud segmentation processing function based on point cloud convolution through the provided interface. The present invention is further explained below in conjunction with the accompanying drawings.

[0079] The present invention will be described in further detail below with reference to the accompanying drawings. Figures 1-4 As shown, the present invention provides a method for soil point cloud segmentation processing based on point cloud convolution, the method comprising the following steps:

[0080] S01. Generate and obtain first data and second data corresponding to the muck to be segmented, establish a first geometric relationship between the first data and the second data, and generate corresponding third data based on the first corresponding relationship; wherein the first data is three-dimensional point cloud data corresponding to the muck; the second data is core point data corresponding to the muck; and the third data is local convolution feature data of the point cloud;

[0081] S02. Based on the third data, the input feature data is divided into multiple attention heads in combination with a multi-head self-attention mechanism, weight data corresponding to the feature data is dynamically adjusted, and fourth data corresponding to the third data is generated and integrated; wherein the fourth data is output feature data of each attention head;

[0082] S03. Based on the fourth data, segment and process the point cloud data corresponding to the slag, and generate fifth data corresponding to the first data; wherein the fifth data is category prediction data corresponding to the slag point cloud.

[0083] The step of generating and acquiring first data and second data corresponding to the soil point cloud to be segmented, establishing a first geometric relationship between the first data and the second data, and generating corresponding third data based on the first corresponding relationship further includes:

[0084] S011. Calculate and generate sixth data between the first data and the second data by combining a weight function; wherein the sixth data is distance weighted data; the calculation formula is as follows:

[0085]

[0086] In the formula, h(p j ,k j ) is the kernel function; p j is a point in the neighborhood;

[0087] S012. Based on the rigid kernel, learn and process the shift corresponding to each convolution position, and generate corresponding third data; wherein the learning and processing calculation formula is as follows:

[0088]

[0089] Where g deform is the deformable convolution kernel function; Δ(p i ) is at point p i The offset learned at h(p j ,k j +Δ j (p i )) represents point p i and the kernel point k after the shift j +Δ j (p i ) between the distance weight; Δ j (p i ) is the core point k j In the convolution position p i The offset is obtained through convolutional network learning; W j is the learnable weight of kernel point j.

[0090] The step of generating and acquiring first data and second data corresponding to the soil to be segmented and processed by the point cloud, establishing a first geometric relationship between the first data and the second data, and generating corresponding third data based on the first corresponding relationship further includes:

[0091] S013. Based on the third data, feature fusion processing is performed in combination with residual connection, and residual convolution with a step size is used to capture feature data.

[0092] The method further includes dividing the input feature data into multiple attention heads based on the third data and combining a multi-head self-attention mechanism, dynamically adjusting weight data corresponding to the feature data, and generating and integrating fourth data corresponding to the third data.

[0093] S021. Based on the third data, divide and process at least one subspace corresponding to the feature space, and create a first matrix, a second matrix, and a third matrix respectively by combining different linear transformations; wherein the first matrix is a query matrix; the second matrix is a key matrix; and the third matrix is a value matrix;

[0094] S022. Calculate, generate, and obtain dot product data between the first matrix and the second matrix, and generate corresponding seventh data based on the dot product data; wherein the seventh data is correlation data between points; the calculation formula is as follows:

[0095]

[0096] Where A h is the weight matrix, Q h is the query matrix; K h is the bond matrix; d k is the dimension of query and key vectors;

[0097] S023. Generate and integrate fourth data corresponding to the attention head based on the weight matrix and the third matrix.

[0098] The method of segmenting and processing the point cloud data corresponding to the muck based on the fourth data and generating fifth data corresponding to the first data further includes:

[0099] S031. Sampling and acquiring eighth data, and processing the sparse and irregularly distributed point cloud data based on the eighth data and in combination with a nearest neighbor interpolation method; wherein the eighth data is fused feature data;

[0100] S032. Based on the fully connected layer, generate fifth data corresponding to the eighth data, and map the fifth data to the output space of each category.

[0101] Specifically, in an embodiment of the present invention, in order to overcome the shortcomings of KPConv in global information modeling and multi-scale feature capture, the solution of the present invention is improved on the basis of KPConv. First, the solution of the present invention introduces a global self-attention mechanism in the middle layer of KPConv to enhance KPConv's ability to capture long-distance dependencies. The self-attention mechanism calculates the global correlation between each point and other points, so that each point can refer to global information when updating features, thereby making up for the shortcomings of KPConv in global information extraction. By introducing the self-attention mechanism, the model can better capture the overall changes in the accumulation morphology of shield slag and enhance the perception of the global structure, which is particularly important for dealing with the dynamic changes in the morphology of slag during shield tunnel construction.

[0102] This paper proposes a KPConv-based point cloud segmentation network architecture, named KPConvAttentionNet. It combines local convolutional feature extraction with a global multi-head attention mechanism to enhance the model's ability to capture complex geometric structures and long-range dependencies. The entire network adopts a hierarchical design, combining KPConv convolutions, simple convolutions, residual connections, and strided residual convolutions to gradually extract and enhance local and global features in the point cloud.

[0103] During feature extraction, KPConv convolution is first applied to initially extract geometric information and local neighborhood features from the point cloud. Subsequently, simple convolutional layers further refine the KPConv-extracted features to capture more detailed information. After several layers of simple convolution, residual connections are added to enhance feature transfer and aggregation through skip connections. Finally, strided residual convolutions are used to downsample at specific levels, gradually reducing the number of points and expanding the receptive range of features, enabling the model to capture structural information over a wider range. A global multi-head attention mechanism is embedded in intermediate layers (i.e., after a strided residual convolution layer). This allows the model to globally capture long-range inter-point dependencies, thereby achieving a more comprehensive understanding of the spatial structure of the point cloud. The number of heads in the multi-head attention mechanism is optimized experimentally to balance feature representation power and computational cost. Next, the global features extracted by the global multi-head attention mechanism are fused with the local features from the KPConv convolution, enabling the model to construct a stronger feature representation based on both local and global information. In the feature reconstruction phase, the resolution of the point cloud is gradually restored, preserving local details and achieving more refined segmentation. Finally, the output layer of the network generates a category prediction for each point, completing the high-precision segmentation of the point cloud. The network technology roadmap proposed by the present invention is as follows: Figure 2 shown.

[0104] In the solution of the present invention, the provided method includes the following processing steps:

[0105] Local convolution feature extraction module (KPConv): The basic unit of KPConvAttentionNet is the local convolution feature extraction module, which is based on KPConv convolution and implements convolution operation by defining discrete kernel points on irregular point clouds. The input is a three-dimensional point cloud P = {p1, p1, p3...p k}, where each point p∈d is a d-dimensional feature vector, denoted by f j , containing geometric location information or other features.

[0106] KPConv defines K core points, which are Distributed in the neighborhood of each convolution area. Assume that point p i The neighborhood of i ), the neighborhood point is represented by p j , the core point set is

[0107] To calculate the point p i With the kernel point k j Based on the geometric relationship between the points, KPConv introduces a weight function, which is used to calculate the distance weight between the point and the kernel point. This weight function determines the contribution of each point in the neighborhood to the convolution operation. Traditional KPConv uses a linear piecewise function as the weight function. The expression is as follows:

[0108]

[0109] The biggest advantage of the Gaussian distribution weight function is its smoothness and continuity. It can smoothly attenuate the distance weight between the point and the kernel point in space, and will not suddenly truncate the weight to 0 at a certain distance like a linear piecewise function. Because the weight decays exponentially, the Gaussian function ensures that even points at a longer distance still have a weak effect on the convolution result. This is very useful for capturing more complex geometric relationships in point cloud data, especially when you want the model to have a certain global perception ability. The Gaussian function behaves more naturally and smoothly, which helps to extract features from scenes with strong continuity. The weight function is as follows:

[0110]

[0111] h(p j ,k j ) is the kernel function, which calculates the distance weight between the point and the kernel point, p j is a point in the neighborhood. Usually calculated based on distance. KPConv learns kernel point weights To optimize the extraction of local features. The output local features F local Can capture the local geometric structure information of point cloud.

[0112] In order to better adapt to the complex geometric structure in the point cloud, KPConv introduces a deformable convolution kernel. In traditional KPConv, the position of the kernel point is fixed and cannot adapt to changes in different geometric shapes. However, in complex scenes, the local geometric structure of the point cloud may have large irregularities, and the convolution kernel with a fixed position may not be able to capture such geometric changes. The offset Δk of the kernel point is j are learnable parameters that allow the convolution kernel to be adaptively adjusted to different geometric structures.

[0113] The final position of the core point is:

[0114] k j ′=k j +Δk j (3)

[0115] k j ′ is the kernel point position after the shift. The kernel point is learned as a parameter. In convolution, we start with a rigid kernel and learn the shift Δ(k) of each convolution position as follows:

[0116]

[0117] The deformable kernel is defined as:

[0118]

[0119] Among them, (F*g)(p i ) means at point p i The result of the convolution operation at N(p i ) is point p i The neighborhood of , including the nearby point p j . g deform is the deformable convolution kernel function. Δ(p i ) is at point p i The offset learned at f. j is the neighborhood midpoint p j The input feature vector of h(p j ,k j +Δ j (p i )) represents point p j and the kernel point k after the shift j +Δ j (p i ) between the distance weight. Δ j (p i ) is the core point k j In the convolution position p iThe offset is learned through convolutional network. j is the learnable weight of kernel point j. During training, the rigid kernel learns the offset Δ(k) through back propagation, while the deformable kernel generates new features. A special regularization is used to constrain the deformable kernel points to the kernel region. The regularization consists of a fitting term and a rejection term. L1(p i ) fitting term ensures that each kernel point is kept close enough to its input neighbors. L2(p i ) The repulsive term ensures that the core points do not collapse into each other and is defined as j With other kernel points k l The sum of the inverse squares of the distances between them and ensures that their influence areas do not overlap.

[0120] Introducing a new loss function L new :

[0121] L new =∑L1(x)+L2(x)(6)

[0122] L1(p i ) and L2(p i ) are:

[0123]

[0124] k j +Δk j It's point p i The kernel point position after the shift. σ is a hyperparameter that controls the distance decay.

[0125] k j +Δ j (p i ) and k l +Δ l (p i ) represent the positions of the jth and lth kernel points after migration, respectively.

[0126] After introducing the deformable kernel, the convolution operation formula becomes:

[0127]

[0128] Among them, f i ′ is point p i The output feature vector of is obtained through KPConv convolution operation. j It's point p j The input feature vector from p i Neighborhood N(p i ). In summary, the convolution operation of KPConv is performed by i Neighborhood N(p i) is convolved, and finally the local features F′ of the entire point cloud of the current KPConv convolution layer are collected local , specifically expressed as:

[0129]

[0130] The above method is a single-layer KPConv convolution method. In the KPConv encoder, KPConv, simple convolution, residual connection, and strided residual convolution are combined in a hierarchical manner: first, KPConv convolution is used to extract the local features of the point cloud, and then simple convolution is used to further enrich the feature expression; then residual connection is added for feature fusion to ensure that information is transmitted in deep layers. The downsampling layer uses residual convolution with strides to complete downsampling, gradually reducing the resolution and expanding the receptive field, thereby effectively capturing global features while retaining important geometric structures. The simple convolution formula is:

[0131] F simple =σ(W*F local +b)(11)

[0132] Among them, F simple is the feature generated by simple convolution. W is the convolution kernel, b is the bias term, and σ is the LeakyReLU activation function.

[0133] Subsequently, residual connections are added for feature fusion to ensure that information is transmitted in deep layers. Residual connections add input features to convolution outputs through skip connections to alleviate the gradient vanishing problem. The formula is:

[0134] F residual =F input +F simple (12)

[0135] Among them, F input is the input feature, F residual It is the feature output after the residual connection, or it can be the feature obtained from the previous layer. This structure enables the network to retain important information in multiple layers of features and improve the learnability of features.

[0136] The downsampling layer uses residual convolution with a stride to downsample, gradually reducing the resolution and expanding the receptive field, preserving important geometric structures, and effectively capturing global features. The stride convolution operation achieves feature downsampling by reducing the size of the input features. The formula is:

[0137] F downsampled =resnetb_strided(F residual ) (13)

[0138] Among them, resnetb_strided is a convolution with a step size, Fdownsampled It is the feature representation after downsampling. Through this hierarchical structural combination, KPConvAttentionNet can effectively process complex point cloud data and achieve efficient feature extraction and segmentation. Through multi-layer KPConv convolution, simple convolution, residual connection, residual connection with stride, the feature is finally output as F local The schematic diagram of KPConv local convolution feature extraction module is as follows: Figure 3 shown.

[0139] Feature Extraction with Global Multi-Head Attention: In the KPConvAttentionNet network, a global multi-head self-attention mechanism is embedded in the KPConv middle layer to enhance the model's ability to model long-range dependencies and perceive global structure. The core of the multi-head self-attention mechanism is to partition the input features into multiple attention heads, each of which independently calculates the correlation between features and dynamically adjusts the feature weights. This approach not only allows the model to capture the interdependencies between long-range points in point cloud data, but also extracts diverse geometric features in parallel across different attention heads, thereby enhancing the richness and comprehensiveness of feature representation.

[0140] Attention head division: In the multi-head self-attention mechanism, the feature space is divided into H subspaces, and each attention head h will process the input features independently. This process maps the input features to different subspaces of query, key, and value through different linear transformations, ensuring that each head can focus on capturing different feature representations. Specifically, the input features are transformed into the query matrix Q, key matrix K, and value matrix V through three different linear transformations. The formula is:

[0141]

[0142] in, and is the linear transformation matrix defined for the h-th head, F′ local Represents the features of the intermediate layer of KPConv. These transformations project the input features into a low-dimensional space so that each head can learn independently in its own feature subspace.

[0143] Calculation of attention weights: After determining the feature representation of each attention head, the next step is to calculate the attention weights. For the h-th head, by calculating the query matrix Q h and bond matrix K h The correlation between points is obtained by the dot product between them. Specifically, the calculation formula is as follows:

[0144]

[0145] In this formula, d kIs the dimension of the query and key vectors, as a scaling factor to avoid the dot product value being too large. After Softmax normalization, the weight matrix A h It reflects the relative importance of the input points, and the sum of each row is 1.

[0146] Weighted summation of features: In KPConv, the calculation formula of local features is:

[0147]

[0148] Among them, f i ′ is the feature extracted by KPConv convolution operation. In the global self-attention mechanism, the attention weight matrix A is obtained h Will be used for the value matrix V h Perform weighted summation to generate the output feature Z of each attention head h :

[0149] Z h =A h V h (17)

[0150] Combined with the local features obtained by KPConv, the final feature representation can be calculated by the following formula:

[0151] F=βZ h +(1-β)F local (18)

[0152] Among them, β is used to adjust the global feature Z h and local features F local This weighting mechanism enables the model to dynamically adjust the influence of local and global information, thereby enhancing feature expression capabilities.

[0153] Integration of multi-head outputs: Finally, the output features of all attention heads are concatenated and the final feature representation is obtained through linear transformation:

[0154] Z=Concat(Z1,Z2,...,Z H )W O (19)

[0155] Among them, W O is a linear transformation matrix used to integrate the outputs of multiple heads. Through this mechanism, each attention head can independently learn different feature spaces, thereby capturing the diverse geometric features in the point cloud.

[0156] Feature upsampling and soil point cloud segmentation: After completing the fusion of the global multi-head self-attention mechanism and local features, the next step is to segment the conveyor belt soil point cloud through upsampling and fully connected layers. The goal of this process is to restore the feature map to the same resolution as the original soil point cloud so as to generate accurate category predictions for each point, thereby improving the accuracy and robustness of point cloud segmentation. Figure 4 shown.

[0157] Feature upsampling: To restore the resolution of the soil point cloud, the fused features must first be upsampled. We chose the nearest neighbor interpolation method. This method is simple and efficient, and is suitable for processing sparse and irregularly distributed point cloud data.

[0158] Let Z be the feature matrix after the fusion of global and local features, with dimensions n × d, where n represents the number of points and d represents the feature dimension. These features contain both the local geometric information of the soil and its global structural characteristics. An upsampling factor s is selected to determine the resolution of the output feature map. For each target coordinate (x′, y′), its corresponding source coordinate (x, y) in the source feature map is calculated using the following formula:

[0159]

[0160] In this way, the source of each upsampled point can be determined. Assign a value to each target point (x′, y′):

[0161] F upsampled (x′,y′)=F(x,y) (21)

[0162] This method ensures that feature information is effectively preserved during the upsampling process and ensures that each point of the output feature matches the corresponding point in the original point cloud.

[0163] The final upsampling operation can be expressed as:

[0164] F upsampled =Upsample(F)(22)

[0165] Among them, F is the feature representation after the fusion of global and local features, F upsampled It is the feature after upsampling operation.

[0166] F seg =FC(F upsampled )(twenty three)

[0167] Fully connected layer: After the feature upsampling is completed, these upsampled features are input into the fully connected layer to generate the category prediction for each point. The role of the fully connected layer is to process the upsampled features and map them to the output space of each category. The specific process can be expressed as:

[0168] F seg =FC(F upsampled )(twenty four)

[0169] In this formula, F seg is the segmentation output of each point, including the probability distribution of the category of each point. FC represents the operation of the fully connected layer. The fully connected layer extracts high-dimensional feature information by integrating upsampled features, helping the model generate accurate classification results.

[0170] Classification output: Finally, the Softmax function is used to convert the output of the fully connected layer into a category prediction for each point:

[0171] P=Softmax(F seg )(25)

[0172] Among them, P is the category probability distribution of each point in the point cloud, ensuring that the sum of the probabilities of all categories is 1. Through this process, the model is able to generate accurate category predictions for each input point, thereby achieving high-precision soil point cloud segmentation.

[0173] The KPConvAttentionNet network proposed in the present invention significantly improves the ability to process point cloud data by innovatively combining local convolutional feature extraction, a global multi-head self-attention mechanism, and an effective feature upsampling strategy, especially showing superior performance in the task of slag point cloud segmentation. The local convolution module (KPConv) uses deformable convolution kernels to adapt to complex geometric structures, ensuring the flexibility and accuracy of feature extraction. At the same time, the multi-head self-attention mechanism effectively enhances the model's ability to model long-range dependencies and promotes the perception of global structure. Through the design of upsampling and fully connected layers, we successfully achieved high-resolution feature recovery and accurate category prediction. Experimental results show that KPConvAttentionNet has good robustness and accuracy in the task of slag point cloud segmentation, providing an important reference and reference for future research in the field of point cloud processing.

[0174] To achieve the above purpose, the present invention also provides a soil point cloud segmentation processing system based on point cloud convolution, such as Figure 5 As shown, the system is applied to the soil point cloud segmentation processing method based on point cloud convolution, and the system includes:

[0175] A first data generating unit is configured to generate and obtain first data and second data corresponding to the muck to be segmented, establish a first geometric relationship between the first data and the second data, and generate corresponding third data based on the first corresponding relationship; wherein the first data is three-dimensional point cloud data corresponding to the muck; the second data is core point data corresponding to the muck; and the third data is local convolution feature data of the point cloud;

[0176] a second data generating unit, configured to divide the input feature data into a plurality of attention heads based on the third data and in combination with a multi-head self-attention mechanism, dynamically adjust weight data corresponding to the feature data, and generate and integrate fourth data corresponding to the third data; wherein the fourth data is output feature data of each attention head;

[0177] The third data generating unit is used to segment and process the point cloud data corresponding to the slag based on the fourth data, and generate fifth data corresponding to the first data; wherein the fifth data is category prediction data corresponding to the slag point cloud.

[0178] The first data generating unit further includes:

[0179] The first generating module is configured to calculate and generate sixth data between the first data and the second data by combining a weight function; wherein the sixth data is distance weight data; and the calculation formula is as follows:

[0180]

[0181] In the formula, h(p j ,k j ) is the kernel function; p j is a point in the neighborhood;

[0182] The second generation module is used to learn and process the shift corresponding to each convolution position based on the rigid kernel, and generate corresponding third data; wherein the learning and processing calculation formula is as follows:

[0183]

[0184] Where g deform is the deformable convolution kernel function; Δ(p i ) is at point p i The offset learned at h(p j ,k j +Δ j (p i )) represents point p i and the kernel point k after the shift j +Δ j (pi ) between the distance weight; Δ j (p i ) is the core point k j In the convolution position p i The offset is obtained through convolutional network learning; W j is the learnable weight of kernel point j;

[0185] And / or, the second data generating unit further includes:

[0186] A first creation module is configured to divide and process at least one subspace corresponding to the feature space based on the third data, and create a first matrix, a second matrix, and a third matrix respectively by combining different linear transformations; wherein the first matrix is a query matrix; the second matrix is a key matrix; and the third matrix is a value matrix;

[0187] The third generating module is configured to calculate, generate, and obtain dot product data between the first matrix and the second matrix, and generate corresponding seventh data based on the dot product data; wherein the seventh data is correlation data between points; and the calculation formula is as follows:

[0188]

[0189] Where A h is the weight matrix, Q h is the query matrix; K h is the bond matrix; d k is the dimension of query and key vectors;

[0190] a fourth generating module, configured to generate and integrate fourth data corresponding to the attention head according to the weight matrix and in combination with the third matrix;

[0191] And / or, the third data generating unit further includes:

[0192] a first processing module, configured to sample and obtain eighth data, and process the sparse and irregularly distributed point cloud data based on the eighth data and in combination with a nearest neighbor interpolation method; wherein the eighth data is fused feature data;

[0193] The second processing module is used to generate fifth data corresponding to the eighth data based on the fully connected layer, and map the fifth data to the output space of each category.

[0194] The first data generating unit further includes:

[0195] The third processing module is used to perform feature fusion processing based on the third data in combination with residual connection, and to capture feature data using residual convolution with a step size.

[0196] In the system solution embodiment of the present invention, the method steps involved in the soil point cloud segmentation processing based on point cloud convolution have been described above in detail. That is to say, the functional modules in the system are used to implement the steps or sub-steps in the above method embodiment, which will not be repeated here.

[0197] To achieve the above objectives, the present invention also provides a soil point cloud segmentation processing platform based on point cloud convolution, such as Figure 6 As shown, it includes a processor, a memory, and a control program for a soil point cloud segmentation processing platform based on point cloud convolution; wherein, the control program for the soil point cloud segmentation processing platform based on point cloud convolution is executed by the processor, the control program for the soil point cloud segmentation processing platform based on point cloud convolution is stored in the memory, and the control program for the soil point cloud segmentation processing platform based on point cloud convolution implements the steps of the soil point cloud segmentation processing method based on point cloud convolution. For example:

[0198] S01. Generate and obtain first data and second data corresponding to the muck to be segmented, establish a first geometric relationship between the first data and the second data, and generate corresponding third data based on the first corresponding relationship; wherein the first data is three-dimensional point cloud data corresponding to the muck; the second data is core point data corresponding to the muck; and the third data is local convolution feature data of the point cloud;

[0199] S02. Based on the third data, the input feature data is divided into multiple attention heads in combination with a multi-head self-attention mechanism, weight data corresponding to the feature data is dynamically adjusted, and fourth data corresponding to the third data is generated and integrated; wherein the fourth data is output feature data of each attention head;

[0200] S03. Based on the fourth data, segment and process the point cloud data corresponding to the slag, and generate fifth data corresponding to the first data; wherein the fifth data is category prediction data corresponding to the slag point cloud.

[0201] The specific details of the steps have been explained above and will not be repeated here.

[0202] In an embodiment of the present invention, the built-in processor of the soil point cloud segmentation processing platform based on point cloud convolution can be composed of an integrated circuit, for example, a single packaged integrated circuit, or a plurality of integrated circuits with the same or different functions, including one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and a combination of various control chips. The processor uses various interfaces and circuits to connect various components, and executes or executes programs or units stored in the memory, as well as calls data stored in the memory, to perform various functions and process data of the soil point cloud segmentation processing based on point cloud convolution.

[0203] The memory is used to store program codes and various data. It is installed in the soil point cloud segmentation processing platform based on point cloud convolution, and realizes high-speed and automatic access to programs or data during operation.

[0204] The memory includes read-only memory (ROM), random access memory (RAM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), one-time programmable read-only memory (OTPROM), electronically erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, magnetic disk storage, magnetic tape storage, or any other computer-readable medium that can be used to carry or store data.

[0205] The present invention generates and obtains first data and second data corresponding to the slag point cloud to be segmented and processed through a method, establishes a first geometric relationship between the first data and the second data, and generates corresponding third data based on the first corresponding relationship; wherein, the first data is three-dimensional point cloud data corresponding to the slag; the second data is core point data corresponding to the slag; and the third data is local convolution feature data of the point cloud; based on the third data, the input feature data is divided and processed into multiple attention heads in combination with a multi-head self-attention mechanism, the weight data corresponding to the feature data is dynamically adjusted and processed, and fourth data corresponding to the third data is generated and integrated; wherein, the fourth data is the output feature data of each attention head; based on the fourth data, the point cloud data corresponding to the slag is segmented and processed, and fifth data corresponding to the first data is generated; wherein, the fifth data is category prediction data corresponding to the slag point cloud, as well as the system and platform corresponding to the method, through the innovative combination of local convolution feature extraction, global multi-head self-attention mechanism and effective feature upsampling strategy, significantly improves the point cloud data processing capability, especially showing superior performance in the slag point cloud segmentation task.

[0206] In other words, the KPConvAttentionNet network proposed by the present invention significantly improves the ability to process point cloud data by innovatively combining local convolution feature extraction, global multi-head self-attention mechanism and effective feature upsampling strategy, especially showing superior performance in the task of slag point cloud segmentation. The local convolution module (KPConv) uses deformable convolution kernels to adapt to complex geometric structures, ensuring the flexibility and accuracy of feature extraction. At the same time, the multi-head self-attention mechanism effectively enhances the model's ability to model long-distance dependencies and promotes the perception of global structure. Through the design of upsampling and fully connected layers, we successfully achieved high-resolution feature recovery and accurate category prediction. Experimental results show that KPConvAttentionNet has good robustness and accuracy in the task of slag point cloud segmentation, providing an important reference and reference for future research in the field of point cloud processing.

[0207] The above-described embodiments merely illustrate several implementations of the present invention, and while their descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art would be able to make numerous variations and improvements without departing from the spirit of the present invention, all of which fall within the scope of protection of the present invention. Therefore, the scope of protection of the present invention shall be determined by the appended claims.

Claims

1. A soil point cloud segmentation processing method based on point cloud convolution, characterized in that: The method comprises the steps of: Generate and obtain first data and second data corresponding to the muck to be segmented, establish a first geometric relationship between the first data and the second data, and generate corresponding third data based on the first corresponding relationship; wherein the first data is three-dimensional point cloud data corresponding to the muck; the second data is core point data corresponding to the muck; and the third data is local convolution feature data of the point cloud; Based on the third data, the input feature data is divided into multiple attention heads in combination with a multi-head self-attention mechanism, weight data corresponding to the feature data is dynamically adjusted, and fourth data corresponding to the third data is generated and integrated; wherein the fourth data is the output feature data of each attention head; Based on the fourth data, the point cloud data corresponding to the muck is segmented and processed, and fifth data corresponding to the first data is generated; wherein the fifth data is category prediction data corresponding to the muck point cloud.

2. The method for soil point cloud segmentation based on point cloud convolution according to claim 1, characterized in that: The step of generating and acquiring first data and second data corresponding to the soil point cloud to be segmented, establishing a first geometric relationship between the first data and the second data, and generating corresponding third data based on the first corresponding relationship further includes: Combined with the weight function, the sixth data between the first data and the second data is calculated and generated; wherein the sixth data is distance weighted data; the calculation formula is as follows: In the formula, h(p j ,k j ) is the kernel function; p j is a point in the neighborhood; k j is the kernel point; σ is the standard deviation of the Gaussian function, which represents the scale parameter of weight decay. Based on the rigid kernel, the shift corresponding to each convolution position is learned and processed to generate the corresponding third data. The learning process calculation formula is as follows: Where g deform is the deformable convolution kernel function; Δ(p i ) is at point p i The offset learned at h(p j ,k j +Δ j (p i )) represents point p i and the kernel point k after the shift j +Δ j (p i ) between the distance weight; Δ j (p i ) is the core point k j In the convolution position p i The offset is obtained through convolutional network learning; W j is the learnable weight of kernel point j; K is the number of kernel points.

3. The method for soil point cloud segmentation based on point cloud convolution according to claim 1 or 2, characterized in that: The step of generating and acquiring first data and second data corresponding to the soil to be segmented and processed by the point cloud, establishing a first geometric relationship between the first data and the second data, and generating corresponding third data based on the first corresponding relationship further includes: Based on the third data, feature fusion processing is performed in combination with residual connection, and feature data is captured by residual convolution with a step size.

4. The method for soil point cloud segmentation based on point cloud convolution according to claim 1, characterized in that: The method further includes dividing the input feature data into multiple attention heads based on the third data and combining a multi-head self-attention mechanism, dynamically adjusting weight data corresponding to the feature data, and generating and integrating fourth data corresponding to the third data. Based on the third data, dividing and processing at least one subspace corresponding to the feature space, and combining different linear transformations to create a first matrix, a second matrix, and a third matrix, respectively; wherein the first matrix is a query matrix; the second matrix is a key matrix; and the third matrix is a value matrix; Calculate, generate, and obtain dot product data between the first matrix and the second matrix, and generate corresponding seventh data based on the dot product data; wherein the seventh data is correlation data between points; the calculation formula is as follows: Where A h is the weight matrix, Q h is the query matrix; K h is the bond matrix; d k is the dimension of query and key vectors; According to the weight matrix and in combination with the third matrix, fourth data corresponding to the attention head is generated and integrated.

5. The method for soil point cloud segmentation based on point cloud convolution according to claim 1, characterized in that: The method of segmenting and processing the point cloud data corresponding to the muck based on the fourth data and generating fifth data corresponding to the first data further includes: Sampling and acquiring eighth data, and processing the sparse and irregularly distributed point cloud data based on the eighth data and in combination with a nearest neighbor interpolation method; wherein the eighth data is fused feature data; Based on the fully connected layer, fifth data corresponding to the eighth data is generated, and the fifth data is mapped to the output space of each category.

6. A soil point cloud segmentation and processing system based on point cloud convolution, characterized in that: The system is applied to the soil point cloud segmentation processing method based on point cloud convolution as claimed in any one of claims 1 to 5, and the system includes: A first data generating unit is configured to generate and obtain first data and second data corresponding to the muck to be segmented, establish a first geometric relationship between the first data and the second data, and generate corresponding third data based on the first corresponding relationship; wherein the first data is three-dimensional point cloud data corresponding to the muck; the second data is core point data corresponding to the muck; and the third data is local convolution feature data of the point cloud; a second data generating unit, configured to divide the input feature data into a plurality of attention heads based on the third data and in combination with a multi-head self-attention mechanism, dynamically adjust weight data corresponding to the feature data, and generate and integrate fourth data corresponding to the third data; wherein the fourth data is output feature data of each attention head; The third data generating unit is used to segment and process the point cloud data corresponding to the slag based on the fourth data, and generate fifth data corresponding to the first data; wherein the fifth data is category prediction data corresponding to the slag point cloud.

7. The soil point cloud segmentation and processing system based on point cloud convolution according to claim 6, characterized in that: The first data generating unit further includes: The first generating module is configured to calculate and generate sixth data between the first data and the second data by combining a weight function; wherein the sixth data is distance weight data; and the calculation formula is as follows: In the formula, h(p j ,k j ) is the kernel function; p j is a point in the neighborhood, k j is the kernel point; σ is the standard deviation of the Gaussian function, which represents the scale parameter of weight attenuation; The second generation module is used to learn and process the shift corresponding to each convolution position based on the rigid kernel, and generate corresponding third data; wherein the learning and processing calculation formula is as follows: Where g deform is the deformable convolution kernel function; Δ(p i ) is at point p i The offset learned at h(p j ,k j +Δ j (p i )) represents point p i and the kernel point k after the shift j +Δ j (p i ) between the distance weight; Δ j (p i ) is the core point k j In the convolution position p i The offset is obtained through convolutional network learning; W j is the learnable weight of kernel point j; And / or, the second data generating unit further includes: A first creation module is configured to divide and process at least one subspace corresponding to the feature space based on the third data, and create a first matrix, a second matrix, and a third matrix respectively by combining different linear transformations; wherein the first matrix is a query matrix; the second matrix is a key matrix; and the third matrix is a value matrix; The third generating module is configured to calculate, generate, and obtain dot product data between the first matrix and the second matrix, and generate corresponding seventh data based on the dot product data; wherein the seventh data is correlation data between points; and the calculation formula is as follows: Where A h is the weight matrix, Q h is the query matrix; K h is the bond matrix; d k is the dimension of query and key vectors; a fourth generating module, configured to generate and integrate fourth data corresponding to the attention head according to the weight matrix and in combination with the third matrix; And / or, the third data generating unit further includes: a first processing module, configured to sample and obtain eighth data, and process the sparse and irregularly distributed point cloud data based on the eighth data and in combination with a nearest neighbor interpolation method; wherein the eighth data is fused feature data; The second processing module is used to generate fifth data corresponding to the eighth data based on a fully connected layer, and map the fifth data to an output space of each category.

8. The soil point cloud segmentation and processing system based on point cloud convolution according to claim 6 or 7, characterized in that: The first data generating unit further includes: The third processing module is used to perform feature fusion processing based on the third data in combination with residual connection, and to capture feature data using residual convolution with a step size.

9. A soil point cloud segmentation processing platform based on point cloud convolution, characterized in that: It includes a processor, a memory and a point cloud convolution-based slag point cloud segmentation processing platform control program; wherein, the point cloud convolution-based slag point cloud segmentation processing platform control program is executed by the processor, the point cloud convolution-based slag point cloud segmentation processing platform control program is stored in the memory, and the point cloud convolution-based slag point cloud segmentation processing platform control program implements the point cloud convolution-based slag point cloud segmentation processing method according to any one of claims 1 to 5.

Images