A slag point cloud segmentation processing method, system and platform based on point cloud convolution

Abstract

The application discloses a kind of based on point cloud convolution's slag soil point cloud segmentation processing method, system and platform, by generating and obtaining and slag soil corresponding first data and second data of point cloud segmentation processing, and create the first geometric relationship of first data and second data, based on first corresponding relationship, generate corresponding third data;According to third data, input feature data is divided and processed into multiple attention heads in combination with multi-head self-attention mechanism, dynamically adjust the weight data corresponding to feature data, and generate and integrate the fourth data corresponding to the third data;Based on fourth data, the point cloud data corresponding to slag soil is segmented and processed, and the fifth data corresponding to the first data is generated;And system platform, innovatively combine local convolution feature extraction, global multi-head self-attention mechanism and effective feature upsampling strategy, significantly improve the ability of point cloud data processing, especially in slag soil point cloud segmentation task shows superior performance.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of slag point cloud segmentation processing, and particularly relates to a slag point cloud segmentation processing method, system and platform based on point cloud convolution. BACKGROUND

[0002] The construction of shield tunnels plays an important role in modern urban infrastructure, especially in underground projects such as subways and sewers. As an efficient underground excavation equipment, shield machines produce a large amount of slag during the excavation process, which is transported to the ground by a belt conveyor system. How to effectively monitor the slag during shield tunneling to improve construction efficiency and safety has become a problem that needs to be solved in the engineering field. Point cloud technology provides a potential solution to this problem. By acquiring and processing three-dimensional data of slag through sensors, accurate measurement and state monitoring of slag volume can be achieved. However, due to the complex shape of slag accumulation and the harsh construction environment, existing point cloud processing technology still faces many challenges, and accurate extraction and segmentation of slag point cloud have become an important research direction for intelligent construction.

[0003] During shield construction, the slag transported by the belt conveyor forms a complex three-dimensional geometric structure in the transportation and accumulation process. These slag forms are not only influenced by the geological environment, but also dynamically change with the progress of excavation. Traditional slag monitoring methods, such as manual observation or simple volume measurement, cannot accurately describe the accumulation state and volume change of the slag, are inefficient, and are prone to human error, making it difficult to meet the needs of modern intelligent construction. By collecting three-dimensional data of slag through point cloud technology and combining automatic segmentation and analysis technology, not only can the efficiency of slag handling be improved, but also reliable data support can be provided for decision-making and process optimization during construction. Therefore, it is of great practical significance to research and develop point cloud technology that can accurately extract and segment the slag of shield tunnel belt conveyors.

[0004] Under this background, it is particularly necessary to carry out research on the extraction and segmentation of slag point cloud. First, point cloud technology can provide high-precision three-dimensional data for slag monitoring in shield construction, so that the volume and form information of the slag can be acquired and analyzed in a timely manner. This is crucial for safety monitoring and quality management during construction. Excessive accumulation of slag can cause belt conveyor blockage, which may lead to construction stagnation and even equipment damage. By monitoring the accumulation state of the slag in real time through point cloud technology, potential risks can be detected in time, and safety hazards caused by improper slag handling can be avoided. Second, accurate point cloud segmentation technology can assist in construction decision-making and optimize slag handling schemes. Through analysis of point cloud data, construction managers can dynamically adjust the slag handling method and transportation plan, reduce resource waste, and improve construction efficiency. Finally, automated point cloud processing methods can reduce reliance on human labor, reduce human error, and improve the level of construction automation, promoting the development of intelligent construction.

[0005] Currently, the extraction and segmentation techniques for three-dimensional point clouds have been applied in many engineering fields. Commonly used point cloud acquisition methods include LiDAR (Light Detection and Ranging) and structured light scanning technology. LiDAR obtains point cloud data of object surfaces by measuring the time difference of laser return, while structured light scanning calculates three-dimensional structures through light deformation. In the construction, mining and other industries, these technologies can provide accurate three-dimensional data for complex engineering environments. However, the spoil in shield tunnel construction has special properties, with irregular surface morphology, and is influenced by geological conditions and construction environment, with dynamic changes in morphology with construction progress, resulting in point cloud data with more noise, which brings challenges to subsequent extraction and segmentation work.

[0006] To address these challenges, current point cloud segmentation methods include geometric feature-based methods, clustering methods, and deep learning methods. Geometric feature-based methods divide regions by analyzing local normal vectors, curvature, and other geometric properties of point clouds, which work well for regular objects but struggle with complex and fuzzy boundary spoil. Clustering methods such as K-means and DBSCAN perform segmentation based on the spatial distribution of point clouds, but mainly rely on data density or distance, and do not fully utilize the geometric information of point clouds, which performs poorly when dealing with irregular objects. In recent years, with the development of deep learning, convolutional neural networks (CNN) have been introduced into point cloud processing tasks, achieving significant progress. Structures such as PointNet and PointNet++ perform well in point cloud classification and segmentation tasks. However, these methods are mostly based on regular convolution operations, which still have limitations for irregular point cloud data.

[0007] KPConv (Kernel Point Convolution) is a convolution operation that can directly act on irregular point cloud data, solving the limitations of traditional convolution networks on regular grids, and is particularly suitable for sparse and irregular point cloud data. In the task of spoil point cloud extraction and segmentation in shield tunnel construction, KPConv can effectively capture the local geometric features of spoil by defining kernel points in space and using kernel point weights for convolution, avoiding feature loss and waste of computing resources caused by point cloud regularization in traditional methods. However, KPConv also has limitations, especially when dealing with complex and dynamically changing three-dimensional point cloud data such as shield spoil, it shows some shortcomings.

[0008] Firstly, KPConv has an advantage in local geometric feature extraction, but it is a local convolution operation in nature, which has a deficiency in capturing long-distance dependence and processing global information. In the process of shield construction, the accumulation form of muck changes dynamically in a large range due to geological conditions or construction progress changes. It is difficult to effectively model the overall form of muck by relying solely on local convolution. Secondly, KPConv has limited ability to capture geometric features of different scales. The accumulation form of muck is complex and diverse in scale, which may exhibit overall accumulation trends at large scales and detailed structures at small scales. The standard convolution operation of KPConv lacks a mechanism to integrate information of different scales, and cannot fully capture these features, resulting in incomplete description of muck form.

[0009] Therefore, in view of the above technical problems and defects, there is an urgent need to design and develop a point cloud convolution-based muck point cloud segmentation processing method, system and platform. SUMMARY

[0010] To overcome the deficiencies and difficulties of the prior art, the purpose of the present application is to provide a point cloud convolution-based muck point cloud segmentation processing method, system and platform, which significantly improves the ability of point cloud data processing by innovatively combining local convolution feature extraction, global multi-head self-attention mechanism and effective feature up-sampling strategy, and exhibits superior performance in muck point cloud segmentation tasks.

[0011] The first purpose of the present application is to provide a point cloud convolution-based muck point cloud segmentation processing method; the second purpose of the present application is to provide a point cloud convolution-based muck point cloud segmentation processing system; and the third purpose of the present application is to provide a point cloud convolution-based muck point cloud segmentation processing platform.

[0012] The first purpose of the present application is achieved by the method comprising the following steps:

[0013] First data and second data corresponding to muck point cloud segmentation processing are generated and obtained, and a first geometric relationship of the first data and the second data is created, and corresponding third data is generated based on the first corresponding relationship; wherein the first data is three-dimensional point cloud data corresponding to muck; the second data is kernel point data corresponding to muck; and the third data is point cloud local convolution feature data;

[0014] According to the third data, the input feature data is divided and processed into multiple attention heads by combining a multi-head self-attention mechanism, the weight data corresponding to the feature data is dynamically adjusted, and the fourth data corresponding to the third data is generated and integrated; wherein the fourth data is the output feature data of each attention head;

[0015] Based on the fourth data, the point cloud data corresponding to the slag is segmented, and the fifth data corresponding to the first data is generated; wherein the fifth data is the category prediction data corresponding to the slag point cloud.

[0016] Further, the first data and the second data corresponding to the slag point cloud segmentation processing are generated and obtained, and the first geometric relationship of the first data and the second data is created, based on the first corresponding relationship, the corresponding third data is generated, and the method further comprises:

[0017] In combination with the weight function, the sixth data between the first data and the second data is calculated and generated; wherein the sixth data is distance weight data; the calculation formula is as follows:

[0018]

[0019] In the formula, h(p j ,k j ) is the kernel function; p j is the point in the neighborhood; k j is the kernel point; σ is the standard deviation of the Gaussian function, which is the scale parameter of weight attenuation.

[0020] Based on the rigid kernel, the displacement corresponding to each convolution position is learned and processed, and the corresponding third data is generated; wherein the calculation formula of the learning and processing is as follows:

[0021]

[0022] In the formula, g deform is the deformable convolution kernel function; Δ(p i ) is the offset learned at the point p i ; h(p j ,k j + Δ j (p i )) represents the distance weight between the point p i and the kernel point k j + Δ j (p i ) after offset; Δ j (p i ) is the offset of the kernel point k j at the convolution position p i , which is learned through the convolution network; W j is the learnable weight of the kernel point j.

[0023] Further, the first data and the second data corresponding to the to-be-point-cloud-segmented spoil are generated and acquired, and a first geometric relationship of the first data and the second data is created, based on the first corresponding relationship, corresponding third data is generated, and the method further comprises:

[0024] Based on the third data, feature fusion processing is performed in combination with residual connection, and step-by-step residual convolution is adopted to capture feature data.

[0025] Further, based on the third data, the input feature data is divided into multiple attention heads in combination with a multi-head self-attention mechanism, the weight data corresponding to the feature data is dynamically adjusted, the fourth data corresponding to the third data is generated and integrated, and the method further comprises:

[0026] Based on the third data, at least one subspace corresponding to a feature space is divided, and a first matrix, a second matrix and a third matrix are respectively created in combination with different linear transformations; the first matrix is a query matrix; the second matrix is a key matrix; and the third matrix is a value matrix.

[0027] Point product data between the first matrix and the second matrix is calculated and acquired, and based on the point product data, corresponding seventh data is generated; the seventh data is point-to-point correlation data; and the calculation formula is as follows:

[0028]

[0029] In the formula, A h is a weight matrix, Q h is a query matrix; K h is a key matrix; d k is the dimension of the query and key vectors.

[0030] Based on the weight matrix, the fourth data corresponding to the attention head is generated and integrated in combination with the third matrix.

[0031] Further, based on the fourth data, the point cloud data corresponding to the spoil is segmented, and fifth data corresponding to the first data is generated.

[0032] The eighth data is sampled and acquired, and the sparse and irregularly distributed point cloud data is processed in combination with the nearest neighbor interpolation method based on the eighth data; the eighth data is the fused feature data.

[0033] Based on the full connection layer, fifth data corresponding to the eighth data is generated, and the fifth data is mapped to the output space of each category.

[0034] A second object of the present application is achieved in that the system is applied to the above-mentioned point cloud convolution-based muck point cloud segmentation processing method, and the system comprises:

[0035] a first data generation unit configured to generate and obtain first data and second data corresponding to muck point cloud segmentation processing, create a first geometric relationship of the first data and the second data, and generate corresponding third data based on the first corresponding relationship; wherein the first data is three-dimensional point cloud data corresponding to muck; the second data is kernel point data corresponding to muck; and the third data is point cloud local convolution feature data;

[0036] a second data generation unit configured to divide and process input feature data into a plurality of attention heads in combination with a multi-head self-attention mechanism based on the third data, dynamically adjust weight data corresponding to the feature data, and generate and integrate fourth data corresponding to the third data; wherein the fourth data is output feature data of each attention head;

[0037] a third data generation unit configured to segment the point cloud data corresponding to the muck based on the fourth data, and generate fifth data corresponding to the first data; wherein the fifth data is category prediction data corresponding to the muck point cloud.

[0038] Further, the first data generation unit further comprises:

[0039] a first generation module configured to calculate and generate sixth data between the first data and the second data in combination with a weight function; wherein the sixth data is distance weight data; and the calculation formula is as follows:

[0040]

[0041] wherein h(p j ,k j ) is a kernel function; p j is a point in the neighborhood; k j is a kernel point; and σ is a standard deviation of a Gaussian function, representing a scale parameter of weight decay.

[0042] a second generation module configured to learn and process a shift corresponding to each convolution position based on a rigid kernel as a reference, and generate corresponding third data; wherein the learning and processing calculation formula is as follows:

[0043]

[0044] wherein g deform is a deformable convolution kernel function; and Δ(p i ) is a learned offset at point p i . h(pj ,k j +Δ j (p i )) represents the distance weight between the point p i and the offset kernel point k j +Δ j (p i ); Δ j (p i ) is the offset of the kernel point k j at the convolution position p i , which is learned by the convolution network; W j is the learnable weight of the kernel point j;

[0045] And / or, the second data generation unit further comprises:

[0046] A first creation module configured to divide at least one subspace corresponding to a feature space based on the third data, and create a first matrix, a second matrix and a third matrix in combination with different linear transformations; wherein the first matrix is a query matrix; the second matrix is a key matrix; and the third matrix is a value matrix;

[0047] A third generation module configured to calculate and obtain dot product data between the first matrix and the second matrix, and generate corresponding seventh data based on the dot product data; wherein the seventh data is correlation data between points and points; and the calculation formula is as follows:

[0048]

[0049] In the formula, A h is a weight matrix, Q h is a query matrix; K h is a key matrix; and d k is the dimension of the query and key vectors;

[0050] A fourth generation module configured to generate and integrate fourth data corresponding to an attention head in combination with the third matrix based on the weight matrix;

[0051] And / or, the third data generation unit further comprises:

[0052] A first processing module configured to sample and obtain eighth data, and process sparse and irregularly distributed point cloud data in combination with a nearest neighbor interpolation method based on the eighth data; wherein the eighth data is fused feature data;

[0053] A second processing module configured to generate fifth data corresponding to the eighth data based on a fully connected layer, and map the fifth data to an output space of each category.

[0054] Further, the first data generation unit further comprises:

[0055] The third processing module is configured to perform feature fusion processing based on the third data in combination with residual connection, and to perform feature data capturing by using residual convolution with a step.

[0056] A third object of the present application is achieved in that the platform comprises a processor, a memory and a platform control program for muck point cloud segmentation based on point cloud convolution; wherein the processor executes the platform control program for muck point cloud segmentation based on point cloud convolution, the platform control program for muck point cloud segmentation based on point cloud convolution is stored in the memory, and the platform control program for muck point cloud segmentation based on point cloud convolution implements the method for muck point cloud segmentation based on point cloud convolution.

[0057] The present application generates and obtains first data and second data corresponding to muck point cloud segmentation processing by a method, and creates a first geometric relationship of the first data and the second data, generates corresponding third data based on the first corresponding relationship; wherein the first data is three-dimensional point cloud data corresponding to muck; the second data is kernel point data corresponding to muck; and the third data is point cloud local convolution feature data;

[0058] According to the third data, input feature data is divided and processed into multiple attention heads in combination with a multi-head self-attention mechanism, weight data corresponding to the feature data is dynamically adjusted and processed, and fourth data corresponding to the third data is generated and integrated; wherein the fourth data is output feature data of each attention head;

[0059] Based on the fourth data, point cloud data corresponding to muck is segmented and processed, and fifth data corresponding to the first data is generated; wherein the fifth data is class prediction data corresponding to muck point cloud, and a system and a platform corresponding to the method significantly improve the ability of point cloud data processing by innovatively combining local convolution feature extraction, global multi-head self-attention mechanism and effective feature upsampling strategy, and exhibit superior performance in muck point cloud segmentation tasks.

[0060] That is, the KPConvAttentionNet network proposed by the scheme of the present application significantly improves the ability of point cloud data processing by innovatively combining local convolution feature extraction, global multi-head self-attention mechanism and effective feature up-sampling strategy, especially showing superior performance in the muck point cloud segmentation task. The local convolution module (KPConv) uses a deformable convolution kernel to adapt to complex geometric structures, ensuring the flexibility and accuracy of feature extraction. At the same time, the multi-head self-attention mechanism effectively enhances the modeling ability of the model for long-distance dependencies, promoting the perception of global structure. Through the design of up-sampling and full connection layer, we successfully realize the recovery of high-resolution features and accurate class prediction. Experimental results show that KPConvAttentionNet has good robustness and accuracy in the muck point cloud segmentation task, providing an important reference and reference for future research in the field of point cloud processing. BRIEF DESCRIPTION OF DRAWINGS

[0061] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0062] Figure 1 A step flowchart of a muck point cloud segmentation processing method based on point cloud convolution according to the present application;

[0063] Figure 2 An embodiment network technology route diagram of a muck point cloud segmentation processing method based on point cloud convolution according to the present application;

[0064] Figure 3 A KPConv local convolution feature extraction module diagram of a muck point cloud segmentation processing method based on point cloud convolution according to the present application;

[0065] Figure 4 A multi-head global attention mechanism diagram of a muck point cloud segmentation processing method based on point cloud convolution according to the present application;

[0066] Figure 5 A system architecture diagram of a muck point cloud segmentation processing system based on point cloud convolution according to the present application.

[0067] Figure 6 A platform structure diagram of a muck point cloud segmentation processing platform based on point cloud convolution according to the present application. DETAILED DESCRIPTION

[0068] For better understanding the object, technical solutions and advantages of the present application, the present application is further described below in conjunction with the drawings and specific embodiments, and the skilled in the art can easily understand other advantages and effects of the present application from the disclosed content.

[0069] The present application can also be implemented or applied through other different specific examples, and various modifications and changes can be made to the details in the specification based on different views and applications without departing from the spirit of the present application.

[0070] It should be noted that if the embodiments of the present application involve directional indications (such as up, down, left, right, front, back, etc.), the directional indications are only used to explain the relative position relationship, movement condition, etc. between the components in a certain specific posture (as shown in the drawings), and if the specific posture changes, the directional indications also change accordingly.

[0071] In addition, if the embodiments of the present application involve descriptions of "first", "second", etc., the descriptions of "first", "second", etc. are only for description purposes and cannot be understood as indicating or implying the relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features limited by "first", "second" can explicitly or implicitly include at least one of the features. Secondly, the technical solutions of each embodiment can be combined with each other, but it must be based on the realization of the skilled in the art, and when the combination of technical solutions contradicts each other or cannot be realized, it should be considered that the combination of technical solutions does not exist and is not within the protection scope required by the present application.

[0072] Preferably, the present application is a kind of based on point cloud convolution's slag point cloud segmentation processing method applied in one or more terminal or server.The terminal is a kind of equipment capable of carrying out numerical calculation and / or information processing automatically according to pre-set or stored instruction, its hardware includes but is not limited to microprocessor, application specific integrated circuit (ASIC), field programmable gate array (FPGA), digital signal processor (DSP), embedded device, etc.

[0073] The terminal can be desktop computer, notebook, palm computer and cloud server and other computing devices.The terminal can be interacted with customer through keyboard, mouse, remote controller, touchpad or sound control device and other ways.

[0074] The present application is a kind of based on point cloud convolution's slag point cloud segmentation processing method, system, platform and storage medium.

[0075] As Figure 1 shown in the figure is a flowchart of a method for slag point cloud segmentation based on point cloud convolution provided by an embodiment of the present application.

[0076] In this embodiment, the method for slag point cloud segmentation based on point cloud convolution can be applied to a terminal or a fixed terminal with display function, and the terminal is not limited to a personal computer, a smart phone, a tablet computer, a desktop computer or an all-in-one computer with a camera, etc.

[0077] The method for slag point cloud segmentation based on point cloud convolution can also be applied to a hardware environment composed of a terminal and a server connected to the terminal through a network. The network includes but is not limited to a wide area network, a metropolitan area network or a local area network. The method for slag point cloud segmentation based on point cloud convolution of the embodiment of the present application can be executed by the server, or by the terminal, or by both the server and the terminal.

[0078] For example, for a terminal that needs to perform slag point cloud segmentation based on point cloud convolution, the terminal can directly integrate the slag point cloud segmentation function based on point cloud convolution provided by the method of the present application, or install a client for implementing the method of the present application. For another example, the method provided by the present application can also run on a device such as a server in the form of a software development kit (Software Development Kit, SDK), and the SDK provides an interface for the slag point cloud segmentation function based on point cloud convolution, and the terminal or other devices can realize the slag point cloud segmentation function based on point cloud convolution through the provided interface. The present application will be further described in conjunction with the accompanying drawings.

[0079] The present application will be further described in detail below in conjunction with the accompanying drawings, as Figures 1-4 shown, the present application provides a method for slag point cloud segmentation based on point cloud convolution, which comprises the following steps:

[0080] S01, generating and acquiring first data and second data corresponding to slag point cloud segmentation processing, and creating a first geometric relationship of the first data and the second data, and generating corresponding third data based on the first corresponding relationship; wherein the first data is three-dimensional point cloud data corresponding to slag; the second data is kernel point data corresponding to slag; and the third data is point cloud local convolution feature data;

[0081] S02, according to the third data, the input feature data is divided into multiple attention heads by combining the multi-head self-attention mechanism, the weight data corresponding to the feature data is dynamically adjusted, and the fourth data corresponding to the third data is generated and integrated; wherein the fourth data is the output feature data of each attention head;

[0082] S03, based on the fourth data, the point cloud data corresponding to the slag is segmented, and the fifth data corresponding to the first data is generated; wherein the fifth data is the category prediction data corresponding to the slag point cloud.

[0083] The first data and the second data corresponding to the slag point cloud segmentation processing are generated and obtained, and the first geometric relationship between the first data and the second data is created, based on the first corresponding relationship, the corresponding third data is generated, and the method further comprises:

[0084] S011, combined with the weight function, the sixth data between the first data and the second data is calculated and generated; wherein the sixth data is distance weight data; the calculation formula is as follows:

[0085]

[0086] In the formula, h(p j ,k j ) is the kernel function; p j is the point in the neighborhood;

[0087] S012, taking the rigid kernel as the benchmark, the shift corresponding to each convolution position is learned and processed, and the corresponding third data is generated; wherein the learning and processing calculation formula is as follows:

[0088]

[0089] In the formula, g deform is a deformable convolution kernel function; Δ(p i ) is the offset learned at point p i ; h(p j ,k j + Δ j (p i )) represents the distance weight between point p i and the kernel point k j + Δ j (p i ) after offset; Δ j (p i ) is the offset of kernel point k j at convolution position p i , which is learned through the convolution network; W j is the learnable weight of kernel point j.

[0090] The first data and the second data corresponding to the slag to be point cloud segmentation processing are generated and acquired, and a first geometric relationship of the first data and the second data is created, based on the first corresponding relationship, corresponding third data is generated, and the method further comprises:

[0091] S013, based on the third data, combining residual connection for feature fusion processing, and using residual convolution with step for feature data capture.

[0092] According to the third data, the input feature data is divided into multiple attention heads by combining the multi-head self-attention mechanism, the weight data corresponding to the feature data is dynamically adjusted, and the fourth data corresponding to the third data is generated and integrated, and the method further comprises:

[0093] S021, based on the third data, at least one subspace corresponding to the feature space is divided and processed, and different linear transformations are combined to create a first matrix, a second matrix and a third matrix respectively; wherein the first matrix is a query matrix; the second matrix is a key matrix; the third matrix is a value matrix;

[0094] S022, the dot product data between the first matrix and the second matrix is calculated and acquired, and based on the dot product data, corresponding seventh data is generated; wherein the seventh data is the correlation data between points and points; the calculation formula is as follows:

[0095]

[0096] In the formula, A h is a weight matrix, Q h is a query matrix; K h is a key matrix; d k is the dimension of the query and key vector;

[0097] S023, according to the weight matrix, combining the third matrix, generating and integrating the fourth data corresponding to the attention head.

[0098] The fourth data is based on the fourth data, and the point cloud data corresponding to the slag is segmented and processed, and the fifth data corresponding to the first data is generated, and the method further comprises:

[0099] S031, sampling and acquiring eighth data, according to the eighth data, and combining the nearest neighbor interpolation method to process sparse and irregular distributed point cloud data; wherein the eighth data is the fused feature data;

[0100] S032, based on the full connection layer, the fifth data corresponding to the eighth data is generated, and the fifth data is mapped to the output space of each category.

[0101] Specifically, in the embodiments of the present application, in order to overcome the shortcomings of KPConv in global information modeling and multi-scale feature capture, the present application improves the KPConv. First, the present application introduces a global self-attention mechanism in the intermediate layer of KPConv, enhancing the ability of KPConv to capture long-distance dependencies. The self-attention mechanism calculates the global correlation between each point and other points, so that each point can refer to global information when updating features, thus making up for the shortcomings of KPConv in global information extraction. By introducing the self-attention mechanism, the model can better capture the overall changes of the shield muck accumulation form and enhance the perception of the global structure, which is particularly important for processing the dynamic changes of the muck form in shield tunnel construction.

[0102] The present application proposes a point cloud segmentation network structure based on KPConv, named KPConvAttentionNet. It combines local convolution feature extraction and global multi-head attention mechanism, aiming to enhance the model's ability to capture complex geometric structures and long-distance dependencies. The entire network adopts a hierarchical design, combining KPConv convolution, simple convolution and residual connection, and step residual convolution to gradually extract and strengthen local and global features in point clouds.

[0103] During feature extraction, KPConv convolution is first applied to extract the geometric information and local neighborhood features of the point cloud. Then, the features extracted by KPConv are further refined through simple convolution layers to capture more detailed information. After several simple convolution layers, a residual connection layer is added to strengthen the transmission and aggregation of features through skip connection. Finally, step residual convolution is used to achieve downsampling at a certain level, gradually reducing the number of points and expanding the receptive field of features, so that the model can capture larger range of structural information. In the middle layer (i.e. after a certain step residual convolution layer), a global multi-head attention mechanism is embedded to enable the model to capture long-distance inter-point dependencies in a global range, thus more comprehensively understanding the spatial structure of the point cloud. The number of heads in the multi-head attention mechanism is optimized through experiments to balance the feature expression ability and computational cost. Next, the global features extracted by the global multi-head attention mechanism are fused with the local features of the KPConv convolution, enabling the model to build a stronger feature representation based on both local and global information. In the feature reconstruction phase, the resolution of the point cloud is gradually restored to preserve local details, thus achieving more fine-grained segmentation. Finally, the output layer of the network generates class predictions for each point, completing the high-precision segmentation of the point cloud. The network technology roadmap proposed by the present application is shown in Figure 2 .

[0104] In the present application, the method provided includes the following processing process:

[0105] Local convolution feature extraction module (KPConv): The basic unit of KPConvAttentionNet is the local convolution feature extraction module, which is based on KPConv convolution and realizes the convolution operation by defining discrete kernel points on irregular point clouds. The input is a three-dimensional point cloud P = {p1, p1, p3... p k}, where each point p∈d is a d-dimensional feature vector, denoted as f j , containing geometric position information or other features.

[0106] KPConv defines K kernel points, which are distributed in the neighborhood of each convolution region. Suppose the neighborhood of point p i is represented as N(p i ), the neighborhood points are represented as p j , and the kernel point set is

[0107] In order to calculate the geometric relationship between point p i and kernel point k j , KPConv introduces a weight function to calculate the distance weight between points and kernel points. This weight function determines the contribution of each point in the neighborhood to the convolution operation. The traditional KPConv uses a linear piecewise function as the weight function expression as follows:

[0108]

[0109] The biggest advantage of the Gaussian distribution weight function is its smoothness and continuity. It can smoothly decay the distance weight between points and kernel points in space, and will not suddenly cut off the weight to 0 at a certain distance like the linear piecewise function. Because the weight is exponentially decaying, the Gaussian function makes even the points at a far distance still have a weak influence on the convolution result. This is very useful for capturing more complex geometric relationships in point cloud data, especially when the model is expected to have certain global perception ability. The Gaussian function performs more naturally and smoothly, which helps to extract features for scenes with strong continuity. The weight function is as follows:

[0110]

[0111] h(p j ,k j ) is the kernel function, which calculates the distance weight between points and kernel points, p j is the point in the neighborhood. Usually, it is calculated according to the distance. KPConv optimizes the extraction of local features by learning the kernel point weight . The output local feature F local can capture the local geometric structure information of the point cloud.

[0112] To better adapt to the complex geometry in point clouds, KPConv introduces a deformable kernel. In traditional KPConv, the positions of kernel points are fixed and cannot adapt to changes in different geometric shapes. However, in complex scenes, the local geometry of point clouds may have large irregularities, and fixed-position convolution kernels may not be able to capture such geometric changes. The offset Δk j of the kernel point is a learnable parameter, allowing the convolution kernel to adaptively adjust to different geometric structures.

[0113] The final position of the kernel point is:

[0114] k j ′=k j +Δk j (3)

[0115] k j ′ is the offset kernel point position. The kernel point is learned as a parameter. In convolution, start with a rigid kernel and learn the shift Δ(k) of each convolution position. The formula is as follows:

[0116]

[0117] The deformable kernel is defined as:

[0118]

[0119] where (F*g)(p i ) represents the convolution operation result at point p i . N(p i ) is the neighborhood of point p i , containing points p j nearby. g deform is the deformable convolution kernel function. Δ(p i ) is the learned offset at point p i . f j is the input feature vector of point p j in the neighborhood. h(p j ,k j +Δ j (p i )) represents the distance weight between point p j and the offset kernel point k j +Δ j (p i ). Δ j (p i ) is the kernel point k j in the convolution position p ioffsets are learned by convolutional networks. W j are the learnable weights of the kernel point j. During training, the rigid kernel learns the offsets Δ(k) by backpropagation, while the deformable kernel generates new features. The deformable kernel points are constrained to the kernel region using special regularization. The regularization consists of a fitting term and a repulsion term. L1(p i ) fitting term ensures that each kernel point stays close enough to the input neighbors. L2(p i ) repulsion term ensures that kernel points do not collapse into each other. It is defined as the sum of the inverse square of the distance between all kernel points k j and other kernel points k l and ensures that their influence regions do not overlap.

[0120] A new loss function L new is introduced:

[0121] L new =∑L1(x)+L2(x)(6)

[0122] L1(p i ) and L2(p i ) are:

[0123]

[0124] k j +Δk j is the kernel point position after offset at point p i . σ is a hyperparameter that controls the distance decay.

[0125] k j +Δ j (p i ) and k l +Δ l (p i ) represent the position of the jth and lth kernel point after offset, respectively.

[0126] After introducing the deformable kernel, the convolution operation formula becomes:

[0127]

[0128] where f i ' is the output feature vector of point p i after the KPConv convolution operation. f j is the input feature vector of point p j from the neighborhood N(p i ) of p i . In summary, the convolution operation of KPConv is performed by taking the neighborhood N(p i ) of each point p i) is performed, and finally the local features F' of the entire point cloud of the current KPConv convolution layer are aggregated local , which is specifically expressed as:

[0129]

[0130] The above method is a method of a single KPConv convolution layer. In the KPConv encoder, KPConv, simple convolution, residual connection, and step residual convolution are combined in a hierarchical manner: first, the local features of the point cloud are extracted using KPConv convolution, and then simple convolution is used to further enrich the feature expression; then, residual connection is added for feature fusion to ensure information transmission in deep layers. The down-sampling layer uses step residual convolution to complete down-sampling, gradually reduces the resolution and expands the receptive field, thereby preserving important geometric structures while effectively capturing global features. The simple convolution formula is:

[0131] F simple = σ (W * F local + b) (11)

[0132] where F simple is the feature generated by simple convolution. W is the convolution kernel, b is the bias term, and σ is the LeakyReLU activation function.

[0133] Subsequently, residual connection is added for feature fusion to ensure information transmission in deep layers. Residual connection adds the input feature and the convolution output through a jump connection, which alleviates the gradient vanishing problem, and the formula is:

[0134] F residual = F input + F simple (12)

[0135] where F input is the input feature, and F residual is the output feature after residual connection, which can also be the feature obtained from the previous layer. This structure enables the network to retain important information in multiple layers of features and improve the learnability of the features.

[0136] The down-sampling layer uses step residual convolution for down-sampling, gradually reduces the resolution and expands the receptive field, preserves important geometric structures, and effectively captures global features. The step convolution operation achieves feature down-sampling by reducing the size of the input feature, and the formula is:

[0137] F downsampled = resnetb_strided (F residual ) (13)

[0138] where resnetb_strided is a convolution with a step, and Fdownsampled is the down-sampled feature representation. Through this hierarchical structure combination, KPConvAttentionNet can effectively handle complex point cloud data and achieve efficient feature extraction and segmentation. Through multi-layer KPConv convolution, simple convolution, residual connection, and residual connection with step, the final feature output is F local . The schematic diagram of the KPConv local convolution feature extraction module is shown in Figure 3

[0139] Global multi-head attention mechanism feature extraction: In the KPConvAttentionNet network, the global multi-head self-attention mechanism is embedded in the middle layer of KPConv to enhance the model's ability to model long-distance dependencies and its ability to perceive global structure. The core of the multi-head self-attention mechanism is to divide the input features into multiple attention heads, each of which can independently calculate the correlation between features and dynamically adjust the weights of the features. This method not only allows the model to capture the mutual dependence between long-distance points in point cloud data, but also enables the parallel extraction of diverse geometric features in different attention heads, thereby enhancing the richness and comprehensiveness of feature expression.

[0140] Attention head division: In the multi-head self-attention mechanism, the feature space is divided into H subspaces, and each attention head h will independently process the input features. This process maps the input features to different subspaces of queries, keys, and values through different linear transformations, ensuring that each head can focus on capturing different feature representations. Specifically, the input features generate a query matrix Q, a key matrix K, and a value matrix V through three different linear transformations, and the formula is:

[0141]

[0142] where and are the linear transformation matrices defined for the hth head, and F' local represents the feature of the intermediate layer of KPConv. These transformations project the input features into a low-dimensional space, allowing each head to learn independently in its own feature subspace.

[0143] Attention weight calculation: After determining the feature representation of each attention head, the next step is to calculate the attention weights. For the hth head, the correlation between points and points is obtained by calculating the dot product between the query matrix Q h and the key matrix K h . Specifically, the calculation formula is as follows:

[0144]

[0145] In this formula, d k ​is the dimension of the query and key vectors, acting as a scaling factor to avoid large dot product values. After Softmax normalization, the resulting weight matrix A h reflects the relative importance between input points, with each row summing to 1.

[0146] Weighted sum of features: In KPConv, the formula for computing local features is:

[0147]

[0148] where f i ′ is the feature extracted by the KPConv convolution operation. In the global self-attention mechanism, the resulting attention weight matrix A h is used to perform a weighted sum on the value matrix V h , generating the output feature Z h for each attention head:

[0149] Z h = A h V h (17)

[0150] Combining the local features obtained from KPConv, the final feature representation can be computed by the following formula:

[0151] F = βZ h + (1 - β)F local (18)

[0152] where β is a hyperparameter used to adjust the weights of global features Z h and local features F local . This weighting mechanism allows the model to dynamically adjust the influence of local and global information, enhancing the feature representation capability.

[0153] Integration of multi-head outputs: Finally, the output features of all attention heads are concatenated and transformed linearly to obtain the final feature representation:

[0154] Z = Concat(Z1, Z2,..., Z H )W O (19)

[0155] where W O is the linear transformation matrix used to integrate the multi-head outputs. Through this mechanism, each attention head can learn different feature spaces independently, capturing diverse geometric features in the point cloud.

[0156] Feature upsampling and slag point cloud segmentation: After completing the fusion of global multi-head self-attention mechanism and local features, the next step is to achieve the segmentation of the belt slag point cloud through upsampling and fully connected layers. The goal of this process is to restore the feature mapping to the same resolution as the original slag point cloud, so as to generate accurate class predictions for each point, thereby improving the accuracy and robustness of point cloud segmentation. Multi-head global attention mechanism Figure 4 is shown.

[0157] Feature upsampling: In order to restore the resolution of the slag point cloud, the first step is to upsample the fused features. We choose the nearest neighbor interpolation method. This method is simple and efficient, and is suitable for processing sparse and irregularly distributed point cloud data.

[0158] Let the feature matrix after global and local feature fusion be Z, with dimensions n x d, where n represents the number of points and d represents the feature dimension. These features contain local geometric information and global structural features of the slag. Choose an upsampling factor s to determine the resolution of the output feature map. For each target coordinate (x', y'), calculate its corresponding source coordinate (x, y) in the source feature map as follows:

[0159]

[0160] In this way, the source of each upsampled point can be determined. Assign a value to each target point (x', y'):

[0161] F upsampled (x', y') = F(x, y) (21)

[0162] This method ensures that feature information is effectively preserved during upsampling and ensures that each point in the output feature matches the corresponding point in the original point cloud.

[0163] The final upsampling operation can be represented as:

[0164] F upsampled = Upsample(F) (22)

[0165] where F is the feature representation after global and local feature fusion, F upsampled is the feature after upsampling operation.

[0166] F seg = FC(F upsampled ) (23)

[0167] Fully connected layer: After upsampling the features, the next step is to input these upsampled features into a fully connected layer to generate class predictions for each point. The role of the fully connected layer is to process the upsampled features and map them to the output space of each class. The specific process can be represented as:

[0168] F seg = FC(F upsampled )(24)

[0169] In this formula, F seg is the segmentation output for each point, containing the class probability distribution for each point, and FC represents the operation of the fully connected layer. The fully connected layer integrates the upsampled features, extracts high-dimensional feature information, and helps the model generate accurate classification results.

[0170] Classification output: Finally, the output of the fully connected layer is converted into class predictions for each point using the Softmax function:

[0171] P = Softmax(F seg )(25)

[0172] Where P is the class probability distribution for each point in the point cloud, ensuring that the sum of all class probabilities is 1. Through this process, the model can generate accurate class predictions for each input point, achieving high-precision slag point cloud segmentation.

[0173] The KPConvAttentionNet network proposed in the present solution significantly improves the ability to process point cloud data by innovatively combining local convolution feature extraction, global multi-head self-attention mechanism, and effective feature upsampling strategies, especially exhibiting superior performance in slag point cloud segmentation tasks. The local convolution module (KPConv) uses a deformable convolution kernel to adapt to complex geometric structures, ensuring the flexibility and accuracy of feature extraction. At the same time, the multi-head self-attention mechanism effectively enhances the model's ability to model long-distance dependencies, promoting global structure perception. Through the design of upsampling and fully connected layers, we successfully achieve high-resolution feature restoration and accurate class prediction. Experimental results show that KPConvAttentionNet has good robustness and accuracy in slag point cloud segmentation tasks, providing an important reference and reference for future research in the field of point cloud processing.

[0174] To achieve the above purpose, the present application also provides a slag point cloud segmentation processing system based on point cloud convolution, as shown in Figure 5 The system is applied to the slag point cloud segmentation processing method based on point cloud convolution, and the system comprises:

[0175] The first data generation unit is used to generate and acquire first data and second data corresponding to the point cloud segmentation processing of the slag, and to create a first geometric relationship between the first data and the second data, and to generate corresponding third data based on the first correspondence relationship; wherein, the first data is three-dimensional point cloud data corresponding to the slag; the second data is kernel point data corresponding to the slag; and the third data is point cloud local convolution feature data.

[0176] The second data generation unit is used to divide the input feature data into multiple attention heads based on the third data and in conjunction with a multi-head self-attention mechanism, dynamically adjust the weight data corresponding to the feature data, and generate and integrate the fourth data corresponding to the third data; wherein the fourth data is the output feature data of each attention head.

[0177] The third data generation unit is used to segment and process the point cloud data corresponding to the construction waste based on the fourth data, and generate the fifth data corresponding to the first data; wherein, the fifth data is the category prediction data corresponding to the construction waste point cloud.

[0178] The first data generation unit further includes:

[0179] The first generation module is used to calculate and generate a sixth data point between the first data point and the second data point by combining a weighting function; wherein, the sixth data point is distance weighted data; the calculation formula is as follows:

[0180]

[0181] In the formula, h(p) j ,k j ) is the kernel function; p j It is a point in the neighborhood;

[0182] The second generation module is used to learn the shift corresponding to each convolution position based on the rigid kernel, and generate the corresponding third data; the learning calculation formula is as follows:

[0183]

[0184] In the formula, g deform It is a deformable convolution kernel function; Δ(p i ) is at point p i The offset learned at h(p) j ,k j +Δ j (p i )) represents point p i With the offset kernel point k j +Δ j (pi ) is the distance weight between the kernel point k j (p i ) is the kernel point k j The offset of the convolution position p i is learned by the convolution network; W j is the learnable weight of the kernel point j

[0185] And / or, the second data generation unit further comprises:

[0186] The first creation module is configured to divide at least one subspace corresponding to a feature space based on the third data, and create a first matrix, a second matrix and a third matrix in combination with different linear transformations; the first matrix is a query matrix; the second matrix is a key matrix; and the third matrix is a value matrix.

[0187] The third generation module is configured to calculate and obtain dot product data between the first matrix and the second matrix, and generate corresponding seventh data based on the dot product data; the seventh data is correlation data between points.

[0188]

[0189] In the formula, A h is a weight matrix, Q h is a query matrix; K h is a key matrix; d k is the dimension of the query and key vectors.

[0190] The fourth generation module is configured to generate and integrate fourth data corresponding to an attention head in combination with the third matrix based on the weight matrix.

[0191] And / or, the third data generation unit further comprises:

[0192] The first processing module is configured to sample and obtain eighth data, and process sparse and irregularly distributed point cloud data in combination with a nearest neighbor interpolation method based on the eighth data; the eighth data is fused feature data.

[0193] The second processing module is configured to generate fifth data corresponding to the eighth data based on a fully connected layer, and map the fifth data to an output space of each category.

[0194] The first data generation unit further comprises:

[0195] The third processing module is configured to perform feature fusion processing in combination with a residual connection based on the third data, and perform feature data capture by using a residual convolution with a step.

[0196] In the system scheme embodiment of the present application, the method steps involved in the slag point cloud segmentation processing based on point cloud convolution have been described in detail above, that is, the function modules in the system are used to realize the steps or sub-steps in the above method embodiments, which will not be described here.

[0197] To achieve the above object, the present application also provides a slag point cloud segmentation processing platform based on point cloud convolution, as shown in Figure 6 The processor executes the slag point cloud segmentation processing platform control program based on point cloud convolution, the slag point cloud segmentation processing platform control program based on point cloud convolution is stored in the memory, and the slag point cloud segmentation processing platform control program based on point cloud convolution realizes the method steps of the slag point cloud segmentation processing based on point cloud convolution. For example:

[0198] S01, generate and obtain first data and second data corresponding to slag point cloud segmentation processing, and create a first geometric relationship of the first data and the second data, generate corresponding third data based on the first corresponding relationship; wherein the first data is three-dimensional point cloud data corresponding to slag; the second data is core point data corresponding to slag; the third data is point cloud local convolution feature data;

[0199] S02, according to the third data, combine the multi-head self-attention mechanism to divide the input feature data into multiple attention heads, dynamically adjust the weight data corresponding to the feature data, and generate and integrate the fourth data corresponding to the third data; wherein the fourth data is the output feature data of each attention head;

[0200] S03, based on the fourth data, segment the point cloud data corresponding to the slag, and generate the fifth data corresponding to the first data; wherein the fifth data is the category prediction data corresponding to the slag point cloud.

[0201] The specific steps have been described above, which will not be described here.

[0202] In the embodiment of the present application, the processor built in the slag point cloud segmentation processing platform based on point cloud convolution can be composed of integrated circuits, for example, can be composed of a single packaged integrated circuit, or can be composed of multiple packaged integrated circuits with the same function or different functions, including one or more combinations of central processing units (CPU), microprocessors, digital processing chips, graphics processors and various control chips, etc. The processor connects various components through various interfaces and lines, executes programs or units stored in the memory, and calls data stored in the memory to perform various functions and process data of the slag point cloud segmentation processing based on point cloud convolution.

[0203] The memory is used to store program codes and various data, is installed in the slag point cloud segmentation processing platform based on point cloud convolution, and realizes high-speed and automatic access of programs or data during running.

[0204] The memory includes read-only memory (ROM), random access memory (RAM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), one-time programmable read-only memory (OTPROM), electrically erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc memory, disk memory, tape memory, or any other computer-readable medium capable of carrying or storing data.

[0205] The application generates and obtains first data and second data corresponding to slag soil point cloud segmentation processing by a method, and creates a first geometric relationship of the first data and the second data, generates corresponding third data based on the first corresponding relationship; wherein the first data is three-dimensional point cloud data corresponding to slag soil; the second data is core point data corresponding to slag soil; the third data is point cloud local convolution feature data; according to the third data, the input feature data is divided and processed into multiple attention heads in combination with a multi-head self-attention mechanism, the weight data corresponding to the feature data is dynamically adjusted and processed, and the fourth data corresponding to the third data is generated and integrated; wherein the fourth data is output feature data of each attention head; based on the fourth data, the point cloud data corresponding to the slag soil is segmented and processed, and the fifth data corresponding to the first data is generated; wherein the fifth data is category prediction data corresponding to the slag soil point cloud, and a system and a platform corresponding to the method, by innovatively combining local convolution feature extraction, global multi-head self-attention mechanism and effective feature upsampling strategy, significantly improve the ability of point cloud data processing, especially in the slag soil point cloud segmentation task, which shows superior performance.

[0206] That is, the KPConvAttentionNet network proposed by the application scheme significantly improves the ability of point cloud data processing by innovatively combining local convolution feature extraction, global multi-head self-attention mechanism and effective feature upsampling strategy, especially in the slag soil point cloud segmentation task, which shows superior performance. The local convolution module (KPConv) uses a deformable convolution kernel to adapt to complex geometric structures, ensuring the flexibility and accuracy of feature extraction. At the same time, the multi-head self-attention mechanism effectively enhances the modeling ability of the model for long-distance dependency relationships, promoting the perception of global structure. Through the design of upsampling and full connection layer, we successfully realize the recovery of high-resolution features and accurate category prediction. Experimental results show that KPConvAttentionNet has good robustness and accuracy in the slag soil point cloud segmentation task, providing an important reference for future research in the field of point cloud processing.

[0207] The above-described embodiments only express several embodiments of the application, which are described in detail and specifically, but should not be understood as limiting the scope of the patent of the application. It should be noted that for ordinary skilled persons in the art, without departing from the concept of the application, several modifications and improvements can be made, which are within the protection scope of the application. Therefore, the protection scope of the patent of the application should be subject to the appended claims.

Claims

1. A method for segmenting and processing construction waste point clouds based on point cloud convolution, characterized in that, The method includes the following steps: First data and second data corresponding to the point cloud segmentation of the slag are generated and acquired, and a first geometric relationship between the first data and the second data is created. Based on the first correspondence, corresponding third data is generated. The first data is three-dimensional point cloud data corresponding to the slag; the second data is kernel point data corresponding to the slag; and the third data is point cloud local convolution feature data. Based on the third data, the input feature data is divided into multiple attention heads using a multi-head self-attention mechanism. The weight data corresponding to the feature data is dynamically adjusted, and a fourth data corresponding to the third data is generated and integrated. The fourth data is the output feature data of each attention head. Based on the fourth data, the point cloud data corresponding to the construction waste is segmented and processed, and a fifth data corresponding to the first data is generated; wherein, the fifth data is category prediction data corresponding to the construction waste point cloud.

2. The method for segmenting and processing construction waste point clouds based on point cloud convolution according to claim 1, characterized in that, The process of generating and acquiring first and second data corresponding to the point cloud segmentation of the construction waste, creating a first geometric relationship between the first and second data, and generating corresponding third data based on the first correspondence relationship, further includes: By combining the weighting function, a sixth data point is calculated between the first data point and the second data point; wherein, the sixth data point is distance-weighted data; the calculation formula is as follows: In the formula, h(p) j ,k j ) is the kernel function; p j It is a point in the neighborhood; k j It is the kernel point; σ is the standard deviation of the Gaussian function, representing the scaling parameter of the weight decay; Using a rigid kernel as a baseline, the learning process is applied to the shift corresponding to each convolutional position, generating corresponding third data. The calculation formula for the learning process is as follows: In the formula, g deform It is a deformable convolution kernel function; Δ(p i ) is at point p i The offset learned at point; h(p) j ,k j +Δ j (p i )) represents point p i With the offset kernel point k j +Δ j (p i Distance weights between ) ; Δ j (p i ) is the kernel point k j At convolution position p i The offset is learned through a convolutional network; W j K is the learnable weight of kernel point j; K is the number of kernel points.

3. A method for segmenting and processing slag point clouds based on point cloud convolution according to claim 1 or 2, characterized in that, The process of generating and acquiring first and second data corresponding to the slag to be segmented and processed from the point cloud, creating a first geometric relationship between the first and second data, and generating corresponding third data based on the first correspondence relationship, further includes: Based on the third data, feature fusion processing is performed using residual connections, and feature data capture is achieved using residual convolution with stride.

4. The method for segmenting and processing construction waste point clouds based on point cloud convolution according to claim 1, characterized in that, The step of dividing the input feature data into multiple attention heads based on the third data and using a multi-head self-attention mechanism, dynamically adjusting the weight data corresponding to the feature data, and generating and integrating the fourth data corresponding to the third data further includes: Based on the third data, at least one subspace corresponding to the feature space is divided and processed, and a first matrix, a second matrix, and a third matrix are created by combining different linear transformations; wherein, the first matrix is ​​a query matrix; the second matrix is ​​a key matrix; and the third matrix is ​​a value matrix. Calculate and obtain the dot product data between the first matrix and the second matrix, and generate corresponding seventh data based on the dot product data; wherein, the seventh data is the correlation data between points; the calculation formula is as follows: In the formula, A h Let Q be the weight matrix. h For query matrix; K h d is the key matrix; k It is the dimension of the query and key vector; Based on the weight matrix and combined with the third matrix, fourth data corresponding to the attention head is generated and integrated.

5. The method for segmenting and processing construction waste point clouds based on point cloud convolution according to claim 1, characterized in that, The step of segmenting and processing the point cloud data corresponding to the slag and soil based on the fourth data, and generating the fifth data corresponding to the first data, further includes: The eighth data is obtained by sampling. Based on the eighth data, the sparse and irregularly distributed point cloud data is processed by the nearest neighbor interpolation method. The eighth data is the fused feature data. Based on the fully connected layer, fifth data corresponding to the eighth data is generated, and the fifth data is mapped to the output space of each category.

6. A system for segmenting and processing construction waste point clouds based on point cloud convolution, characterized in that, The system is applied to the point cloud segmentation and processing method for slag and soil based on point cloud convolution as described in any one of claims 1 to 5, and the system includes: The first data generation unit is used to generate and acquire first data and second data corresponding to the point cloud segmentation processing of the slag, and to create a first geometric relationship between the first data and the second data, and to generate corresponding third data based on the first correspondence relationship; wherein, the first data is three-dimensional point cloud data corresponding to the slag; the second data is kernel point data corresponding to the slag; and the third data is point cloud local convolution feature data. The second data generation unit is used to divide the input feature data into multiple attention heads based on the third data and in conjunction with a multi-head self-attention mechanism, dynamically adjust the weight data corresponding to the feature data, and generate and integrate the fourth data corresponding to the third data; wherein the fourth data is the output feature data of each attention head. The third data generation unit is used to segment and process the point cloud data corresponding to the construction waste based on the fourth data, and generate the fifth data corresponding to the first data; wherein, the fifth data is the category prediction data corresponding to the construction waste point cloud.

7. A soil and waste point cloud segmentation and processing system based on point cloud convolution according to claim 6, characterized in that, The first data generation unit further includes: The first generation module is used to calculate and generate a sixth data point between the first data point and the second data point by combining a weighting function; wherein, the sixth data point is distance weighted data; the calculation formula is as follows: In the formula, h(p) j ,k j ) is the kernel function; p j It is a point in the neighborhood, k j It is the kernel point; σ is the standard deviation of the Gaussian function, representing the scaling parameter of the weight decay; The second generation module is used to learn the shift corresponding to each convolution position based on the rigid kernel, and generate the corresponding third data; the learning calculation formula is as follows: In the formula, g deform It is a deformable convolution kernel function; Δ(p i ) is at point p i The offset learned at point; h(p) j ,k j +Δ j (p i )) represents point p i With the offset kernel point k j +Δ j (p i Distance weights between ) ; Δ j (p i ) is the kernel point k j At convolution position p i The offset is learned through a convolutional network; W j It is the learnable weight of kernel point j; And / or, the second data generation unit further includes: The first creation module is used to divide and process at least one subspace corresponding to the feature space based on the third data, and create a first matrix, a second matrix, and a third matrix by combining different linear transformations; wherein, the first matrix is ​​a query matrix; the second matrix is ​​a key matrix; and the third matrix is ​​a value matrix. The third generation module is used to calculate, generate, and obtain the dot product data between the first matrix and the second matrix, and generate corresponding seventh data based on the dot product data; wherein, the seventh data is the correlation data between points; the calculation formula is as follows: In the formula, A h Let Q be the weight matrix. h For query matrix; K h d is the key matrix; k It is the dimension of the query and key vector; The fourth generation module is used to generate and integrate fourth data corresponding to the attention head based on the weight matrix and the third matrix. And / or, the third data generation unit further includes: The first processing module is used to sample and acquire the eighth data, and process the sparse and irregularly distributed point cloud data based on the eighth data and the nearest neighbor interpolation method; wherein, the eighth data is the fused feature data; The second processing module is used to generate fifth data corresponding to the eighth data based on the fully connected layer, and to map the fifth data to the output space of each category.

8. A point cloud segmentation and processing system for slag and soil based on point cloud convolution according to claim 6 or 7, characterized in that, The first data generation unit further includes: The third processing module is used to perform feature fusion processing based on the third data and residual connections, and to capture feature data using residual convolution with stride.

9. A point cloud segmentation and processing platform for construction waste based on point cloud convolution, characterized in that, The system includes a processor, a memory, and a control program for a construction waste point cloud segmentation processing platform based on point cloud convolution. The control program is executed on the processor and stored in the memory. The control program implements the construction waste point cloud segmentation processing method based on point cloud convolution as described in any one of claims 1 to 5.

Images