Deep sea mineral resource segmentation method and device based on dynamic anchor points and iterative optimization

Abstract

The invention discloses a deep sea mineral resource segmentation method and device based on dynamic anchor points and iterative optimization, and relates to the field of computer vision, and the method comprises the steps: obtaining sonar point clouds and laser radar point clouds of deep sea mineral resources, combining the sonar point clouds and the laser radar point clouds into unified point clouds, and obtaining high-definition image texture features; fusing the initial geometric features and the texture features of the unified point cloud to obtain initial multi-modal features; constructing a kernel point, calculating a local structure feature and obtaining a multi-scale feature; using a graph attention network to extract global features and performing clustering to obtain class anchor points; fusing the preliminary multi-modal features, the texture features and the global features to obtain a final feature representation, calculating the similarity between the final feature representation and the class anchor points, and performing classification to obtain a preliminary segmentation result; boundary smoothing and texture correction are carried out on the preliminary segmentation result, and then fusion with multi-scale features is carried out; suspicious points are detected, error correction is carried out on the suspicious points, and segmentation is completed. According to the method, multi-modal feature iterative optimization, misclassification point continuous correction and class anchor point timely updating are realized.

Description

Technical Field

[0001] The present invention relates to the field of computer vision, and in particular to a method and device for segmenting deep-sea mineral resources based on dynamic anchor points and iterative optimization. Background Art

[0002] Accurate segmentation of deep-sea mineral resources is a crucial foundation for resource exploration, assessment, and subsequent utilization. However, the complexity and uncertainty of the deep-sea environment pose significant challenges to the research and application of related technologies. Existing deep-sea mineral resource segmentation research typically utilizes sonar and laser ranging (LiDAR) to acquire point cloud data to describe seafloor topography and structural features. High-definition imagery (RGB images) complements the point cloud's lack of color and texture information, providing a reference for resource identification and segmentation. However, this multimodal data exhibits significant differences in coordinate systems, scales, and statistical properties, and deep-sea point clouds are often accompanied by noise and uneven density. Existing research has primarily focused on data coordinate alignment, extraction of basic geometric and texture features, and primary modal fusion. Some technical solutions can achieve simple feature overlay and basic classification. However, such research often lacks a comprehensive, integrated process from data preprocessing to segmentation result verification, hindering the continuous optimization of segmentation results in complex and changing deep-sea scenarios. When faced with issues such as blurred mineral class boundaries, dynamic feature distribution, and frequent misclassified points, traditional methods lack sufficient flexibility and adaptability, making it difficult to ensure the reliability and accuracy of the final segmentation results.

[0003] In existing technologies, even if multimodal data fusion and basic feature enhancement methods are introduced, class anchor points (class centers) are often not dynamically updated after initial determination, and cannot adapt to changes in feature distribution in a timely manner. At the same time, the processing of misclassified points is mostly a single operation, lacking a strategy for continuous optimization and correction in subsequent iterations. This makes the final segmentation results prone to residual errors and instability, making it difficult to meet the requirements of accurate identification of deep-sea resources. In view of this, there is an urgent need for a complete set of technical processes, starting from multimodal data acquisition, coordinate alignment, and feature fusion, through to dynamic optimization of anchor points, multiple rounds of error point correction and adaptive adjustment of feature weights, to ensure that in a deep-sea environment where data distribution, noise level and category characteristics are constantly changing, the segmentation results can continue to approach the true distribution and improve classification accuracy and robustness. Summary of the Invention

[0004] In response to the above problems, the present invention proposes a deep-sea mineral resource segmentation method and device based on dynamic anchor points and iterative optimization. By constructing an integrated process from data preprocessing, feature fusion to final segmentation verification, introducing dynamic anchor point optimization and multi-round iterative error correction mechanism, and combining the adaptive adjustment method of feature weights, the multimodal feature distribution can be continuously optimized with iteration, the misclassification points can be continuously corrected, and the class anchor points can be updated in a timely manner in the complex deep-sea environment, thereby realizing a deep-sea mineral resource segmentation method with high precision, robustness and excellent adaptability.

[0005] On the one hand, the deep-sea mineral resource segmentation method based on dynamic anchor points and iterative optimization has the following specific steps:

[0006] S1, acquires sonar point clouds, lidar point clouds, and high-definition images of deep-sea mineral resources; aligns the coordinates of the sonar point clouds and lidar point clouds and merges them into a unified point cloud; and obtains texture features from the high-definition images.

[0007] S2, constructing a spherical neighborhood of the unified point cloud and obtaining the initial geometric features of the unified point cloud based on the spherical neighborhood;

[0008] S3, performing block processing on the unified point cloud and smoothing the overlapping areas to obtain a processed unified point cloud;

[0009] S4, the initial geometric features are fused with the texture features to obtain preliminary multimodal features;

[0010] S5, constructing the core points of the spherical neighborhood of the processed unified point cloud, and calculating the local structural features based on the core points;

[0011] S6, calculating the local structural features of the processed unified point cloud under a set of spherical neighborhood radii of different sizes and splicing them to obtain multi-scale features;

[0012] S7, converts multi-scale features into graph structure data and uses graph attention network to extract global features from graph structure data;

[0013] S8, cluster the global features to obtain class anchors;

[0014] S9, fuses preliminary multimodal features, texture features and global features to obtain the final feature representation;

[0015] S10, calculate the similarity between the final feature representation and the class anchor, classify according to the similarity, and obtain the preliminary segmentation result;

[0016] S11, performing boundary smoothing and texture correction on the preliminary segmentation result to obtain a corrected feature representation;

[0017] S12, fuses multi-scale features and corrected feature representation to obtain enhanced segmentation results;

[0018] S13, detecting suspicious points in the enhanced segmentation result, performing error correction on the suspicious points, obtaining a corrected segmentation result, and completing the segmentation of deep-sea mineral resources.

[0019] Preferably, the initial geometric features and texture features are fused to obtain preliminary multimodal features, specifically as follows:

[0020] Normalize the initial geometric features and texture features of the unified point cloud;

[0021] The standardized initial geometric features are fused with the texture features through weighted summation to obtain the preliminary multimodal features, which can be expressed as:

[0022] f′(p)=εf norm (p)+τt norm (p)

[0023] Among them, f′(p) represents the preliminary multimodal features; f norm (p) represents the initial geometric features after normalization; t norm (p) represents the normalized texture feature; ε and τ represent f norm (p) and t norm The weight of (p), ε+τ=1; p represents the unified point cloud point p.

[0024] Preferably, the core points of the spherical neighborhood of the processed unified point cloud are constructed, and local structural features are calculated based on the core points, specifically as follows:

[0025] The kernel points in the spherical neighborhood of each point in the unified point cloud after calculation are expressed as:

[0026]

[0027] Among them, k p represents the core point of point p; N(p) represents the spherical neighborhood of point p, q represents the neighborhood point of point p, q∈N(p); w k (q) represents the weight of generating kernel points; σ1 represents the Gaussian parameter of generating kernel points; |||| 2 represents the square of the Euclidean distance;

[0028] Calculate the explicit geometric relationship between the kernel point and the points in its neighborhood, expressed as:

[0029] g(q,k p )=[|qk p |,cos(θ)]

[0030] Among them, g(q,k p ) represents q and k pThe explicit geometric relationship of cos(θ)=n(q)·n(k p ), n(q) represents the normal vector of point q, n(k p ) represents the core point k p Normal vector of |qk p | represents q and k p distance;

[0031] Aggregate the neighborhood relationship features of the core points to obtain local structural features; expressed as:

[0032]

[0033] Among them, f(k p ) represents the local structural features; w g (q) represents the feature distance weight.

[0034] Preferably, the graph attention network performs context propagation on the input graph structure data and introduces a global attention mechanism; the context propagation is expressed as:

[0035]

[0036] Among them, f s () represents the multi-scale feature vector; f s ′() represents the multi-scale feature vector after propagation; w ctx (i, j) represents the context weight; i and j represent the index of the i-th node and the j-th node in the graph structure data respectively; N(i) represents the neighborhood of the i-th node in the graph structure data; |||| 2 represents the square of the Euclidean distance; σ3 represents the bandwidth parameter of the Gaussian kernel;

[0037] The global attention mechanism maps the propagated multi-scale feature vector to the query, key, and value space, expressed as:

[0038]

[0039] Among them, F g (p i ) represents the global feature of the i-th specific point in the unified point cloud point p; α ij represents the attention weight; q i 、k j and v j Respectively, k represents the vectors that map the propagated multi-scale feature vector to query, key, and value; t Represents the t-th key vector among all keys.

[0040] Preferably, the global features are clustered to obtain class anchors, specifically as follows:

[0041] Clustering of global features is expressed as:

[0042]

[0043] in, Indicates the The center point of the class in the feature space is the class anchor point. L represents the total number of categories; Indicates the The point set of the class, express The number of elements of F g (p) represents the global feature;

[0044] The anchor points are optimized by minimizing the intra-class variance, which can be expressed as:

[0045]

[0046] in, Indicates taking the minimum value; |||| 2 Represents the square of the Euclidean distance.

[0047] Preferably, the fusion of preliminary multimodal features, texture features and global features to obtain the final feature representation is as follows:

[0048] F final (p)=αf′(p)+βF g (p)+γt(p)

[0049] Among them, F final (p) represents the final feature representation; f′(p) represents the preliminary multimodal feature; F g (p) represents the global feature; t(p) represents the texture feature; α, β and γ represent f′(p), F g The weight of (p) and t(p), α+β+γ=1.

[0050] Preferably, the similarity between the final feature representation and the class anchor is calculated, and classification is performed based on the similarity to obtain a preliminary segmentation result, as follows:

[0051] Calculate the cosine similarity between the final feature representation and the class anchor;

[0052] Classification is performed based on cosine similarity, expressed as:

[0053]

[0054] Among them, class(p) represents the class to which the unified point cloud point p belongs, that is, the preliminary segmentation result; Indicates the method that maximizes the similarity Value; Ffinal (p) represents the final feature representation; Representation category Class anchor of Indicates F final (p) and The cosine similarity of .

[0055] Preferably, the suspicious points in the detection classification result are corrected for errors to obtain a corrected segmentation result, specifically as follows:

[0056] S131, calculating the sparsity of each point; points whose sparsity exceeds a preset sparsity threshold are considered suspicious points;

[0057] S132, calculating the cosine similarity between the suspicious point and the anchor point, and reclassifying the suspicious point based on the cosine similarity;

[0058] S133, collect the suspicious points that still exist after reclassification, use the neighborhood feature mean to correct the features of the remaining suspicious points, and obtain the mean correction feature; it is expressed as:

[0059]

[0060] Among them, F upd (p) represents the mean correction characteristic; w e (p,q) represents weight; F corr (p) represents the corrected feature representation, F corr (q) represents the corrected feature representation of point q; N(p) represents the spherical neighborhood of point p; q∈N(p);

[0061] S134, dynamically adjusting the multimodal feature weights in the area where the suspicious point is located to obtain adjusted multimodal features; expressed as:

[0062] F opt (p) = γF upd (p)+δt(p)

[0063] Among them, F opt (p) represents the adjusted multimodal features; γ and δ represent F upd (p) and t(p) weights; t(p) represents texture features, γ+δ=1;

[0064] S135, calculate the feature deviation between the adjusted multimodal feature and the anchor point; if the feature deviation is greater than the preset feature deviation threshold, use the adjusted multimodal feature as the updated point feature and execute S136; otherwise, correct the adjusted multimodal feature to obtain the updated point feature, which is expressed as:

[0065]

[0066] Among them, F adj (p) represents the updated point feature; λ represents a real number parameter used to control the correction strength; Representation category Class anchor of

[0067] S136, using the updated point features to calculate the core point features, and calculating the average feature value of the core points to obtain the refined local structure features;

[0068] S137, performing multi-scale smoothing processing on the refined local structural features;

[0069] S138, repeat S131-S137 until the termination condition is reached, the error correction is completed, and the corrected segmentation result is obtained; the termination condition may be that the maximum number of iterations is met, or that the change in the number of suspicious points is less than a preset change threshold.

[0070] Preferably, after S13, the following steps are further included:

[0071] S14, detecting the uniformity of the category distribution of the segmentation results; if the category distribution uniformity is less than a preset uniformity threshold, dynamically adjusting the class anchor point; the category uniformity is based on the difference in the number of samples in each category, or based on the difference between the feature mean and the actual sample distribution; the dynamically adjusted class anchor point is expressed as:

[0072]

[0073] in, represents the class anchor after fine-tuning; represents the class anchor point; η represents the adjustment coefficient; Represents the average difference in feature values ​​of points within a class; Indicates the Class point set;

[0074] S15, recalculate the feature deviation and sparsity, take the points whose feature deviation is higher than the preset feature deviation threshold or the points whose sparsity is higher than the preset sparsity threshold as seriously misclassified points, and calculate the neighborhood average of the seriously misclassified points; the neighborhood average is expressed as:

[0075]

[0076] Among them, F sem (p) represents the neighborhood average; N(p) represents the spherical neighborhood, q∈N(p); |N(p)| represents the number of elements in N(p);

[0077] S16, the neighborhood average is fused with the global feature to obtain the second final feature representation, which is expressed as:

[0078] F final2 (p)=ωF sem (p)+μF g (p)

[0079] Among them, F final2 (p) represents the second final feature representation; ω and μ represent F sem (p) and F g The weight of (p), ω + μ = 1; F g (p) represents the global feature;

[0080] S17, calculating the similarity between the second final feature and the dynamically adjusted anchor point, performing classification based on cosine similarity, and obtaining an optimized segmentation result.

[0081] On the other hand, the deep-sea mineral resource segmentation device based on dynamic anchor points and iterative optimization includes the following:

[0082] The data acquisition module is used to obtain sonar point clouds, lidar point clouds, and high-definition images of deep-sea mineral resources; align the coordinates of sonar point clouds and lidar point clouds and merge them into a unified point cloud; and obtain texture features from high-definition images;

[0083] The geometric feature acquisition module is used to construct a spherical neighborhood of the unified point cloud and obtain the initial geometric features of the unified point cloud based on the spherical neighborhood;

[0084] A block processing module is used to perform block processing on the unified point cloud and smooth the overlapping area to obtain a processed unified point cloud;

[0085] Multimodal feature fusion module, used to fuse initial geometric features with texture features to obtain preliminary multimodal features;

[0086] The local structure feature acquisition module is used to construct the core points of the spherical neighborhood of the processed unified point cloud and calculate the local structure features based on the core points;

[0087] The multi-scale feature acquisition module is used to calculate the local structural features of the processed unified point cloud under a set of spherical neighborhood radii of different sizes and stitch them together to obtain multi-scale features;

[0088] The global feature acquisition module is used to convert multi-scale features into graph structure data and use the graph attention network to extract global features from the graph structure data;

[0089] The class anchor acquisition module is used to cluster global features and obtain class anchors;

[0090] The final feature representation acquisition module is used to fuse preliminary multimodal features, texture features and global features to obtain the final feature representation;

[0091] The preliminary segmentation module is used to calculate the similarity between the final feature representation and the class anchor, classify according to the similarity, and obtain the preliminary segmentation result;

[0092] The correction module is used to perform boundary smoothing and texture correction on the preliminary segmentation results to obtain the corrected feature representation;

[0093] The enhancement module is used to fuse multi-scale features and corrected feature representation to obtain enhanced segmentation results;

[0094] The error correction module is used to detect suspicious points in the enhanced segmentation results, perform error correction on the suspicious points, obtain the corrected segmentation results, and complete the segmentation of deep-sea mineral resources.

[0095] Compared with the prior art, the present invention has the following beneficial effects:

[0096] (1) In the segmentation iteration process of the present invention, the class anchor point (class center) can be updated as the feature distribution changes, so that it always maintains a higher degree of match with the current data features, thereby avoiding the accumulation of deviations caused by the fixed class anchor point; this process helps to make full use of the updated feature information in subsequent steps, guiding the segmentation results to be more stable and accurate;

[0097] (2) The present invention uses multiple iterations to continuously correct suspicious points, avoiding the limitation of only processing misclassified points once after the initial segmentation. When the data features change in subsequent links or new information appears, the misclassified areas can be compared for similarity, feature deviations corrected, and categories reallocated again, so that the segmentation results tend to be better in the continuous updating and correction.

[0098] (3) The present invention makes the fusion of multimodal data more flexible by automatically adjusting the weight ratio of different modal features according to the current data distribution and feature changes during the iterative process; different data sources can play a greater role at different stages according to actual conditions, so that the feature fusion process maintains high adaptability and effectiveness in the changing deep-sea environment;

[0099] (4) The present invention does not simply superimpose several independent modules, but realizes the mutual correlation and promotion of each step in the complete technical chain from data alignment and feature extraction to anchor point update, iterative correction of suspicious points and feature weight adjustment; the update of dynamic anchor points can provide a more accurate category benchmark for multiple rounds of correction of suspicious points, and the new feature information generated by multiple rounds of error correction can reversely affect the weight adjustment and anchor point distribution, thereby forming a closed-loop effect of continuous optimization in the overall process. BRIEF DESCRIPTION OF THE DRAWINGS

[0100] The present invention will be described in further detail below with reference to the accompanying drawings;

[0101] Figure 1 This is a flow chart of a deep-sea mineral resource segmentation method based on dynamic anchor points and iterative optimization according to an embodiment of the present invention;

[0102] Figure 2 This is a structural block diagram of a deep-sea mineral resource segmentation device based on dynamic anchor points and iterative optimization according to an embodiment of the present invention. DETAILED DESCRIPTION

[0103] The present invention is further described below through specific embodiments.

[0104] like Figure 1 As shown in the figure, the deep-sea mineral resource segmentation method based on dynamic anchor points and iterative optimization has the following specific steps:

[0105] S1, obtains sonar point clouds, lidar point clouds, and high-definition images of deep-sea mineral resources; aligns the coordinates of the sonar point clouds and lidar point clouds and merges them into a unified point cloud; and obtains texture features from the high-definition images.

[0106] Multimodal data is obtained from deep-sea sonar and lidar equipment, where the sonar point cloud is denoted as P s ={p s}, the lidar point cloud is recorded as P l ={p l Here p s =(x s ,y s ,z s ) and p l =(x l ,y l ,z l ) is the three-dimensional coordinate point, x s ,y s ,z s ,x l ,y l ,z l is the real coordinate value. Get high-definition image I v (u,v), where (u,v) is the pixel coordinate, I v (u,v) gives the RGB value of the pixel. Using the known 4×4 external parameter calibration matrix T s With T l Align the coordinate systems of the sonar and lidar point clouds, expressed as:

[0107] P s ′=T s ·P s ,P l ′=T l ·P l

[0108] Where · represents multiplying the point in homogeneous coordinate form with the transformation matrix. s ′ and P l ' is merged into a unified point cloud P. Define a 3×4 projection matrix M, for any point p=(x p ,y p ,z p ) is mapped to the image coordinate system (u,v), from I v (u, v) obtains the texture feature t(p) = [R, G, B], where R, G, B are the red, green, and blue components of the pixel. This step completes multimodal data alignment and texture mapping.

[0109] For each point p∈P, its k-nearest neighbor set is defined as:

[0110] N k (p)={p1,p2,…,p k}

[0111] Among them, k is a preset positive integer, p i is the i-th point closest to p. According to N k The coordinates of the point set in (p) construct the local covariance matrix and calculate its eigenvalues ​​and eigenvectors. Let the eigenvalues ​​be arranged in ascending order as follows:

[0112] λ0≤λ1≤λ2,λ0,λ1,λ2≥0

[0113] The following curvature definition is used to characterize the local curvature of the surface around point p:

[0114]

[0115] Here, λ0 is the minimum eigenvalue. When the local point cloud presents a relatively flat surface distribution, λ0 is very small and the curvature c(p) is also small. When there are sharp geometric changes in the local area, λ0 increases relatively, resulting in an increase in c(p).

[0116] Next, to measure the local sparsity around point p, the average neighborhood distance (which can be regarded as "local point spacing") is defined:

[0117]

[0118] Among them, ||pq|| is the point p=(x p ,y p ,z p ) and q=(x q ,y q ,z q ) A smaller d(p) indicates that the neighborhood of p is denser, and a larger d(p) indicates that the neighborhood is sparser.

[0119] After obtaining the curvature c(p) and average neighborhood distance d(p), the point cloud can be preprocessed based on actual needs. For example, points with abnormally high curvature and large local point spacing can be removed, as these are often noise points. After noise point removal, the farthest point sampling method is then used to reduce the number of points while maintaining a uniform spatial distribution of the point cloud.

[0120] S2, constructs a spherical neighborhood of the unified point cloud and obtains the initial geometric features of the unified point cloud based on the spherical neighborhood.

[0121] For each point p, a spherical neighborhood with radius r>0 is defined as:

[0122] N(p)={q∈P||pq||≤r}

[0123] Principal component analysis is performed on the point set within N(p) to estimate the normal vector n(p). n(p) is a three-dimensional vector of unit length that describes the normal direction of the local surface on which point p lies. The normal vector here is calculated using PCA (Principal Component Analysis). After updating the curvature c(p) and the average neighborhood distance d(p) using N(p), an initial geometric feature vector f(p) is defined, containing geometric quantities such as c(p), n(p), and d(p), which describes the local geometric properties of the point.

[0124] S3, performs block processing on the unified point cloud and smoothes the overlapping areas to obtain a processed unified point cloud.

[0125] Divide the point cloud P into several sub-blocks P1, P2, ..., P according to the spatial position n , where n is the number of sub-blocks. There are a small number of overlapping points between adjacent sub-blocks to ensure the continuity between regions. Define the sub-block center c blk is the centroid of the points in the sub-block, i.e., the average coordinate point. In order to reduce the feature discontinuity at the junction of sub-blocks, a smoothing weight is defined for the overlapping point p, which is expressed as:

[0126]

[0127] Where σ>0 is the Gaussian smoothing parameter, which is a real constant. blk (p) Perform weighted average smoothing on the point features in the overlapping area to eliminate feature jumps at the sub-block boundaries.

[0128] This embodiment divides the point cloud into blocks according to spatial positions and applies Gaussian smoothing weighting to the overlapping areas, thereby ensuring the continuity and smooth transition of features between sub-blocks. This provides robust and continuous input data for subsequent feature normalization, context propagation, and global attention construction, thereby implicitly enhancing the robustness and accuracy of the overall segmentation process.

[0129] S4, the initial geometric features are fused with the texture features to obtain preliminary multimodal features.

[0130] Normalize the geometric features f(p) and define the global geometric feature mean vector μ f and the standard deviation vector σ f , normalize each component independently, then:

[0131]

[0132] Similarly, the global mean μ of texture feature t(p) = [R, G, B] is defined as t and standard deviation σ t , standardized as:

[0133]

[0134] Assuming ε and τ are real weights and ε + τ = 1, the geometric and texture features are fused by weighting, which can be expressed as:

[0135] f′(p)=εf norm (p)+τt norm (p)

[0136] Among them, f′(p) is the preliminary multimodal feature after fusion, and ε and τ represent the weights respectively.

[0137] S5, constructing the core points of the spherical neighborhood of the processed unified point cloud, and calculating the local structural features based on the core points.

[0138] Introduce a kernel point k in the neighborhood N(p) of each point p p Characterize the local feature center. Define the kernel point generation weight as follows:

[0139]

[0140] in, The Gaussian parameters generated for the kernel points can be different from or the same as the Gaussian smoothing parameters in the smoothing weights, and can be set according to actual needs. The kernel points are represented as:

[0141]

[0142] k p is the local structure center point obtained by weighted average.

[0143] Nucleus point k p and its neighborhood N(k p ) to calculate the explicit geometric relationship of the point q in the . Let |qk p | is the distance, n(q) and n(k p ) is point q and core point k pThe normal vector of the θ is defined as cos(θ)=n(q)·n(k p ), where "." is the dot product operation, which is defined as the sum of vector component-wise multiplication. Definition:

[0144] g(q,k p )=[|qk p |,cos(θ)]

[0145] Among them, g(q,k p ) contains explicit geometric features of the relationship between distance and normal vector; the expression [|qk p |,cos(θ)] means that a vector with two components is constructed, |qk p | represents point q and core point k p The Euclidean distance between them, cos(θ) represents the normal vector of point q and the kernel point k p The cosine of the angle between the normal vectors of .

[0146] is the core point k p Aggregate neighborhood relationship features. Define feature distance weight as:

[0147]

[0148] Among them, f′(k p ) is N(k p ) is the weighted mean of the internal point features f′(q), and σ2 represents the Gaussian kernel parameter. The kernel point feature aggregation is:

[0149]

[0150] This embodiment uses w g (q) can highlight k p Points with similar features are generated to generate local structural features f(k p ).

[0151] S6, calculates and splices the local structural features of the processed unified point cloud under a set of spherical neighborhood radii of different sizes to obtain multi-scale features.

[0152] In order to obtain multi-scale features, a set of radius {r1,r2,…,r m} (m is a positive integer) Repeat S5. s Get the feature of point p Splicing to form multi-scale features, expressed as:

[0153]

[0154] f s (p) Capture both local and larger-scale geometric structure properties simultaneously.

[0155] S7 converts multi-scale features into graph-structured data and uses a graph attention network to extract global features from the graph-structured data.

[0156] The graph attention network in this embodiment is a lightweight message passing module based on the graph structure. It performs feature diffusion on the graph structure, so that local features can be propagated to the global level, thereby improving the consistency and robustness of features and providing a more robust feature representation for subsequent segmentation and classification.

[0157] Consider the point cloud P as a graph in is a set of points, and ε is the connection relationship between point pairs, which is usually established by k-nearest neighbor or radius search. For each point i, f s (i) is the initial feature, and the context propagation mechanism is introduced, which is expressed as:

[0158]

[0159] Among them, f s (i) represents the initial multi-scale feature vector of the i-th point; N(i) refers to the neighborhood of the i-th point in this graph. i and j represent the node indices in the graph structure data respectively.

[0160] The context weight is expressed as:

[0161]

[0162] Among them, |||| 2 represents the square of the Euclidean distance; σ3 represents the bandwidth parameter of the Gaussian kernel; this embodiment introduces context propagation to diffuse features globally and improve global consistency.

[0163] The graph attention network in this embodiment introduces a global attention mechanism to s ′(p) maps to query q i , key k j 、value v j Space. For point p i The global feature representation is:

[0164]

[0165] The attention weight is expressed as:

[0166]

[0167] Here, q i ,k j ,v j For f s ′(p) is the vector mapped from it, q i ·kj is the vector dot product. Global attention can capture the long-range correlation between any pair of points. i Used to identify the specific i-th point in the point cloud.

[0168] For the global feature F g (p) Clustering, divide the points into L classes, L is a pre-set positive integer. The class point set is The class anchor is represented as:

[0169]

[0170] in, is the center point of the class in the feature space; F g (p) represents the global feature vector obtained after a point passes through the global attention mechanism. It integrates the relationship between the point and other points in the entire point cloud and reflects the global semantic information.

[0171] S8, cluster the global features to obtain class anchors.

[0172] By minimizing the intra-class variance, the anchor points are optimized so that More accurately characterize the class center feature, expressed as:

[0173]

[0174] This optimization can usually be done in an iterative way to make the anchor points fit the class distribution better.

[0175] S9, fuses preliminary multimodal features, texture features and global features to obtain the final feature representation.

[0176] The initial fusion feature f′(p), the global feature F g (p) is fused with the original texture t(p) again. Let α, β and γ be non-negative weights, and α+β+γ=1, define

[0177] F final (p)=αf′(p)+βF g (p)+γt(p)

[0178] Among them, F final (p) serves as the final feature representation of multimodal and multi-level fusion.

[0179] S10, calculate the similarity between the final feature representation and the class anchor, classify according to the similarity, and obtain the preliminary segmentation result.

[0180] According to F final (p) and anchor point The similarity is used for preliminary segmentation. Definition of similarity:

[0181]

[0182] Among them, ||x|| represents the Euclidean norm of vector x, that is, The preliminary categories of point p are:

[0183]

[0184] Among them, arg max means taking the one that maximizes the similarity value.

[0185] S11, performing boundary smoothing and texture correction on the preliminary segmentation result to obtain a corrected feature representation.

[0186] Context propagation is performed on the boundary points of the preliminary segmentation results to smooth the boundaries. The boundary weight is defined as:

[0187]

[0188] Update the feature representation as follows:

[0189]

[0190] Here, neighborhood feature diffusion is used to reduce discontinuities at classification boundaries. Here, q represents any neighboring point within the neighborhood N(p) of point p. q is a point in the adjacent region of p, used for feature smoothing and context propagation. By weighting the feature information of these neighboring points, the features of p are updated, making the features at the boundary more continuous and smooth. The context propagation module is a simplified graph neural network layer. Its input is the preliminary fused features of each point and its neighborhood, and its output is the smoothed features, which are used to eliminate boundary discontinuities.

[0191] For points that may be misclassified, we combine texture features to correct them. definition:

[0192]

[0193] By adjusting ρ, It can improve the consistency of features in misclassified areas. corr (p) represents the corrected feature representation.

[0194] S12, fuses the multi-scale features and the corrected feature representation to obtain the enhanced segmentation result.

[0195] The segmentation results are fused using multi-scale features. s Extracted features Voting is performed to obtain the enhanced classification results, which are expressed as:

[0196]

[0197] The multi-scale fusion of this embodiment enhances the stability of the classification results. By performing weighted voting on features obtained at different scales, the fusion of multi-scale information is achieved. This step calculates the similarity between the features at each scale and the anchor points of each category, and accumulates the similarities of each scale, selecting the category with the largest total similarity as the preliminary classification result of the point. Using multi-scale fusion to enhance the classification results provides a robust initial judgment for the entire segmentation process, helps reduce the errors that may be introduced by single-scale features, and provides a more reliable foundation for subsequent error correction and iterative optimization.

[0198] S13, detecting suspicious points in the enhanced segmentation result, performing error correction on the suspicious points, obtaining a corrected segmentation result, and completing the segmentation of deep-sea mineral resources.

[0199] S131, use sparsity to detect potential suspicious points. Sparsity is expressed as:

[0200]

[0201] If sparsity(p) exceeds the preset threshold, p may be a misclassified point.

[0202] S132, recalculate the similarity between the suspicious point and the anchor point and update the category, which is expressed as:

[0203]

[0204] This provides an opportunity to correct the misclassified points. After recalculating and classifying the suspicious points, those points that are still judged to be misclassified constitute the point set P error .

[0205] S133, for the point set P that is still misclassified error Use the neighborhood feature mean correction to define the weight w e (p,q) is similar to S11 (real weighting factor), and the updated features are:

[0206]

[0207] Reduce the impact of isolated outliers.

[0208] S134, dynamically adjust the multimodal feature weights in the misclassified area. Let γ + δ = 1, and define:

[0209] F opt (p) = γF upd (p)+δt(p)

[0210] By minimizing Optimizing γ and δ makes the feature closer to the correct class center. upd (p) represents the feature corrected by the mean of the neighborhood features, and t(p) is the texture feature of point p.

[0211] S135, determine the feature deviation between the adjusted multimodal feature and the anchor point; if the feature deviation is greater than the preset feature deviation threshold, use the adjusted multimodal feature as the updated point feature and execute S136; otherwise, correct the adjusted multimodal feature to obtain the updated point feature.

[0212] Define the characteristic deviation as:

[0213]

[0214] in, For F opt (p) is the nearest anchor point. If δ(p) exceeds the threshold, then F opt (p) is corrected and expressed as:

[0215]

[0216] λ>0 is a real number parameter that controls the correction strength to make the point feature closer to the cluster center.

[0217] S136, after updating the features, calculate the core point features again. Definition:

[0218]

[0219] w′ k (p) is a real weight, f′(k p ) is the characteristic mean of the kernel point after update (real vector, obtained by a similar method to S108),

[0220]

[0221] Further refine local structural features.

[0222] S137, smoothing the multi-scale results to reduce the conflict between different scales, is expressed as:

[0223]

[0224] If there is a conflict, the feature distribution can be weighted and balanced.

[0225] Repeat the correction steps S131-S137 until the result is stable. This iterative process ensures that the final segmentation result is free of obvious anomalies and converges stably. Stable results can be observed by observing the change in the number of misclassified or outlier points during successive iterations. When this change falls below a predetermined threshold, or when the key features change minimally, the result is considered stable. A maximum number of iterations can also be set as a termination condition. The specific criteria need to be determined based on the application scenario and experimental data.

[0226] This embodiment also includes the following steps:

[0227] S14, detecting the uniformity of the category distribution of the segmentation result; if the category distribution uniformity is less than a preset uniformity threshold, dynamically adjusting the category anchor point.

[0228] In this embodiment, class uniformity is based on the difference in the number of samples across categories, or the difference between the feature mean and the actual sample distribution. When the number of samples in a particular category is significantly lower or higher than that of other categories, or when there is a significant deviation between the feature mean and the actual sample distribution, the class distribution is considered to be imbalanced.

[0229] If the class distribution is uneven, the anchor point can be dynamically fine-tuned. The adjusted class anchor point is expressed as:

[0230]

[0231] Among them, mean represents the average difference of intra-class point features, and η is the adjustment coefficient, which helps to balance the distribution of inter-class features.

[0232] S15, recalculating feature deviation and sparsity, taking points whose feature deviation is higher than a preset feature deviation threshold or whose sparsity is higher than a preset sparsity threshold as severely misclassified points, and calculating the neighborhood average of the severely misclassified points.

[0233] For areas with serious misclassification, the point features are updated to the neighborhood average, which is expressed as:

[0234]

[0235] F sem (p) Improving semantic consistency in local regions by averaging neighborhood features.

[0236] This step performs supplementary corrections on severely misclassified areas, updating point features using the neighborhood mean to improve local semantic consistency. This helps further optimize results in noisy or misclassified scenes.

[0237] S16, fusing the neighborhood average with the global feature to obtain the second final feature representation.

[0238] Fsem (p) and the global feature F g (p) is fused again, assuming ω + μ = 1; it can be expressed as:

[0239] F final2 (p)=ωF sem (p)+μF g (p)

[0240] By optimizing ω and μ, the overall classification accuracy is improved and the features are more consistent with the class anchors.

[0241] S17, calculating the similarity between the second final feature and the dynamically adjusted anchor point, performing classification based on cosine similarity, and obtaining an optimized segmentation result.

[0242] Using the final feature F final2 (p) and anchor point The final classification is performed based on the similarity, which is expressed as:

[0243]

[0244] After obtaining the segmentation results, the intersection-and-union (IU) Verify the segmentation quality. Intersection and union represent the intersection and union range of the predicted and true regions, respectively. A higher IoU indicates a more accurate segmentation.

[0245] like Figure 2 As shown, the present invention also discloses a deep-sea mineral resource segmentation device based on dynamic anchor points and iterative optimization, comprising:

[0246] The data acquisition module 201 is used to acquire sonar point clouds, lidar point clouds, and high-definition images of deep-sea mineral resources; align the coordinates of the sonar point clouds and lidar point clouds and merge them into a unified point cloud; and obtain texture features from the high-definition images.

[0247] A geometric feature acquisition module 202 is used to construct a spherical neighborhood of the unified point cloud and acquire initial geometric features of the unified point cloud based on the spherical neighborhood;

[0248] A block processing module 203 is used to perform block processing on the unified point cloud and smooth the overlapping areas to obtain a processed unified point cloud;

[0249] The multimodal feature fusion module 204 is used to fuse the initial geometric features with the texture features to obtain preliminary multimodal features;

[0250] A local structural feature acquisition module 205 is used to construct core points of a spherical neighborhood of the processed unified point cloud and calculate local structural features based on the core points;

[0251] A multi-scale feature acquisition module 206 is used to calculate the local structural features of the processed unified point cloud under a set of spherical neighborhood radii of different sizes and to perform splicing to obtain multi-scale features;

[0252] A global feature acquisition module 207 is used to convert multi-scale features into graph structure data and extract global features from the graph structure data using a graph attention network;

[0253] A class anchor point acquisition module 208 is used to cluster global features to obtain class anchor points;

[0254] The final feature representation acquisition module 209 is used to fuse the preliminary multimodal features, texture features and global features to obtain the final feature representation;

[0255] A preliminary segmentation module 210 is used to calculate the similarity between the final feature representation and the class anchor point, classify according to the similarity, and obtain a preliminary segmentation result;

[0256] Correction module 211, used to perform boundary smoothing and texture correction on the preliminary segmentation result to obtain a corrected feature representation;

[0257] Enhancement module 212, used to fuse multi-scale features and corrected feature representation to obtain enhanced segmentation results;

[0258] The error correction module 213 is used to detect suspicious points in the enhanced segmentation results, perform error correction on the suspicious points, obtain a corrected segmentation result, and complete the segmentation of deep-sea mineral resources;

[0259] The class anchor point dynamic adjustment module 214 is used to detect the uniformity of the class distribution of the segmentation results; if the class distribution uniformity is less than a preset uniformity threshold, the class anchor point is dynamically adjusted; the class uniformity is based on the difference in the number of samples in each class, or based on the difference between the feature mean and the actual sample distribution;

[0260] A severely misclassified point determination module 215 is configured to recalculate feature deviation and sparsity, and to define points with feature deviations higher than a preset feature deviation threshold or sparsity higher than a preset sparsity threshold as severely misclassified points, and to calculate the neighborhood average of the severely misclassified points.

[0261] A second final feature representation acquisition module 216 is configured to fuse the neighborhood average with the global feature to obtain a second final feature representation;

[0262] The optimization result acquisition module 217 is used to calculate the similarity between the second final feature and the dynamically adjusted anchor point, perform classification based on cosine similarity, and obtain an optimized segmentation result.

[0263] The specific implementation of the deep-sea mineral resource segmentation device based on dynamic anchor points and iterative optimization is the same as the deep-sea mineral resource segmentation method based on dynamic anchor points and iterative optimization, and will not be repeated in this embodiment.

[0264] The above is only a specific implementation of the present invention, but the design concept of the present invention is not limited to this. Any non-substantial changes to the present invention using this concept shall be deemed as an infringement of the protection scope of the present invention.

Claims

1. A deep-sea mineral resource segmentation method based on dynamic anchor points and iterative optimization, characterized in that: The specific steps are as follows: S1, acquires sonar point clouds, lidar point clouds, and high-definition images of deep-sea mineral resources; aligns the coordinates of the sonar point clouds and lidar point clouds and merges them into a unified point cloud; and obtains texture features from the high-definition images. S2, construct a spherical neighborhood of the unified point cloud and obtain the initial geometric features of the unified point cloud based on the spherical neighborhood; S3, performing block processing on the unified point cloud and smoothing the overlapping areas to obtain a processed unified point cloud; S4, the initial geometric features are fused with the texture features to obtain preliminary multimodal features; S5, constructing the core points of the spherical neighborhood of the processed unified point cloud, and calculating the local structural features based on the core points; S6, calculating the local structural features of the processed unified point cloud under a set of spherical neighborhood radii of different sizes and splicing them to obtain multi-scale features; S7, converts multi-scale features into graph structure data and uses graph attention network to extract global features from graph structure data; S8, cluster the global features to obtain class anchors; S9, fuses preliminary multimodal features, texture features and global features to obtain the final feature representation; S10, calculate the similarity between the final feature representation and the class anchor point, classify according to the similarity, and obtain the preliminary segmentation result; S11, performing boundary smoothing and texture correction on the preliminary segmentation result to obtain a corrected feature representation; S12, fuses multi-scale features and corrected feature representation to obtain enhanced segmentation results; S13, detecting suspicious points in the enhanced segmentation result, performing error correction on the suspicious points, obtaining a corrected segmentation result, and completing the segmentation of deep-sea mineral resources.

2. The deep-sea mineral resource segmentation method based on dynamic anchor points and iterative optimization according to claim 1 is characterized in that: The initial geometric features and texture features are fused to obtain preliminary multimodal features, as follows: Normalize the initial geometric features and texture features of the unified point cloud; The standardized initial geometric features are fused with the texture features through weighted summation to obtain the preliminary multimodal features, which can be expressed as: f′(p)=εf norm (p)+τt norm (p) Among them, f′(p) represents the preliminary multimodal features; f norm (p) represents the initial geometric features after normalization; t norm (p) represents the normalized texture feature; ε and τ represent f norm (p) and t norm The weight of (p), ε+τ=1; p represents the unified point cloud point p.

3. The deep-sea mineral resource segmentation method based on dynamic anchor points and iterative optimization according to claim 1 is characterized in that: The core points of the spherical neighborhood of the processed unified point cloud are constructed, and local structural features are calculated based on the core points, as follows: The kernel points in the spherical neighborhood of each point in the unified point cloud after calculation are expressed as: Among them, k p represents the core point of point p; N(p) represents the spherical neighborhood of point p, q represents the neighborhood point of point p, q∈N(p); w k (q) represents the weight of generating kernel points; σ1 represents the Gaussian parameter of generating kernel points; || || 2 represents the square of the Euclidean distance; Calculate the explicit geometric relationship between the kernel point and the points in its neighborhood, expressed as: g(q,k p )=[|q-k p |,cos(θ)] Among them, g(q,k p ) represents q and k p The explicit geometric relationship of cos(θ)=n(q)·n(k p ), n(q) represents the normal vector of point q, n(k p ) represents the core point k p Normal vector of |qk p | represents q and k p distance; Aggregate the neighborhood relationship features of the core points to obtain local structural features; expressed as: Among them, f(k p ) represents the local structural features; w g (q) represents the feature distance weight.

4. The deep-sea mineral resource segmentation method based on dynamic anchor points and iterative optimization according to claim 1 is characterized in that: The graph attention network performs context propagation on the input graph structure data and introduces a global attention mechanism; the context propagation is expressed as: Among them, f s () represents the multi-scale feature vector; f s ′() represents the multi-scale feature vector after propagation; w ctx (i, j) represents the context weight; i and j represent the index of the i-th node and the j-th node in the graph structure data respectively; N(i) represents the neighborhood of the i-th node in the graph structure data; || || 2 represents the square of the Euclidean distance; σ3 represents the bandwidth parameter of the Gaussian kernel; The global attention mechanism maps the propagated multi-scale feature vector to the query, key, and value space, expressed as: Among them, F g (p i ) represents the global feature of the i-th specific point in the unified point cloud point p; α ij represents the attention weight; q i 、k j and v j Respectively, k represents the vectors that map the propagated multi-scale feature vector to query, key, and value; t Represents the t-th key vector among all keys.

5. The deep-sea mineral resource segmentation method based on dynamic anchor points and iterative optimization according to claim 1 is characterized in that: The global features are clustered to obtain class anchor points, as follows: Clustering of global features is expressed as: Among them, c l Represents the center point of the lth class in the feature space, that is, the class anchor point, l∈{1,2,…,L}, L represents the total number of categories; represents the set of points of the lth class, express The number of elements of F g (p) represents the global feature; The anchor points are optimized by minimizing the intra-class variance, which can be expressed as: in, Indicates taking the minimum value; || || 12 Represents the square of the Euclidean distance.

6. The deep-sea mineral resource segmentation method based on dynamic anchor points and iterative optimization according to claim 1 is characterized in that: The fusion of preliminary multimodal features, texture features and global features to obtain the final feature representation is as follows: F final (p)=αf′(p)+βF g (p)+γt(p) Among them, F final (p) represents the final feature representation; f′(p) represents the preliminary multimodal feature; F g (p) represents the global feature; t(p) represents the texture feature; α, β and γ represent f′(p), F g The weight of (p) and t(p), α+β+γ=1.

7. The deep-sea mineral resource segmentation method based on dynamic anchor points and iterative optimization according to claim 1 is characterized in that: The similarity between the final feature representation and the class anchor is calculated, and classification is performed based on the similarity to obtain a preliminary segmentation result, as follows: Calculate the cosine similarity between the final feature representation and the class anchor; Classification is performed based on cosine similarity, expressed as: Among them, class(p) represents the class to which the unified point cloud point p belongs, that is, the preliminary segmentation result; Indicates taking the l value that maximizes the similarity; F final (p) represents the final feature representation; c l represents the class anchor of category l; sim(F final (p),c k ) indicates F final (p) and c l The cosine similarity of .

8. The deep-sea mineral resource segmentation method based on dynamic anchor points and iterative optimization according to claim 1 is characterized in that: The suspicious points in the detection and classification results are corrected to obtain the corrected segmentation results, as follows: S131, calculating the sparsity of each point; points whose sparsity exceeds a preset sparsity threshold are considered suspicious points; S132, calculating the cosine similarity between the suspicious point and the anchor point, and reclassifying the suspicious point based on the cosine similarity; S133, collect the suspicious points that still exist after reclassification, use the neighborhood feature mean to correct the features of the remaining suspicious points, and obtain the mean correction feature; it is expressed as: Among them, F upd (p) represents the mean correction characteristic; w e (p,q) represents weight; F corr (p) represents the corrected feature representation, F corr (q) represents the corrected feature representation of point q; N(p) represents the spherical neighborhood of point p; q∈N(p); S134, dynamically adjusting the multimodal feature weights in the area where the suspicious point is located to obtain adjusted multimodal features; expressed as: F opt (p)=γF upd (p)+δt(p) Among them, F opt (p) represents the adjusted multimodal features; γ and δ represent F upd The weights of (p) and t(p), γ+δ=1; t(p) represents the texture feature; S135, calculate the feature deviation between the adjusted multimodal feature and the anchor point; if the feature deviation is greater than the preset feature deviation threshold, use the adjusted multimodal feature as the updated point feature and execute S136; otherwise, correct the adjusted multimodal feature to obtain the updated point feature, which is expressed as: F adj (p)=F opt (p)+λ[c l -F opt (p)] Among them, F adj (p) represents the updated point feature; λ represents a real number parameter used to control the correction strength; c l represents the class anchor of category l; S136, using the updated point features to calculate the core point features, and calculating the average feature value of the core points to obtain the refined local structure features; S137, performing multi-scale smoothing processing on the refined local structural features; S138, repeat S131-S137 until the termination condition is reached, the error correction is completed, and the corrected segmentation result is obtained; the termination condition may be that the maximum number of iterations is met, or that the change in the number of suspicious points is less than a preset change threshold.

9. The deep-sea mineral resource segmentation method based on dynamic anchor points and iterative optimization according to claim 8, characterized in that: After S13, the following steps are also included: S14, detecting the uniformity of the category distribution of the segmentation results; if the category distribution uniformity is less than a preset uniformity threshold, dynamically adjusting the class anchor point; the category uniformity is based on the difference in the number of samples in each category, or based on the difference between the feature mean and the actual sample distribution; the dynamically adjusted class anchor point is expressed as: Among them, c′ l represents the class anchor after fine-tuning; c l represents the class anchor point; η represents the adjustment coefficient; represents the set of points of the lth type; Represents the average difference in feature values ​​of points within a class; S15, recalculate the feature deviation and sparsity, take the points whose feature deviation is higher than the preset feature deviation threshold or the points whose sparsity is higher than the preset sparsity threshold as seriously misclassified points, and calculate the neighborhood average of the seriously misclassified points; the neighborhood average is expressed as: Among them, F sem (p) represents the neighborhood average; N(p) represents the spherical neighborhood, q∈N(p); |N(p)| represents the number of elements in N(p); S16, the neighborhood average is fused with the global feature to obtain the second final feature representation, which is expressed as: F final2 (p)=ωF sem (p)+μF g (p) Among them, F final2 (p) represents the second final feature representation; ω and μ represent F sem (p) and F g The weight of (p), ω + μ = 1; F g (p) represents the global feature; S17, calculating the similarity between the second final feature and the dynamically adjusted anchor point, performing classification based on cosine similarity, and obtaining an optimized segmentation result.

10. A deep-sea mineral resource segmentation device based on dynamic anchor points and iterative optimization, comprising the following: The data acquisition module is used to obtain sonar point clouds, lidar point clouds, and high-definition images of deep-sea mineral resources; align the coordinates of sonar point clouds and lidar point clouds and merge them into a unified point cloud; and obtain texture features from high-definition images; The geometric feature acquisition module is used to construct a spherical neighborhood of the unified point cloud and obtain the initial geometric features of the unified point cloud based on the spherical neighborhood; A block processing module is used to perform block processing on the unified point cloud and smooth the overlapping area to obtain a processed unified point cloud; Multimodal feature fusion module, used to fuse initial geometric features with texture features to obtain preliminary multimodal features; The local structure feature acquisition module is used to construct the core points of the spherical neighborhood of the processed unified point cloud and calculate the local structure features based on the core points; The multi-scale feature acquisition module is used to calculate the local structural features of the processed unified point cloud under a set of spherical neighborhood radii of different sizes and stitch them together to obtain multi-scale features; The global feature acquisition module is used to convert multi-scale features into graph structure data and use the graph attention network to extract global features from the graph structure data; The class anchor acquisition module is used to cluster global features and obtain class anchors; The final feature representation acquisition module is used to fuse preliminary multimodal features, texture features and global features to obtain the final feature representation; The preliminary segmentation module is used to calculate the similarity between the final feature representation and the class anchor, classify according to the similarity, and obtain the preliminary segmentation result; The correction module is used to perform boundary smoothing and texture correction on the preliminary segmentation results to obtain the corrected feature representation; The enhancement module is used to fuse multi-scale features and corrected feature representation to obtain enhanced segmentation results; The error correction module is used to detect suspicious points in the enhanced segmentation results, perform error correction on the suspicious points, obtain the corrected segmentation results, and complete the segmentation of deep-sea mineral resources.

Images