Defect identification and positioning method

Abstract

The invention relates to the technical field of pipeline inspection, in particular to a defect identifying and positioning method. Comprising the following steps: generating a uniform node feature tensor through coordinate mapping and feature fusion by synchronously collecting a pipeline inner wall image, an ultrasonic echo and an electromagnetic eddy current signal; constructing a space-time heterogeneous feature graph, integrating three types of relationships of a space adjacent edge, a time evolution edge and a semantic similarity edge, and dynamically optimizing a graph structure by utilizing a trainable fusion factor; a heterogeneous edge decoupling convolution and dynamic attention mechanism is designed, space-time semantic features are extracted through channels, neighborhood information is aggregated, and high-resolution defect classification is achieved; based on a classification result and a residual tensor of an original feature, a defect space position is accurately predicted through a coordinate inversion network, and positioning robustness is improved by combining positioning confidence score and weighted aggregation; and finally fusing the equipment track and the pipeline three-dimensional model to realize defect geographic coordinate mapping and interactive visualization. According to the method, the defect identification precision and the positioning reliability in a complex pipeline environment are remarkably improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of pipeline inspection, in particular to a defect identification and positioning method. BACKGROUND

[0002] As a key carrier of energy transportation and municipal infrastructure, the structural health state of the pipeline directly affects public safety and operation efficiency. With the continuous expansion of the pipeline network scale and the growth of the service life, the accurate detection of multiple defects such as surface cracks, internal holes and metal corrosion is increasingly urgent. Current technologies mainly rely on single-mode sensing methods such as visible light vision, ultrasonic waves or electromagnetic eddy current, which can achieve basic defect identification but are difficult to fully capture the spatio-temporal correlation characteristics of defects in complex pipeline environments (such as uneven lighting, metal surface reflection interference, elbow weld structure shielding).

[0003] A pipeline baseline detection system and method based on multi-technology fusion are disclosed in Chinese patent CN118731172B. The system collects position coordinate data of multiple detection points of the pipeline baseline through a global navigation satellite system, which is used as the basis data for subsequent return of abnormal detection points. Then, an ultrasonic detector is used to collect pipeline ultrasonic receiving signals at each detection point, and a data analysis algorithm based on artificial intelligence and signal processing is introduced in the backend to analyze the ultrasonic receiving signals, so as to detect the pipeline baseline state and identify whether there are pipeline cracks, leakage points, corrosion and other problems. When an abnormal condition of the pipeline baseline is detected, the position coordinate data of the abnormal point is returned to facilitate timely pipeline anomaly repair and ensure the safety and stability of the pipeline baseline.

[0004] With the development of multi-modal sensing fusion and spatio-temporal graph computing technology, it is urgent to build a dynamic detection model that takes into account spatial topology, temporal evolution and semantic correlation to improve defect identification accuracy and positioning reliability and meet high-standard engineering operation needs. Therefore, developing an intelligent detection method that integrates multi-source sensing data, dynamically models defect evolution rules and realizes centimeter-level positioning has become an important technical direction in the field of pipeline safety. SUMMARY

[0005] The present application aims to solve the problems in the background art and provides a defect identification and positioning method. The technical solution of the present application is as follows:

[0006] S1, synchronously collect a pipeline inner wall image sequence, ultrasonic echo signals and electromagnetic eddy current response signals, and generate a unified node feature vector through coordinate mapping and feature fusion;

[0007] S2, construct a spatial adjacency edge based on the Euclidean distance in three-dimensional space, establish a time evolution edge according to the position continuity between continuous time points, calculate a semantic similar edge based on the cosine similarity of the feature vector, and form a space-time heterogeneous feature graph by weighted fusion of the three edge weights;

[0008] S3, adopt three-channel graph convolution to extract and splice and fuse the spatial, temporal and semantic edge features, introduce a graph attention mechanism to dynamically calculate the adjacent node weights for feature aggregation, and finally output the defect classification result through a fully connected layer;

[0009] S4, construct a residual tensor based on the defect classification result and the original node feature, input it into the coordinate inversion network to predict the three-dimensional spatial position of the defect, and combine the position confidence scoring mechanism for weighted aggregation positioning;

[0010] S5, register the defect prediction position with the three-dimensional model of the pipeline, combine the detection device trajectory to map the defect in the geographic coordinate system, generate an interactive three-dimensional visual marker object, and finally present the defect recognition result in three-dimensional visualization.

[0011] Preferably, the unified node feature vector includes image features, ultrasonic features and eddy current features, and the image texture features are extracted by Retinex enhancement and ResNet-18, the ultrasonic time-frequency features are extracted by STFT, and the eddy current frequency energy spectrum is extracted by wavelet packet decomposition.

[0012] Preferably, the space-time heterogeneous feature graph construction process is as follows:

[0013] Based on the Euclidean distance in three-dimensional space, the Gaussian distance attenuation weight between nodes, i.e. the spatial adjacency edge weight, is calculated.

[0014] Between continuous time points t and t+1, the node correspondence relationship is established based on the spatial position continuity, and the semantic similar edge weight is constructed.

[0015] The semantic similar edge weight is calculated based on the cosine similarity of the node feature vector.

[0016] The edge weight matrix is formed by weighted fusion of the three types of edge weights, i.e. the spatial adjacency edge, the time evolution edge and the semantic similar edge, and the space-time heterogeneous feature graph is constructed including the node set, the edge set and the matrix.

[0017] Preferably, the recognition process of the defect classification result is as follows:

[0018] Three-channel graph convolution mechanism is adopted to independently weight and aggregate the features of the spatial, temporal and semantic edges, and the features are spliced and fused into unified node features after ReLU activation.

[0019] The graph attention mechanism is introduced to dynamically calculate the adjacent node weights, the similarity is calculated through a shared transformation matrix and a trainable vector, and the features are weighted and aggregated after LeakyReLU normalization.

[0020] The node features after the fusion of graph convolution and attention are mapped by a fully connected layer, and the defect type probability is output by Softmax normalization. The defect classification result is recognized.

[0021] Preferably, the weighted aggregation positioning process is as follows:

[0022] A residual tensor is constructed based on the difference between the product of the node classification probability vector and the preset defect feature center and the original feature vector;

[0023] A coordinate inversion network is constructed based on the residual tensor through two-layer linear transformation and ReLU activation, and the feature difference is mapped to physical space coordinates;

[0024] Based on the defect type probability, residual feature and predicted coordinate splicing vector, a location confidence score is output by linear transformation and Sigmoid activation;

[0025] The predicted coordinates of multiple associated nodes are weighted and aggregated based on the node location confidence score, and the final defect position center is output.

[0026] Preferably, the generation process of the interactive three-dimensional visual marker object is as follows:

[0027] A pipeline three-dimensional model is constructed, and is registered to the actual geographic coordinate system through a set of geographic anchor points, so that the model coordinate system is aligned with the real geographic space;

[0028] IMU, odometry and depth sensor data are fused, and the three-dimensional position and attitude of the detection device are estimated in real time based on an extended Kalman filter framework;

[0029] Based on the time correlation of the detection trajectory, the defect local coordinates are mapped to the pipeline three-dimensional geographic model coordinate system through rigid transformation and registration of the set of geographic anchor points, and the final aligned position result is output.

[0030] Based on the defect type label, geographic coordinates and confidence, an interactive three-dimensional visual marker object is generated.

[0031] Preferably, the coordinate inversion network includes two layers of linear transformation and one layer of ReLU activation function, which is used to map the residual feature to physical coordinates.

[0032] Preferably, the unified node feature fusion process is as follows:

[0033] For any node i, the graph convolution update at the lth layer is represented as follows:

[0034]

[0035] Then, the final output is fused:

[0036] where, represents the feature vector input of node i at the l-1th layer; and respectively represent the edge weight of node j to node i in the space, time, semantic edge; and respectively represent the space neighbor, time neighbor, semantic neighbor set of node i; and σ(·) represents the ReLU activation function; and respectively represent the feature vectors of node i in the space, time, semantic dimensions at the lth layer; and Concat(·) represents a vector splicing operation, which fuses the three types of convolution results into a unified feature representation; represents the feature vector of node i at the lth layer; and respectively represent the feature transformation matrix of the current graph convolution layer at the lth layer under the three edge types of space, time and semantic edge.

[0037] Compared with the prior art, the above technical scheme of the present application has the following beneficial technical effects:

[0038] The present application designs a defect recognition and positioning method, which significantly improves the accuracy and efficiency of pipeline defect detection by integrating multi-modal sensing data (visible light image, ultrasonic echo, electromagnetic eddy current) and spatio-temporal graph neural network:

[0039] (1) Multi-modal collaborative complementation: fusion of surface texture, internal structure and metal corrosion features, overcoming the limitations of single sensor (such as light interference, metal reflection), achieving high recall rate recognition of all defect types such as cracks, shedding, holes and corrosion;

[0040] (2) Spatio-temporal modeling advantage: construction of spatio-temporal heterogeneous feature graph (ST-HFG), integration of three types of edge relationships of spatial adjacency, temporal evolution and semantic similarity, dynamic capture of defect evolution law (such as corrosion diffusion), and improvement of robustness in complex scenes;

[0041] (3) High-precision positioning mechanism: end-to-end defect positioning is realized based on residual tensor and coordinate inversion network, combined with positioning confidence score and weighted aggregation, positioning error is controlled within centimeter level, which is significantly better than traditional interpolation method;

[0042] (4) Engineering landing closed loop: from multi-source data acquisition, graph neural network processing to three-dimensional geographic mapping and interactive visualization whole process automation, which can be directly integrated into pipeline inspection robot, reducing the cost of manual interpretation and improving the detection efficiency;

[0043] (5) Algorithm generalization ability: the decoupling convolution of heterogeneous edges and the dynamic attention mechanism adaptively optimize the dependence of the graph structure in different scenarios, and still maintain stable recognition and positioning performance in sparse areas (such as pipe bends). BRIEF DESCRIPTION OF DRAWINGS

[0044] Figure 1 A flow chart of a defect recognition and positioning method is provided. DETAILED DESCRIPTION

[0045] Embodiment one, as shown in the figure, the defect recognition and positioning method provided by the application comprises the following specific implementation steps: Figure 1

[0046] S1, through the integrated sensing module installed on the pipeline inspection robot, synchronously collect the image sequence, ultrasonic echo data and electromagnetic eddy current response signal inside the pipeline, and unify the image sequence, ultrasonic echo data and ultrasonic echo data through coordinate mapping, feature extraction and structured fusion into a multi-modal tensor set, specifically:

[0047] S11, use the integrated sensing module installed at the front end of the pipeline inspection robot to collect the following three types of complementary information in real time: visible light image sequence I t (x,y) of the inner wall of the pipeline, ultrasonic echo signal sequence U t (τ) and electromagnetic eddy current response signal E t (f), the three types of signals are indexed with time stamp t for synchronization, ensuring that the observation data at the corresponding time point is for the same physical space position;

[0048] Wherein, I t (x,y) represents the two-dimensional image collected at the t time, that is, the brightness value at the pixel position (x, y); U t (τ) represents the ultrasonic echo signal collected at the t time, τ is the signal time delay; E t (f) represents the eddy current response signal collected at the t time, f is the frequency variable;

[0049] It should be noted that the pipeline inspection robot used is a wall-climbing robot that climbs the wall inside the pipeline or tank;

[0050] S12, introduce the IMU (inertial navigation) and SLAM (simultaneous localization and mapping) module to construct the space mapping for each frame of collection point, specifically:

[0051] Define the camera frame coordinate as The current attitude quaternion of the pipeline inspection robot is Qt, and the position vector is T t ;

[0052] Then the position of each observation point in the world coordinate system is:​

[0053]

[0054] where, and denote the horizontal and vertical coordinates of the i-th pixel in the camera image frame; d i denotes the depth information obtained by the camera, i.e., the distance from pixel i to the object surface; Q t denotes the robot pose at time t, represented by a quaternion; T t denotes the position vector of the robot in the global space [X, Y, Z] T ; R(Q t ) denotes a function that converts the quaternion Q t into a 3x3 rotation matrix; K denotes the intrinsic matrix of the camera, containing information such as focal length and principal point coordinates;

[0055] S13, convert various raw signals into a unified structure of feature tensors, specifically:

[0056] Image feature extraction: enhance the local contrast of the image by Retinex illumination normalization algorithm, eliminate the interference of illumination, and then use lightweight convolution network ResNet-18 to extract spatial texture features F t I ;

[0057] Ultrasonic signal processing: use short-time Fourier transform (STFT) to map one-dimensional echo (ultrasonic echo signal sequence U t (τ)) to two-dimensional time-frequency domain F i U ;

[0058] Eddy current signal processing: extract frequency band energy spectrum by wavelet packet decomposition (WPT) to obtain frequency energy feature vector F t E ;

[0059] Unify the dimensions and normalize the feature tensors of the above three modalities to obtain normalized image feature tensors ultrasonic feature tensors and eddy current feature tensors

[0060] S14, combine the normalized feature tensors of the above three modalities into a unified node feature vector, denoted as:

[0061]

[0062] where, F t(i) denotes the fused unified structured node feature vector; λ1, λ2, λ3 denote modal weighting factors for balancing the importance of different modalities, initially equal weight, and subsequently trainable to obtain the optimal proportion;

[0063] S2, based on the multi-modal structured node features F t (i), a spatio-temporal heterogeneous feature graph (ST-HFG) is constructed, which integrates spatial adjacency, temporal sequence and modal semantic edges into a unified graph structure. A semantic cosine similarity edge construction method is proposed to strengthen the connection in areas with insufficient spatial adjacency. Trainable fusion factors (α, β, γ) are introduced to adapt to the differences in structural dependence of different scene defect types. Specifically:

[0064] S21, for the structured node set at the same time t, the spatial adjacency edge is constructed according to the Euclidean position distance in three-dimensional space; the spatial positions of the i-th and j-th nodes at time t are defined as P t (i) and P t (j), respectively, and the spatial adjacency weight is defined as a Gaussian distance decay function:

[0065]

[0066] where W t spatial (i,j) represents the weight value of the connection between nodes i and j in space, with a value range of (0, 1], and the closer the value is, the larger the value is; δ s is a sensitive parameter for spatial adjacency Gaussian decay, which controls the decay rate of edge weight with distance, obtained by parameter tuning;

[0067] S22, in order to capture the evolution pattern of pipeline defects with time advancing, the time evolution edge is established between the same spatial position nodes between consecutive time t and t+1, based on spatial re-projection to establish the corresponding relationship:

[0068] W temporal (i t ,i t+1 )=Ι(||P t (i t )-P t (i t+1 )||2<δ p );

[0069] where W temporal (i t ,i t+1 ) represents the consistency edge weight of time evolution, (i t ,it+1 ) respectively represent two node numbers (variable, need spatial relocation) at time (t, t+1) respectively; P t (i t ) and P t (i t+1 ) respectively represent the three-dimensional positions of two nodes at two times; Ι(·) represents an indicator function, 1 if the condition is true, otherwise 0;

[0070] S23, in order to further exert the synergistic advantages of multi-modal perception, a modal perception coupling weight mechanism is proposed, feature semantic similarity is introduced as a graph structure supplement, and the fusion feature of nodes i and j is F t (i) and F t (j), and the modal semantic similarity edge weight is defined as:

[0071] Where, W t semantic (i,j) represents the semantic similarity weight of nodes i and j, the value range is [-1, 1], and the larger the value, the more similar the semantics; ||·|| represents the L2 norm (Euclidean length) of the vector; F t (i)·F t (j) represents the vector dot product, which measures the correlation of the features of two nodes;

[0072] S24, by weighted fusion of the above three types of edge weights, a spatio-temporal heterogeneous feature graph G t is formed: G t =(V t ,E t ,W t );

[0073] Where, V t represents the node set in the graph, that is, all structured data points (detection points) at the current time t; E t represents the edge set in the graph, including space, time and semantic three kinds of relations; W t represents the weighted edge weight matrix, that is, the connection strength between all node pairs;

[0074] The edge weight matrix is integrated as:

[0075] W t (i,j)=α·W t spatial (i,j)+β·W temporal (i,j)+γ·W t semantic (i,j);

[0076] wherein a, b, g represent the weighting coefficients of the space edge, the time edge, and the semantic edge respectively, and satisfy a+b+g=1; W temporal (i,j) represents the consistency edge weight (0 or 1) of nodes i and j from time evolution;

[0077] S3, a heterogeneous edge channel decoupling convolution is constructed, three types of edge graph convolution paths of space, time and semantics are modeled respectively, high-resolution defect semantic extraction is performed, and a dynamic attention aggregation mechanism is used to solve the problem of uneven importance of adjacent nodes, and the robustness of defect recognition in a complex scene is improved, specifically as follows:

[0078] S31, a three-channel graph convolution mechanism is introduced, feature propagation and aggregation are independently performed for each type of edge, and then fused into a unified encoding vector, for any node i, the graph convolution update at the lth layer is represented as follows:

[0079]

[0080] Then the final output is fused:

[0081] wherein, represents the feature vector input of node i at the (l-1)th layer, and is initially generated in step S1 F t (i); and represent the edge weight of node j to node i in the space edge, the time edge and the semantic edge respectively; and represent the set of spatial neighbors, time neighbors and semantic neighbors of node i respectively, which come from the three types of edge structures of the graph in step S2; and σ(·) represents the ReLU activation function; and represent the feature vectors of node i in the space, time and semantic dimensions at the lth layer; and Concat(·) represents the vector concatenation operation, which fuses the three types of convolution results into a unified feature representation; represents the feature vector of node i at the lth layer; and represent the feature transformation matrix (trainable parameter) of the current graph convolution layer at the lth layer under the three types of edge of the space edge, the time edge and the semantic edge respectively;

[0082] S32, a graph attention mechanism (GAT) based on node pair similarity is introduced to dynamically adjust the importance weight of adjacent node information, for any pair of adjacent nodes i, j, the attention weight is calculated as follows:

[0083]

[0084] Then the feature aggregation is performed:

[0085] where W represents a node feature transformation matrix (shared), projecting node features from input dimension to attention space; a represents a weight vector of attention mechanism, participating in the calculation of neighbor importance (trainable parameter); represents the attention coefficient of node j to node i, that is, the information contribution size, with a value range of [0, 1]; N i represents the adjacent node of node i; LeakyReLU(·) represents a LeakyReLU activation function; represents the aggregated features;

[0086] S33, node representation after multi-layer graph convolution and attention fusion (L is the final number of layers), which is sent into a fully connected decoder to realize defect type classification:

[0087] where, represents the feature representation of node i after the last layer of graph attention / convolution; W c represents a fully connected layer weight matrix, used to map the graph embedding to the category space; b c represents a bias term, providing a basic score correction term for each category (trainable parameter); represents the prediction result of node i, which is a probability vector belonging to the probability distribution of each defect type; Softmax(·) represents a Softmax function, which normalizes the output so that the sum of scores of all categories is 1, and outputs a probability distribution;

[0088] S4, on the basis of the defect prediction result of each node output by the graph neural network , through the construction of joint residual tensor, spatial coordinate inversion network and position confidence optimization, high-precision dynamic positioning of defect spatial position is realized, specifically:

[0089] S41, to realize the reverse deduction of the possible physical space position from the node classification result, first construct the residual tensor of the classification output and the original multi-modal features:

[0090]

[0091] where, ΔR i represents the feature residual vector of node i, that is, the difference between the classification prediction and the original observed features; represents the defect type prediction probability vector (including but not limited to: normal / crack / falling / corrosion); T type represents a pre-defined defect type feature center tensor, with a feature prototype vector corresponding to each defect type (obtained from historical data statistics); F t(i) represents the unified structured feature vector of node i (output by step S1);

[0092] S42, based on the residual tensor AR i A multi-layer coordinate inversion network is constructed to map the residual feature space to the physical space coordinates:

[0093] wherein, represents the predicted space coordinates of node i; W1 and W2 represent the weight matrices of the neural network, respectively for linear transformation of the input layer and the hidden layer (as learnable parameters); b1 and b2 represent the bias terms, corresponding to the output offset of each layer (learnable parameters); ReLU(·) represents the ReLU activation function;

[0094] It should be noted that the multi-layer coordinate inversion network is a feedforward neural network structure for mapping the residual feature vector AR i to the physical space coordinates, the core components of which include two layers of linear mapping layers and one layer of nonlinear activation function layers. First, the input residual features are linearly transformed by the first layer weight matrix W1, and the bias term b1 is added. Then, the ReLU activation function is introduced to enhance the nonlinear fitting ability, and the result is taken as the intermediate hidden layer expression. Then, the second layer weight matrix W2 and the bias term b2 are used to complete the final output mapping, and the predicted value of the space coordinates of the defect node is obtained. The structure as a whole extracts deep spatial position patterns from feature residual information, and has the characteristics of trainability, end-to-end optimization capability and strong generalization capability for complex defect feature to space projection relationship, which is a key module to realize the defect space inversion positioning task;

[0095] S43, to enhance the positioning stability and accuracy, a positioning confidence score mechanism is constructed:

[0096]

[0097] wherein, S i represents the positioning confidence score of node i, reflecting the reliability of the predicted position of the node, with a value range of [0, 1]; W s represents a linear transformation matrix, used to map the input concatenated vector to the confidence score (as a trainable parameter); b s represents a bias term, which corrects the output distribution center (as a trainable parameter); Sigmoid(·) represents the Sigmoid activation function; || represents the vector concatenation symbol;

[0098] S44, when multiple nodes have correlations, a spatial weighted aggregation operation is performed to improve the positioning robustness:

[0099]

[0100] Among them, P final Represents the final aggregated position coordinates, reflecting the defect location center consistently pointed to by multiple associated nodes; N represents the number of nodes participating in the reasoning within the aggregated area;

[0101] S5, based on the defect spatial coordinate result P obtained by residual inversion positioning in step S4 final , combining the trajectory data of the inspection equipment with the known 3D model of the pipeline, through coordinate registration, trajectory reconstruction, defect mapping and visual interactive rendering, the spatial reproduction and 3D visualization of the defects are achieved. Specifically:

[0102] S51. Build a 3D topological model of the pipeline based on the design drawings or BIM model. pipe , and georeference anchor binding: M geo =Φ ref (M pipe ,G ref );

[0103] Among them, Φ ref represents the geographic anchoring function, which is used to align the model coordinate system with the actual geographic space; M geo Represents a three-dimensional spatial pipeline model calibrated by geographic anchor points, with a reference system consistent with the geographic coordinate system (such as WGS84); G ref Represents a set of geographic anchor points, such as starting manhole covers, pipeline inflection points, and other known spatial location points with GPS coordinates;

[0104] It should be noted that the geographic anchor function Φ ref It is a spatial mapping function used to align the three-dimensional pipeline model coordinate system with the actual geographic coordinate system. Its core function is to align the local coordinate points in the modeling space through the reference point set G ref Perform rigid or non-rigid transformation (including but not limited to GPS coordinates of start and end points, measured coordinates of pipeline nodes) to achieve unique positioning of each point in the 3D model in geographic space; this function can be implemented based on affine transformation, ICP (Iterative Closest Point) registration or georeferencing algorithm to ensure that the structural topology in the model space remains consistent in the registered geographic space, thereby providing a standardized geographic reference basis for the actual location of defect location results;

[0105] S52. Perform trajectory reconstruction based on the IMU data, odometer data, and relative displacement collected by the detection device: T det (t) = K fuse (D imu (t),D odo (t),D depth (t));

[0106] Among them, Tdet (t) represents the three-dimensional position and pose (pose) of the detection device at time t; K fuse represents a multi-source data fusion algorithm that combines sensor data of different modalities for trajectory inference; D imu (t) represents the inertial measurement unit (IMU) data at time t; D odo (t) represents the odometry measurement at time t; D depth (t) represents the depth or inclination sensor data at time t;

[0107] It should be noted that the multi-source data fusion algorithm K fuse is an algorithm that fuses multiple sensor data from the detection device, including inertial measurement unit (IMU) data D imu (t), odometry data D odo (t), and depth or inclination sensor data D depth (t), in real time, aiming to overcome the shortcomings of single sensor data being easily affected by noise and errors by fusing the advantages and complementary information of each sensor, and to improve the accuracy and stability of trajectory estimation of the detection device; in specific implementation, K fuse based on the extended Kalman filter (EKF) framework, the system state is defined as the three-dimensional position, velocity and attitude angle of the device, by establishing a state transition model and a sensor observation model, combining the time stamp synchronization and error covariance matrix of the sensor data, dynamically updating the state estimation and covariance, completing the smoothing prediction and correction of the device motion trajectory;

[0108] S53, according to the defect position P final,i (the final positioning coordinates of the i-th defect) output in step S4, it is mapped from the detection relative coordinate system to the geographical reference three-dimensional model:

[0109]

[0110] wherein, represents the three-dimensional position of defect i in the unified geographical coordinate system, i.e. the final alignment position result; Ψ align represents a coordinate registration function, which maps local coordinates to model global coordinates according to detection trajectory, device relative position, model coordinate conversion; P final,i represents the final positioning coordinates of the i-th defect;

[0111] It should be noted that the coordinate registration function Ψ align is a core transformation mechanism that maps the predicted position of the defect in the detection device relative coordinate system to the pipeline three-dimensional geographical model coordinate system, and its implementation process mainly includes three steps: first, according to the spatial topology structure established in steps S2 and S3, determine the node i in the detection trajectory T detthe time correspondence on (t); secondly, based on the spatial posture information (such as Euler angle, acceleration and relative displacement) of the device at the moment, the predicted position is rotated and translated in three-dimensional coordinates by using a rigid transformation matrix to obtain the transformed point in the local geographic reference frame; finally, the geographic reference anchor point set G ref established in step S5.1 is combined to further accurately register the local coordinates to the model coordinate system M geo , realizing continuous mapping from the detection relative coordinate space to the geographic model coordinate space, and ensuring the physical semantic consistency and spatial positioning accuracy of the defect points in the three-dimensional visual pipeline model;

[0112] S54, generating an interactive three-dimensional visual marker object according to the type label, spatial position and recognition confidence of each defect:

[0113] wherein A i represents the annotation information set of defect i, which is used for three-dimensional visual presentation and interactive operation; Label i represents the defect type label, including but not limited to crack, shedding, blockage and corrosion (from the recognition result of the graph neural network); S i represents the confidence score of defect recognition and positioning; View i represents the image segment, multi-modal detection snapshot and other original visual information associated with defect i, which is used for user click backtracking;

[0114] S55, integrating the constructed defect annotation and trajectory, and pipeline model into a three-dimensional visual engine (including but not limited to Unity, Cesium, Three.js), and providing multi-dimensional interactive functions:

[0115] Multi-view browsing (orthogonal / perspective / sectional);

[0116] Defect level filtering and hierarchical highlighting;

[0117] Linkage presentation with engineering GIS system;

[0118] Supporting export of defect coordinate report and risk distribution map.

[0119] The embodiments of the application are described in detail above in combination with the drawings, but the application is not limited thereto, and various changes can be made within the knowledge range of those skilled in the art without departing from the purpose of the application.

Claims

1. A defect identification and positioning method, characterized in that: The specific implementation steps include the following: S1, synchronously collect pipeline inner wall image sequence, ultrasonic echo signal and electromagnetic eddy current response signal, and generate a unified node feature vector through coordinate mapping and feature fusion; S2: Construct spatial adjacency edges based on Euclidean distance in three-dimensional space, establish temporal evolution edges based on position continuity between consecutive moments, calculate semantic similarity edges through cosine similarity of feature vectors, and weightedly fuse the three types of edge weights to form a spatiotemporal heterogeneous feature graph; S3: Use three-channel graph convolution to extract spatial, temporal, and semantic edge features respectively and merge them. Introduce a graph attention mechanism to dynamically calculate the weights of adjacent nodes for feature aggregation, and finally output the defect classification results through a fully connected layer. S4. Based on the defect classification results and the original node features, a residual tensor is constructed and input into the coordinate inversion network to predict the three-dimensional spatial position of the defect, and a weighted aggregation positioning is performed in combination with the positioning confidence scoring mechanism; S5. Align the predicted defect location with the pipeline 3D model, and map the defect in the geographic coordinate system based on the detection equipment trajectory to generate an interactive 3D visual marker object. Finally, the defect identification results are visualized in 3D.

2. A defect identification and positioning method according to claim 1, characterized in that: The unified node feature vector includes image features, ultrasonic features and eddy current features. The image texture features are extracted by Retinex enhancement and ResNet-18, the ultrasonic time-frequency features are extracted by STFT, and the eddy current frequency domain energy spectrum is extracted by wavelet packet decomposition.

3. A defect identification and positioning method according to claim 2, characterized in that: The process of constructing spatiotemporal heterogeneous feature maps is as follows: The Gaussian distance attenuation weight between nodes is calculated based on the Euclidean distance in three-dimensional space, that is, the spatial adjacent edge weight; Between consecutive times t and t+1, node correspondence is established based on spatial position continuity, and semantically similar edge weights are constructed; Calculate semantic similarity edge weights based on the cosine similarity of node feature vectors; The three types of edge weights, namely spatial adjacency edges, temporal evolution edges and semantic similarity edges, are weightedly fused to form an edge weight matrix, and a spatiotemporal heterogeneous feature graph containing a node set, an edge set and the matrix is ​​constructed.

4. A defect identification and positioning method according to claim 3, characterized in that: The identification process of defect classification results is: A three-channel graph convolution mechanism is used to independently weight and aggregate features of spatial, temporal, and semantic edges, which are then spliced ​​and fused into unified node features after ReLU activation. The graph attention mechanism is introduced to dynamically calculate the weights of adjacent nodes, and the similarity is calculated by sharing the transformation matrix and the trainable vector. The weighted aggregation features are normalized by LeakyReLU. The node features after graph convolution and attention fusion are mapped through the fully connected layer, and the Softmax normalization output is used to output the defect type probability and identify the defect classification result.

5. A defect identification and positioning method according to claim 4, characterized in that: The weighted aggregation positioning process is as follows: Construct a residual tensor based on the difference between the product of the node classification probability vector and the preset defect feature center and the original feature vector; Based on the residual tensor, a coordinate inversion network is constructed through two layers of linear transformation and ReLU activation to map the feature differences into physical space coordinates; Based on the defect type probability, residual features and predicted coordinate splicing vector, the positioning confidence score is output after linear transformation and Sigmoid activation; The predicted coordinates of multiple associated nodes are weighted and aggregated based on the node location confidence score to output the final defect location center.

6. A defect identification and positioning method according to claim 5, characterized in that: The generation process of interactive 3D visual marker objects is as follows: Build a 3D pipeline model and register it to the actual geographic coordinate system through a set of geographic anchor points, aligning the model coordinate system with the real geographic space; Fusion of IMU, odometer, and depth sensor data to estimate the 3D position and attitude of the detection device in real time based on the extended Kalman filter framework; Based on the temporal correlation of the detection trajectory, the local coordinates of the defect are mapped to the 3D geographic model coordinate system of the pipeline through rigid transformation and registration with the geographic anchor point set, and the final aligned position result is output; Generate interactive 3D visual marker objects based on defect type labels, geographic coordinates, and confidence levels.

7. A defect identification and positioning method according to claim 5, characterized in that: The coordinate inversion network consists of two layers of linear transformation and one layer of ReLU activation function, which are used to map the residual features into physical coordinates.

8. A defect identification and positioning method according to claim 4, characterized in that: The unified node feature fusion process is: For any node i, its graph convolution update at layer l is expressed as follows: This is then fused to form the final output: in, Represents the feature vector input of node i at the l-1th layer; and Respectively represent the edge weights of node j to node i in space, time, and semantics; and denote the spatial neighbors, temporal neighbors, and semantic neighbors of node i, respectively; σ(·) denotes the ReLU activation function; and Respectively represent the feature vectors of node i in the spatial, temporal, and semantic dimensions at the lth layer; Concat(·) represents the vector concatenation operation, which fuses the three types of convolution results into a unified feature representation; Represents the feature vector of node i at layer l; and They represent the feature transformation matrices of the current graph convolutional layer l under the three edge types of space, time, and semantic edges.

Images