A tank weld three-dimensional extraction method based on multi-modal fusion and topological prior

Abstract

The application discloses a kind of based on multi-modal fusion and topological prior tank weld three-dimensional extraction method, by successively constructing three-dimensional point cloud, cylindrical development, two-dimensional multi-modal image, topological constraint optimization, the technical idea of three-dimensional back projection, three-dimensional point cloud is developed as two-dimensional regular image along cylindrical surface, and high complexity three-dimensional calculation problem is reduced to efficient two-dimensional image processing problem;RGB color assignment chart, reflection intensity color assignment chart and normal change rate color assignment chart three multi-modal images are generated in development domain, and three types of complementary features such as color, physical reflection and microscopic geometry are fused;Using the strong topological prior that vertical storage tank weld longitudinal is plumb line, annular is horizontal circular ring, weld candidate is denoised and fracture adaptive completion in two-dimensional development domain;By establishing two-dimensional pixel and three-dimensional point cloud two-way index mapping, the optimized two-dimensional weld result is back projected back to three-dimensional space, and the weld point set with real physical coordinates is output.

Description

Technical Field

[0001] This invention belongs to the interdisciplinary field of computer graphics, 3D vision and industrial tank inspection, and specifically relates to a 3D extraction method for tank weld seams based on multimodal fusion and topological prior. Background Technology

[0002] Vertical cylindrical storage tanks are crucial infrastructure in the energy and chemical industries. During long-term operation, the tank structure is highly susceptible to corrosion, cracking, and localized deformation in the weld areas due to factors such as internal media corrosion, alternating external environmental loads, and uneven foundation settlement. Therefore, accurate location and extraction of tank welds are essential for subsequent digital inspections, quantitative assessment of wall panel deformation, non-destructive testing of welds, and full life-cycle health management.

[0003] Currently, the detection and extraction technologies for tank welds mainly include the following three categories:

[0004] 1. Manual Inspection Method: This method mainly relies on inspectors setting up scaffolding or using suspended platforms for close-range visual and manual non-destructive testing. This method is labor-intensive, carries high operational risks, and the test results are subject to significant subjective errors, making digital documentation difficult.

[0005] 2. Two-dimensional image-based detection methods: These methods utilize wall-climbing robots or drones equipped with high-definition cameras to acquire images of the tank walls, and then identify weld seams through image processing or deep learning algorithms. However, due to factors such as uneven lighting on the tank surface, wall panel reflections, coating peeling, and rust, the stability of image recognition is poor. More importantly, simple two-dimensional images cannot directly provide the absolute coordinates of the weld seams in three-dimensional space, making it difficult to accurately correlate them with the three-dimensional geometric model of the tank. Furthermore, due to the enormous physical volume and vast wall surface of large tanks, without the guidance of three-dimensional coordinates of the weld seams, wall-climbing robots often struggle to perform intelligent three-dimensional path planning based on the distribution characteristics of the weld seams, often relying on blind and time-consuming global traversal searches. This not only significantly reduces the efficiency of the inspection operation but also limits the autonomy and intelligence level of the equipment.

[0006] 3. Defect Detection Method Based on 3D Point Clouds: A 3D laser scanner is used to acquire the overall geometric topography point cloud of the tank, thereby assessing overall deformation or localized corrosion. However, due to the small geometric dimensions of the weld itself, the subtle geometric features of the weld are easily obscured by point cloud noise and wall undulations in large-scale tank point clouds. Existing point cloud processing algorithms mainly target macroscopic topography analysis and lack the ability to automatically extract fine weld structures.

[0007] Based on the above analysis, the existing technical solutions mainly have the following drawbacks:

[0008] 1. Lack of three-dimensional spatial positioning: Two-dimensional image recognition methods can only obtain the weld morphology in the image coordinate system, and cannot directly correspond to the actual three-dimensional spatial coordinates of the tank, which limits its application in digital twins and spatial morphology analysis.

[0009] 2. Weak anti-interference ability of complex surfaces: Relying on only a single color feature (RGB) or a single geometric feature (such as curvature), it is easy to generate a large number of false detections and missed detections when facing complex working conditions such as rusted panels, paint peeling, stain interference or local dents.

[0010] 3. Conflict between point cloud extreme noise and computational efficiency: Large-scale 3D point cloud data is massive. If local micro-features are searched directly in 3D space, the computational complexity is extremely high. At the same time, edge noise caused by the scanner can easily lead to a large number of isolated false weld responses.

[0011] 4. Lack of structural prior integration: The existing algorithm does not incorporate the strong topological constraints of weld distribution in the tank manufacturing specifications, and cannot adaptively complete and logically correct fractures and local deformations that occur during the detection process. Summary of the Invention

[0012] To address the shortcomings of existing technologies and achieve the goals of improving weld seam identification accuracy, efficiency, and robustness while reducing costs, this invention adopts the following technical solution:

[0013] A method for 3D extraction of tank weld seams based on multimodal fusion and topological prior is presented below:

[0014] The point cloud data of the tank's sidewalls is acquired and laterally segmented along the main axis of the sidewalls. The tank's sidewalls are then fitted with the point sets within each slice to obtain the main axis information of the tank.

[0015] Based on the main axis information, the point cloud data of the side wall plate around the main axis is unfolded to obtain the unfolded point cloud data of the side wall plate.

[0016] The unfolded sidewall panel is divided into blocks and meshed, and the attributes of each point are projected onto the mesh pixels to generate a multimodal image;

[0017] Based on the multimodal image, weld seam pixels are extracted, and the extracted weld seam pixels are back-projected onto the point cloud of the side wall plate to obtain the weld seam points of the tank.

[0018] Furthermore, based on the three-dimensional coordinates in the point cloud data, the extreme midpoint of the tank cross-section in the XY plane is calculated, and combined with the minimum value of the Z-axis in the height direction of the tank, a translation vector is constructed to perform translation transformation on all points, resulting in a normalized point cloud dataset.

[0019] Based on the histogram distribution characteristics of the tank along the Z-axis after the translation transformation, the top cover and bottom plate of the tank are removed to obtain the point cloud data of the side wall plates of the tank.

[0020] Furthermore, the tank body cross-section is divided into equally spaced slices along the height direction of the tank sidewall, and the point set within each slice is obtained;

[0021] Set slice height percentage and the percentage of slice steps In the point cloud height range of the tank side wall panel Inside, This indicates the lowest point of the point cloud on the side wall of the tank. This indicates the highest point of the point cloud on the side wall of the tank, from... Equal step generation of height and position There are 3 slices, each with a center height of 1. Extracting values ​​falling within the interval The points within the slice are taken as the point set of that slice. This indicates the height value that belongs to the height range;

[0022] For each slice point set, its two-dimensional planar coordinates are extracted, and the shape of the slice point set is fitted; for cylindrical tanks, the random sampling consensus 2D RANSAC algorithm is used to fit the slice circle through planar coordinates.

[0023] Specifically, within each slice point set, three or more points are randomly selected, and the temporary center and radius are calculated using the formula for calculating the circumcircle. The distance from all points in the slice to the circle is calculated. If the distance is less than a threshold, it is recorded as an interior point. After iterative calculation, the circle model with the most interior points is obtained, and all interior points under the circle model are extracted. The least squares method is used to solve for the center and radius of the purified circle in the slice.

[0024] Based on the shape fitted by the slice point set, the main axis information of the tank is estimated;

[0025] Specifically, the center and radius of the main axis, which serve as the main axis information of the tank, are obtained based on the median of the center and radius of each purified slice or the mean of the center and radius after removing abnormal slices.

[0026] Further, the radial distance of a point on the sidewall panel relative to the center of the circle is calculated;

[0027] Based on the difference in azimuth angle between each point and the center of the circle, the seam azimuth angle is set between the azimuth differences;

[0028] Map the azimuth of each point to a standard period interval starting from the azimuth of the joint to obtain the mapped azimuth.

[0029] Construct the three-dimensional point cloud coordinates of the unfolded tank sidewall panel, wherein the product of the mapped azimuth angle and the radius is the X-axis coordinate, the difference between the radial distance and the radius is the Y-axis coordinate, and the difference between the original Z-axis coordinate of the current point and the Z-axis coordinate of the lowest point of the tank sidewall panel point cloud is used as the new Z-axis coordinate of the current point.

[0030] Further, the azimuth angle of each point relative to the center of the circle is calculated;

[0031] Sort all azimuth angles by size to calculate the difference between azimuth angles of adjacent points, and calculate the cross-boundary angle difference at the junction of the first and last points to form an angle difference set;

[0032] The midpoint angle of the maximum angular difference (i.e., the maximum gap in the point cloud distribution) is taken as the seam azimuth angle. .

[0033] This mechanism ensures that during unfolding, the cutting line is always located at the gaps in the discontinuity of the point cloud, preventing the continuous weld from being split.

[0034] Furthermore, the unfolded tank sidewalls are physically divided into blocks; in large-scale storage tanks, the aspect ratio of the unfolded point cloud is usually extremely large, and direct whole-image processing will lead to resolution limitations. Therefore, this invention calculates the height range of the unfolded point cloud along the Z-axis, uses the size covered by the height range as the side length of the square physical block, and divides the point cloud into multiple non-overlapping sub-blocks along the X-axis.

[0035] Set a target resolution for the sub-blocks and adaptively adjust the pixel width W and height H based on the actual physical extreme aspect ratio of the sub-blocks to maintain the physical proportions of the image without distortion.

[0036] Map the coordinates of each point within the sub-block to pixel coordinates;

[0037] Neighborhood grid dilation is performed on pixels, and the attributes of all grid pixels within the dilation region are projected, thereby overcoming image holes and discrete noise caused by sparse point clouds.

[0038] Multimodal images are generated based on pixels that contain point attributes.

[0039] Furthermore, the point cloud data includes three-dimensional coordinates, color information, and reflection intensity; the multimodal image includes a color map, a reflection intensity color map, and a normal change rate color map.

[0040] Assigning color to the corresponding point of the pixel within the grid to obtain a color map;

[0041] Calculate the relative values ​​of the reflection intensity of the point cloud within the sub-block, and assign grayscale values ​​to the pixels within the grid based on the relative values ​​to obtain the reflection intensity color map;

[0042] The surface normal change rate of the point is calculated and nonlinear logarithmic transformation is performed to suppress extreme noise and amplify the geometric abrupt change signal at the weld edge. The obtained logarithmic curvature is normalized and grayscale stretched to obtain the single-point geometric abrupt change feature value. The mean of the feature values ​​in the neighborhood is used as the grayscale value of the normal change rate map pixel to obtain the normal change rate color map.

[0043] Furthermore, the weld seam pixel extraction process is as follows:

[0044] Edge extraction is performed on each modal image to generate edge binary maps. Linear structuring elements are used to perform closing and opening operations on the edge binary maps to obtain reconstructed candidate weld binary maps for each modality.

[0045] Calculate the grayscale standard deviation of the current segmented multimodal image to measure the saliency of modal information, and construct the weight coefficients of each modal image based on the grayscale standard deviation;

[0046] The reconstructed binary images of each modal candidate weld are weighted and superimposed using the weighting coefficients to obtain the pixel-level probability of weld existence.

[0047] Based on the probability of weld seam existence and the set judgment threshold, when a pixel is judged as a weld seam in multiple high-contrast modes, or has a strong response in a certain high-contrast mode, the pixel is activated, and a fused weld seam binary image is obtained.

[0048] Furthermore, on the fused binary weld image, constraint optimization is performed using the unique physical laws of the longitudinal and transverse weld grid of the tank body, as follows:

[0049] Run a cumulative probability Hough transform on the fused weld binary map to set angular constraints to filter longitudinal and circumferential weld lines.

[0050] Candidate longitudinal weld lines: line segment inclination angle (Among them, the angular tolerance deviation) Set as );

[0051] Candidates for circumferential weld lines: Segment inclination angle ;

[0052] One-dimensional mean-shift clustering is performed on the selected line segments to fit the main weld skeleton baseline within the block, including the set of vertical baselines and the set of horizontal baselines.

[0053] For any in the fused weld binary image pixel Calculate its Euclidean distance to the nearest set of vertical or horizontal reference lines;

[0054]

[0055] If the Euclidean distance is greater than the set maximum pixel deviation threshold, the point is determined to be a non-weld noise point, and the corresponding pixel value on the fused weld binary image is set to 0.

[0056] Scan along the fitted vertical reference line set and the circumferential reference line set. If a pixel interval with a continuous value of 0 is found on the grid line, and the break length of the interval is less than the set critical pixel length, then a linear interpolation algorithm is used to adaptively repair and fill the pixel value of the broken interval with 1.

[0057] Furthermore, during block projection, the original index of the sidewall point cloud enclosed by each pixel is recorded;

[0058] Extract all point indices corresponding to weld pixels and remove duplicates to obtain a weld index set;

[0059] In the point cloud of the side wall plate after the top and bottom plates have been removed, the corresponding points are extracted according to the weld index set to serve as the final high-precision three-dimensional weld positioning point set.

[0060] Because it originates from the original normalized point cloud system, each point carries real, high-precision three-dimensional physical coordinates. The final output point set This refers to a high-precision three-dimensional weld seam point cloud of the tank body.

[0061] The advantages and beneficial effects of this invention are as follows:

[0062] 1. This invention achieves high-precision three-dimensional spatial coordinate positioning of welds: By establishing a two-way mapping mechanism of "two-dimensional image - unfolded point cloud - original point cloud", the topology-optimized two-dimensional weld centerline is back-projected back into three-dimensional physical space, obtaining a weld point set with real physical dimensions and three-dimensional coordinates, providing basic data for the full life cycle health assessment of storage tanks.

[0063] 2. The multimodal feature fusion of this invention improves the robustness of weld inspection: it organically integrates color information (RGB), physical reflection characteristics, and micro-geometric undulation features. The complementarity of multi-dimensional information effectively overcomes the failure problem of single features when faced with interference such as corrosion, reflection, and dirt.

[0064] 3. This invention achieves adaptive optimization and noise reduction by introducing topological priors: it fully utilizes the physical topological regularity in the welded structure of large vertical storage tanks, where "longitudinal welds are vertical lines and circumferential welds are horizontal rings." By applying polar coordinate constraints to the two-dimensional expanded domain, isolated non-weld noise (such as wall panel attachments and marking lines) can be efficiently eliminated, and weld fractures caused by occlusion or weak features can be adaptively completed.

[0065] 4. This invention significantly improves computational efficiency: it transforms the highly complex three-dimensional point cloud computing problem into regular two-dimensional multimodal image space processing, which not only preserves the geometric and intensity information of the point cloud, but also greatly reduces the space and time complexity of the algorithm, making it suitable for efficient processing of large-scale storage tank point clouds.

[0066] 5. This invention supports 3D weld inspection and intelligent operation and maintenance: The 3D point set of the weld output by this invention can be directly used for 3D spatial positioning, morphological deviation analysis, and weld quality assessment, realizing an end-to-end automated process from the original point cloud to the weld inspection results. Simultaneously, the extracted 3D spatial position information of the weld can be further used as a path planning basis for wall-climbing robots or autonomous inspection devices—using the 3D coordinates of the weld as the navigation target point, guiding the robot to move autonomously along the weld trajectory, and carrying eddy current probes, ultrasonic sensors, or phased array detection equipment for close-range automated weld flaw detection, thereby realizing an intelligent closed-loop inspection operation of "first positioning, then navigation, then flaw detection," significantly improving the automation level and operational efficiency of non-destructive testing of tank welds. Attached Figure Description

[0067] Figure 1 This is a flowchart of the method in an embodiment of the present invention.

[0068] Figure 2 This is a flowchart of multimodal image line segment detection and topology prior optimization in an embodiment of the present invention. Detailed Implementation

[0069] The specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are for illustration and explanation only and are not intended to limit the present invention.

[0070] To address the technical problems of existing tank weld inspection methods, such as the inability to directly obtain the three-dimensional physical coordinates of the weld; poor robustness of single-feature detection under complex surface conditions like rust, reflection, and dirt; low computational efficiency of directly searching for microscopic features in three-dimensional space in large-scale scenarios; and high false detection rate and inability to adaptively repair weld fractures due to the lack of topological prior constraints on the weld's physical structure, this invention proposes a three-dimensional extraction method for tank welds based on multimodal fusion and topological priors. Figure 1 As shown, it includes the following steps:

[0071] Step S1: Obtain point cloud data of the tank through laser scanning.

[0072] The target storage tank was scanned from all angles using a 3D laser scanner (such as a terrestrial 3D laser scanner TLS) to obtain the original 3D point cloud dataset of the storage tank. :

[0073]

[0074] In the formula, Indicates the first The three-dimensional coordinates of a point in the laser scanner coordinate system; This indicates the true color information of that point; This indicates the intensity of laser reflection at that point; This represents the total number of points in the point cloud.

[0075] Step S2: Based on the point cloud data, establish the tank's principal axis coordinate system and perform point cloud preprocessing to remove the point clouds of the tank's top cover and bottom plate, thus obtaining the point cloud of the tank's side wall plate.

[0076] To eliminate the influence of the instrument's orientation on the coordinate system during scanning, the original point cloud needs to be repositioned and non-tank wall noise removed. This involves the following steps:

[0077] Step S2.1: Calculate the extreme midpoint and minimum Z-axis value of the tank in the XY plane based on the point cloud data, and construct a translation vector using the extreme midpoint and minimum Z-axis value. Perform coordinate system translation transformation on the point cloud data based on the translation vector so that the center of the tank bottom surface moves to the origin of the global coordinate system, thereby eliminating the offset of the initial scan coordinates.

[0078] Specifically, coordinate system normalization: calculate the maximum and minimum values ​​of the X and Y axis coordinates to obtain the center coordinates on the XY plane.

[0079]

[0080] Get the minimum coordinate value in the Z-axis direction. And construct a translation vector by combining the center coordinates. , This represents the Z-axis coordinate of the i-th point. Based on the translation vector, a translation transformation is performed on all points to obtain the normalized point cloud dataset. , This represents the i-th normalized point. At this point, the center of the tank bottom plate approximately coincides with... The central axis is approximately along the Z-axis.

[0081] Step S2.2: Based on the height direction of the tank after translation transformation Histogram distribution characteristics, removing the top cover and bottom plate of the tank.

[0082] Specifically, by detecting the boundary of abrupt changes in point cloud density in the histogram, the height threshold of the tank top cover and bottom plate is determined, and the point clouds of the tank top cover and bottom plate at the height abrupt change are identified and stripped to preserve the pure point cloud of the tank cylindrical side wall plate.

[0083] Step S3: Divide the tank sidewall plate laterally along the main axis direction, and fit the tank sidewall plate with the point set in each slice to obtain the main axis information of the tank.

[0084] Specifically, for a cylindrical tank, the point cloud on the side of the tank is sliced ​​horizontally at equal intervals along the height direction. The cylinder parameters are then accurately and robustly estimated by combining the Random Sample Consensus (2D RANSAC) algorithm with multi-level least-squares processing. The specific steps include the following:

[0085] Step S3.1: Divide the tank cross-section into equally spaced slices perpendicular to the height of the tank sidewall and obtain the point set within each slice.

[0086] Set slice height percentage and the percentage of slice steps In the point cloud height range of the tank side wall panel Inside, This indicates the lowest point of the point cloud on the side wall of the tank. This indicates the highest point of the point cloud on the side wall of the tank, from... Equal step generation of height and position There are 3 slices, each with a center height of 1. Extracting values ​​falling within the interval The points within the slice are taken as the point set of that slice. This indicates the height value that falls within the height range.

[0087] Step S3.2: For each slice point set, extract its two-dimensional planar coordinates. Shape fitting of the slice point set is performed.

[0088] Specifically, for cylindrical tanks, the Random Sampling Consensus 2D RANSAC algorithm is used to fit slice circles using planar coordinates. This involves randomly selecting three points within each slice point set and calculating the temporary center using the circumcircle calculation formula. and radius Calculate the distance from all points in the slice to the circle. If the distance is less than a threshold (e.g., ...), then... If any of these points are interior points, then they are denoted as interior points. After a specified number of iterations, the circle model with the most interior points is obtained. All interior points under this RANSAC circle model are extracted, and the least squares method is used for secondary purification to solve for a more accurate slice circle center. With radius .

[0089] Step S3.3: Estimate the main axis information of the tank based on the shape fitted by the slice point set.

[0090] Specifically, for a cylindrical tank, calculate the median of the center coordinates and radii of all successfully fitted slice circles. As the center and radius of the main axis of the tank.

[0091] In another embodiment, each slice is strictly filtered to remove abnormal slices with severe local deformation or adhesion interference. The arithmetic mean of the center and radius of the remaining valid slices is calculated and used as the final estimated center of the tank cylinder. and radius .

[0092] Step S4: Based on the tank's main axis information, unfold the tank surface of the side wall panels surrounding the main axis to obtain the unfolded point cloud data of the side wall panels. This specifically includes the following steps:

[0093] Step S4.1: Calculate the radial distance of a point on the tank sidewall relative to the center of the circle.

[0094] Specifically, for each point in the point cloud of the tank sidewall panel Calculate its offset relative to the center of the circle. , And calculate the actual radial distance of the point relative to the center of the circle based on the offset. .

[0095] Step S4.2: Based on the difference in azimuth angle between each point and the center of the circle, set the seam azimuth angle between the azimuth differences.

[0096] First, calculate the azimuth angle of each point relative to the center of the circle. Secondly, for all azimuth angles Sort the points in ascending order. Calculate the difference in azimuth angles between adjacent points, and calculate the cross-boundary azimuth difference at the junction of the first and last points, forming a set of azimuth differences. Finally, find the index corresponding to the largest azimuth difference. The midpoint angle of the largest azimuth difference (i.e., the largest gap in the point cloud distribution) is defined as the seam azimuth angle. This mechanism ensures that during unfolding, the cutting line is always located at the gaps in the discontinuity of the point cloud, preventing the continuous weld from being split.

[0097] Step S4.3: Map the azimuth of each point to a standard periodic interval starting from the azimuth of the joint, and construct the three-dimensional point cloud coordinates after the tank sidewall is unfolded.

[0098] The X-axis coordinate is the product of the mapped azimuth and the radius, the Y-axis coordinate is the difference between the radial distance and the radius, and the new Z-axis coordinate is the difference between the original Z-axis coordinate of the point and the Z-axis coordinate of the lowest point of the tank sidewall plate point cloud.

[0099] Specifically, the azimuth of each point is converted to the seam azimuth. The interval starting from :

[0100]

[0101] Construct the unfolded 3D point cloud coordinates : Axis (arc length unfolded): ; Shaft (radial deviation): , representing the radial offset of the point relative to the fitted cylindrical surface. The weld reinforcement is manifested as a local radial outward convexity. Through this radial deviation, the local geometric features of the weld are preserved. Axis (height): .

[0102] Step S5: Divide the unfolded tank sidewall into blocks and mesh them, then project point attributes onto the mesh pixels to generate a multimodal image.

[0103] In large-scale storage tanks, the aspect ratio of the unfolded point cloud is usually extremely large. Direct whole-image processing would lead to resolution limitations. Therefore, this step adopts a block square projection and neighborhood grid dilation mechanism for imaging, specifically including the following steps:

[0104] Step S5.1: Physically divide the unfolded tank sidewall panels into sections.

[0105] Specifically, calculate the height range of the unfolded point cloud. .by The side length of the square physical block ,along Along the axis, the point cloud is divided into multiple non-overlapping sub-blocks, each sub-block having... The coordinate range is: .

[0106] Step S5.2: Set the target resolution for the sub-block and adaptively adjust the pixel size based on the actual physical extreme aspect ratio of the sub-block.

[0107] Specifically, for each extracted sub-block, a target resolution is set. Based on the actual physical extreme aspect ratio of the sub-blocks, the pixel width W and height H of the image are adaptively adjusted to maintain the physical proportions of the image without distortion.

[0108] Step S5.3: Map the coordinates of each point within the sub-block to pixel coordinates.

[0109] Specifically, calculate the local extrema of the point cloud within the sub-block. and scaling Set the coordinates of each point Mapped to corresponding pixel coordinates .

[0110] Step S5.4: Perform neighborhood grid dilation on the pixel, and project the attributes of all grid pixels within the dilated area.

[0111] To overcome image holes and discrete noise caused by sparse point clouds, when projecting point attributes onto the image, a radius of [missing value] is applied to the pixel. The circular grid expands, for each point, to... Centered on, within radius Within the circle, assign its attribute value to all grid pixels within that circle. .

[0112] Step S5.5: Multimodal graph generation.

[0113] Assign colors to the corresponding points of the pixels within the grid to obtain a color map; calculate the relative values ​​of the reflection intensity of the point cloud within the sub-block, and assign grayscale values ​​to the pixels within the grid based on the relative values ​​to obtain a reflection intensity color map; calculate the surface normal change rate of the points and perform a nonlinear logarithmic transformation to suppress extreme noise and amplify the geometric abrupt change signal at the weld edge, and perform normalized grayscale stretching on the obtained logarithmic curvature to obtain the single-point geometric abrupt change feature value. Use the mean of the feature values ​​in the neighborhood as the grayscale value of the pixel in the normal change rate map to obtain the normal change rate color map.

[0114] Specifically, RGB color map Assign RGB color channel values ​​to the pixels within the projected circular grid, using the arithmetic mean of the color channel values ​​of all pixels falling within that pixel's neighborhood.

[0115]

[0116] In the formula, Represents a set The number of interior points; Indicates the first Points in the channel The original color byte value, with a value range of 100%. ).

[0117] Reflection Intensity Color Map To eliminate the absolute intensity differences between different blocks caused by scanning distance, the point cloud reflection intensity within the current sub-block is first linearly stretched and normalized to obtain the relative intensity value of each point. :

[0118]

[0119] In the formula, For the first The original reflection intensity at each point and These are the minimum and maximum values ​​of the reflection intensity of all points within the current sub-block, respectively; based on this, the reflection intensity color map pixels are... The grayscale value is calculated as follows:

[0120]

[0121] Color plot of normal change rate : Calculate the rate of change of the surface normal at the point A nonlinear logarithmic transformation is then performed to suppress extreme noise and amplify the geometric abrupt change signal at the weld edge.

[0122]

[0123] Next, calculate the extreme values ​​of the logarithmic curvature of all points within the current sub-block. and Then, normalized grayscale stretching is performed to obtain the single-point geometric abrupt change eigenvalue. :

[0124]

[0125] Finally, the normal rate of change map pixels The gray value is calculated using the mean of the feature values ​​within the neighborhood:

[0126] .

[0127] Step S6: Based on the multimodal image, perform image line segment detection and topological prior extraction of weld pixels.

[0128] In the generated and Pixel-level edge detection and topological filtering are performed on the three sub-block images to extract the weld seam pixel image, such as... Figure 2 As shown, the specific steps include the following:

[0129] Step S6.1: Extraction of independent weld features from each modal image.

[0130] Edge extraction was performed on the three images using the double-threshold Canny operator to generate binary edge maps. Then, linear structural elements are used. (In both the horizontal and vertical directions) the above binary images are processed by closing and opening operations to connect small dashed line segments and filter out isolated noise points, thus obtaining the reconstructed binary images of candidate weld seams for each mode:

[0131] .

[0132] Step S6.2: Adaptive weighted decision-level weld feature fusion.

[0133] To adaptively adjust the confidence level of each channel under different surface conditions, this invention introduces an adaptive weighting mechanism based on the local standard deviation of the image:

[0134] First, calculate the current sub-block image. gray standard deviation As a measure of the saliency of its modal information (the larger the standard deviation, the clearer the boundary between the weld and the background, and the higher the contrast):

[0135]

[0136] In the formula, This represents the average gray level of the corresponding image.

[0137] Secondly, define the weighting coefficients of each modality in the fusion process. :

[0138]

[0139] By weighted superposition of the reconstructed binary images of candidate welds for each mode, a pixel-level weld existence probability map is constructed. :

[0140]

[0141] because ,therefore .

[0142] Finally, set the fusion determination threshold. A pixel is activated when it is identified as a weld seam in multiple high-contrast modes, or when it has a strong response in a very high-contrast mode.

[0143]

[0144] Step S6.3: Denoise and repair the fused weld features based on topological distribution constraints.

[0145] In the binary image of the fused weld The constraints are optimized by utilizing the unique physical laws of the longitudinal and transverse welded grid of the tank.

[0146] First of all, Run the Probability Cumulative Hough Transform (PPHT) and set angle constraints:

[0147] Candidate longitudinal weld lines: line segment inclination angle (Among them, the angular tolerance deviation) Set as );

[0148] Candidates for circumferential weld lines: Segment inclination angle .

[0149] Secondly, one-dimensional mean-shift clustering is performed on the selected line segments to fit the main weld skeleton baseline system within the sub-block: a set of vertical reference lines. Circumferential baseline set .for any candidate pixel in Calculate its distance to the nearest baseline. or European distance :

[0150]

[0151] like (in The maximum pixel deviation threshold is set, corresponding to the actual physical distance. If the point is identified as a non-weld noise point, it will be forced to be within the weld seam. The value is set to 0.

[0152] Finally, along the fitted baseline and Perform a scan; if a continuous pixel range with a value of 0 is found on the grid line, and the break length of this range is less than the set critical pixel length... Then, a linear interpolation algorithm is used to adaptively fill the pixel values ​​of the broken interval with 1.

[0153] Step S6.4: Output the final weld pixel image to obtain the optimized binary weld pixel image. The coordinates of pixels with a value of 1 are marked as .

[0154] Step S7: Back-project the extracted weld seam pixels onto the point cloud of the tank side wall to segment the weld seam points of the tank.

[0155] By establishing an efficient bidirectional index, the detected weld seam pixels are... Back-projecting back into the sidewall point cloud after removing interference from the top and bottom plates, the three-dimensional weld points are quickly segmented, specifically including the following steps:

[0156] Step S7.1: Pixel-to-point index mapping.

[0157] Since the point coordinate to pixel coordinate mapping in step S5 is deterministic (the sub-block index k and scaling factor are known) and the radius of circular expansion During block projection, each pixel is recorded synchronously. The original set of indices of the enclosed lateral point cloud:

[0158] .

[0159] Step S7.2: Point cloud segmentation and back projection.

[0160] Extract all pixels belonging to the weld seam The corresponding point indices are used to remove duplicates, resulting in a weld index set. In the sidewall point cloud where the top and bottom plates have been removed, based on the index set... Extract the corresponding points to form the final high-precision three-dimensional weld positioning point set. Because it originates from the original normalized point cloud system, each point carries real, high-precision three-dimensional physical coordinates. The final output point set This refers to a high-precision three-dimensional weld seam point cloud of the tank body.

[0161] The three-dimensional weld point set output by this invention has wide application value. Two typical application scenarios are listed below:

[0162] Application Scenario 1: 3D inspection and quality assessment of welds.

[0163] The three-dimensional point set of the weld extracted in this invention can be directly used for three-dimensional spatial positioning and morphological analysis of the weld. By comparing the weld point set with the tank design model and reference geometry, morphological parameters such as weld position deviation, width variation, and excess height distribution can be quantitatively calculated, enabling three-dimensional and quantitative evaluation of weld quality. Simultaneously, this three-dimensional weld information can serve as core data in the digital archive of the storage tank, supporting subsequent wall deformation analysis, corrosion thinning assessment, and full life-cycle health management.

[0164] Application Scenario 2: Path planning and automatic flaw detection for wall-climbing robots based on weld seam positioning.

[0165] The three-dimensional spatial location information of the weld seam extracted by this invention can be further used as the basis for path planning for wall-climbing robots or autonomous inspection devices. Specifically, using the three-dimensional coordinates of the weld seam as the navigation target point, an optimal inspection path covering all weld seams is generated through a path planning algorithm, guiding the wall-climbing robot to move autonomously along the weld seam trajectory. The robot is equipped with an eddy current probe, ultrasonic sensor, or phased array detection equipment to perform close-range automated non-destructive testing of the weld seam along the planned path. This application mode realizes an intelligent closed-loop inspection operation of "first positioning (three-dimensional weld seam extraction of this invention), then navigation (path planning based on weld seam coordinates), and finally flaw detection (close-range detection by sensors on the robot)," significantly improving the automation level and operation efficiency of non-destructive testing of tank weld seams and reducing the safety risks of manual high-altitude operations.

[0166] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for three-dimensional extraction of tank weld seams based on multimodal fusion and topological prior, characterized in that: The point cloud data of the tank's sidewalls is acquired and laterally segmented along the main axis of the sidewalls. The tank's sidewalls are then fitted with the point sets within each slice to obtain the main axis information of the tank. Based on the main axis information, the point cloud data of the side wall plate around the main axis is unfolded to obtain the unfolded point cloud data of the side wall plate. The unfolded sidewall panel is divided into blocks and meshed, and the attributes of each point are projected onto the mesh pixels to generate a multimodal image; Based on the multimodal image, weld seam pixels are extracted, and the extracted weld seam pixels are back-projected onto the point cloud of the side wall plate to obtain the weld seam points of the tank.

2. The method for three-dimensional extraction of tank weld seams based on multimodal fusion and topological prior as described in claim 1, characterized in that: Based on the three-dimensional coordinates in the point cloud data, the extreme midpoint of the tank cross section in the XY plane is calculated, and combined with the minimum value of the Z-axis in the height direction of the tank, a translation vector is constructed to perform translation transformation on all points, and a normalized point cloud dataset is obtained. Based on the histogram distribution characteristics of the tank along the Z-axis after the translation transformation, the top cover and bottom plate of the tank are removed to obtain the point cloud data of the side wall plates of the tank.

3. The method for three-dimensional extraction of tank welds based on multimodal fusion and topological prior as described in claim 1, characterized in that: Divide the tank cross-section into slices along the height direction of the tank sidewall and obtain the point set within each slice; For each slice point set, its planar coordinates are extracted, and the shape of the slice point set is fitted; for cylindrical tanks, the random sampling consensus algorithm is used to fit the slice circle through the planar coordinates. Specifically, within each slice point set, three or more points are randomly selected, and the temporary center and radius are calculated using the formula for calculating the circumcircle. The distance from all points in the slice to the circle is calculated. If the distance is less than a threshold, it is recorded as an interior point. After iterative calculation, the circle model with the most interior points is obtained, and all interior points under the circle model are extracted. The least squares method is used to solve for the center and radius of the purified circle in the slice. Based on the shape fitted by the slice point set, the main axis information of the tank is estimated; Specifically, the center and radius of the main axis, which serve as the main axis information of the tank, are obtained based on the median of the center and radius of each purified slice or the mean of the center and radius after removing abnormal slices.

4. The method for three-dimensional extraction of tank weld seams based on multimodal fusion and topological prior as described in claim 3, characterized in that: Calculate the radial distance of a point on the sidewall panel relative to the center of the circle; Based on the difference in azimuth angle between each point and the center of the circle, the seam azimuth angle is set between the azimuth differences; Map the azimuth of each point to a standard period interval starting from the azimuth of the joint to obtain the mapped azimuth. Construct the three-dimensional point cloud coordinates of the unfolded tank sidewall panel, wherein the product of the mapped azimuth angle and the radius is the X-axis coordinate, the difference between the radial distance and the radius is the Y-axis coordinate, and the difference between the original Z-axis coordinate of the current point and the Z-axis coordinate of the lowest point of the tank sidewall panel point cloud is used as the new Z-axis coordinate of the current point.

5. The method for three-dimensional extraction of tank weld seams based on multimodal fusion and topological prior as described in claim 4, characterized in that: Calculate the azimuth angle of each point relative to the center of the circle; Sort all azimuth angles by size to calculate the difference between azimuth angles of adjacent points, and calculate the cross-boundary angle difference at the junction of the first and last points to form an angle difference set; The midpoint angle of the maximum angular difference is taken as the azimuth angle of the joint.

6. The method for three-dimensional extraction of tank weld seams based on multimodal fusion and topological prior as described in claim 1, characterized in that: The unfolded tank sidewalls are physically divided into sections; Set a target resolution for the sub-blocks and adaptively adjust the pixel size based on the actual physical extreme aspect ratio of the sub-blocks; Map the coordinates of each point within the sub-block to pixel coordinates; Perform neighborhood grid dilation on a pixel, and define the attributes of all grid pixel projection points within the dilated region; Multimodal images are generated based on pixels that contain point attributes.

7. The method for three-dimensional extraction of tank weld seams based on multimodal fusion and topological prior as described in claim 6, characterized in that: The point cloud data includes three-dimensional coordinates, color information, and reflection intensity; the multimodal image includes a color map, a reflection intensity color map, and a normal change rate color map. Assigning color to the corresponding point of the pixel within the grid to obtain a color map; Calculate the relative values ​​of the reflection intensity of the point cloud within the sub-block, and assign grayscale values ​​to the pixels within the grid based on the relative values ​​to obtain the reflection intensity color map; The surface normal change rate of a point is calculated and subjected to nonlinear logarithmic transformation. The logarithmic curvature is normalized and grayscale stretched to obtain the geometric abrupt change feature value of a single point. The mean of the feature values ​​in the neighborhood is used as the grayscale value of the pixel in the normal change rate map to obtain the normal change rate color map.

8. The method for three-dimensional extraction of tank weld seams based on multimodal fusion and topological prior as described in claim 1, characterized in that: The process for extracting weld seam pixels is as follows: Edge extraction is performed on each modal image to generate edge binary maps. Linear structuring elements are used to perform closing and opening operations on the edge binary maps to obtain reconstructed candidate weld binary maps for each modality. Calculate the grayscale standard deviation of the current segmented multimodal image to measure the saliency of modal information, and construct the weight coefficients of each modal image based on the grayscale standard deviation; The reconstructed binary images of each modal candidate weld are weighted and superimposed using the weighting coefficients to obtain the pixel-level probability of weld existence. Based on the probability of weld seam existence and the set judgment threshold, when a pixel is judged as a weld seam in multiple high-contrast modes, or has a strong response in a certain high-contrast mode, the pixel is activated, and a fused weld seam binary image is obtained.

9. The method for three-dimensional extraction of tank weld seams based on multimodal fusion and topological prior as described in claim 8, characterized in that: On the fused binary weld image, constraint optimization is performed using the physical properties of the longitudinal and transverse weld grid of the tank, as follows: Angle constraints are set on the fused binary weld map to filter weld lines; Cluster the selected line segments to fit the set of baseline lines for the main weld skeleton within the block; For any pixel in the fused binary weld image, calculate its distance to the nearest set of baselines; If the distance is greater than the set maximum pixel deviation threshold, the point is determined to be a non-weld noise point, and the corresponding pixel value on the fused weld binary image is set to 0. For a well-fitted baseline set, if a pixel interval with a continuous value of 0 is found on the grid line, and the break length of the interval is less than the set critical pixel length, then the pixel value of the broken interval is repaired and filled with 1.

10. The method for three-dimensional extraction of tank weld seams based on multimodal fusion and topological prior as described in claim 1, characterized in that: During block projection, record the original index of the sidewall point cloud enclosed by each pixel; Extract all point indices corresponding to weld pixels and remove duplicates to obtain a weld index set; In the point cloud of the side wall plate, the corresponding points are extracted according to the weld index set, which serves as the final three-dimensional weld positioning point set.

Images