A complex curved surface pipe defect reconstruction method and system and medium

Abstract

The present application relates to the technical field of image data processing and three-dimensional image reconstruction, and discloses a complex curved surface pipe defect reconstruction method, system and medium, comprising: collecting annular multi-view projection fringe images of a complex curved surface pipe, generating fringe images and initial three-dimensional point clouds, and converting the point clouds to a unified three-dimensional coordinate system under a clamp reference; extracting fringe centers, recording fringe image quality data and collecting optical path data; reconstructing a defect-free reference pipe curved surface based on a point cloud with measurement data and determining reference curved surface data; calculating point cloud measurement reliability and pipe curved surface defect residual determination values according to fringe image quality data, collecting optical path data, reference curved surface data and signed normal residual, generating a defect starting point set, a defect connected region and a defect boundary; performing secondary curved surface reconstruction in the defect connected region to obtain defect area reconstruction point clouds, and generating a complex curved surface pipe defect three-dimensional model and a single-channel defect depth map.

Description

Technical Field

[0001] This invention relates to the fields of image data processing and three-dimensional image reconstruction technology, and more specifically, to a method, system, and medium for reconstructing defects in complex curved pipe fittings. Background Technology

[0002] Complex curved pipe fittings are widely used in fluid transportation, pressure pipelines, heat exchange pipelines, and equipment connection parts. Elbows, reducers, tees, and pipe fittings with transition fillets simultaneously exhibit cylindrical curvature, bending curvature, and transition curvature on their surfaces. When inspecting defects in these fittings, on-site methods often include projected fringe 3D measurement, laser fringe scanning, or multi-view point cloud acquisition to obtain surface data. Defects such as dents, corrosion pits, and scratches are then identified through fringe center positioning, triangulation, point cloud registration, reference surface fitting, and residual analysis.

[0003] In conventional processing, stripe images are typically filtered and centered to form point clouds. Defects are then screened based on differences in neighborhood normals, the distance from points to the fitted surface, or grayscale changes in the 2D unfolded image. For straight pipe outer walls, gently curved surfaces, or areas with weak surface reflection, this process meets general inspection needs. However, for complex curved pipes, the surface normal changes continuously along the arc length, and the angle between the projection illumination direction, camera observation direction, and surface normal changes with position. Furthermore, specular reflections are easily generated on the pipe's metal surface, coated surface, or wetted inner wall, causing localized overexposure, stripe breaks, and center positioning shifts in the stripe image. The sampling intervals in curved inner and outer arcs and transition regions also exhibit directional differences, resulting in a different point cloud neighborhood distribution compared to the straight pipe region.

[0004] Under the aforementioned conditions, relying solely on adjacent normal angle thresholds, fixed neighborhood point cloud fitting, or ordinary 2D unfolded maps for defect judgment can easily lead to the identification of spurious residuals caused by specular reflection, overexposure, occlusion, and differences in sampling intervals as defect residuals. Furthermore, at the junctions of inner and outer arcs of elbows, tee intersections, and transitions between different diameters, actual depressions, corrosion pits, or scratch boundaries may be filled in during surface fitting, hole patching, or smoothing processes, resulting in deviations in defect boundary and depth representation. For inspection processes that require outputting 3D defect models and single-channel defect depth maps, providing only defect scores or single threshold judgment results is insufficient to meet the data requirements for subsequent dimensional verification, defect archiving, and operational condition comparison. Summary of the Invention

[0005] This invention provides a method, system, and medium for reconstructing defects in complex curved pipe fittings, which solves the technical problems mentioned in the background art.

[0006] This invention provides the following technical solution: Firstly, a method for reconstructing defects in complex curved surface pipe fittings is provided, applied to the image data processing of three-dimensional reconstruction of defects in complex curved surface pipe fittings. The complex curved surface pipe fittings include elbows, reducing sections, tee junctions, and fittings with transition fillets, comprising: Annular multi-view structured light images of the complex curved surface pipe are acquired to generate stripe images and initial three-dimensional point clouds. The initial three-dimensional point clouds are then converted to a unified three-dimensional coordinate system under the fixture reference to obtain a unified point cloud. Extract the stripe center of the stripe image, and record the stripe image quality data and acquisition optical path data of the sampling points in the unified point cloud to obtain a point cloud with measurement data; Based on the point cloud with measurement data, the surface of the defect-free reference pipe fitting is reconstructed, and the reference surface data is determined. Based on the stripe image quality data, the acquisition optical path data, the reference surface data, and the signed normal residual of the sampling point relative to the defect-free reference pipe surface, the point cloud measurement reliability and the defect residual judgment value of the pipe surface are calculated. Based on the residual judgment value of the defect on the pipe fitting surface, generate a set of defect starting points, a defect connected region, and a defect boundary; Within the defective connected region, a quadratic surface is reconstructed according to the point cloud measurement reliability and the defect boundary to obtain the defective region reconstructed point cloud; Based on the point cloud reconstruction of the defect area, a 3D model of the defect in the complex curved pipe fitting and a single-channel defect depth map are generated.

[0007] Preferably, obtaining the unified point cloud includes: A structured light acquisition system consisting of a camera and a projector with calibrated intrinsic and extrinsic parameters was used to perform ring-shaped multi-view acquisition. The camera intrinsic parameters, projector intrinsic parameters, camera and projector extrinsic parameters, fixture coordinate system, and rigid body pose of the pipe remain unchanged during a single scan. Record the pose matrix of each sampling frame from the coordinate system of the acquisition device to the unified three-dimensional coordinate system; Local point clouds are obtained based on the structured light triangulation results, and the pose matrix is ​​used to convert each local point cloud into the unified point cloud. A point cloud source record is established for the 3D points in the unified point cloud. The point cloud source record includes point coordinates, source image, pixel neighborhood, camera line of sight, and projection illumination direction.

[0008] Preferably, obtaining the point cloud with measurement data includes: The source image and pixel neighborhood corresponding to the sampling point are determined based on the point cloud source record; Perform edge-preserving filtering on each frame of the striped image; The position of the bright ridge line is determined on the fringe normal section, and the sub-pixel center is corrected according to the gray centroid of the ridge line neighborhood. Update the sampling point coordinates according to the sub-pixel center and the calibration parameters of the structured light acquisition system; Record the local saturation ratio, fringe gradient intensity, fringe break number, camera line of sight, and projection illumination direction for each sampling point; In this context, overexposed pixels in the stripe image are included in the local saturation ratio, and the stripe break segments in the stripe image are numbered to form the stripe break number.

[0009] Preferably, determining the reference surface data includes: Local covariance normal and curvature estimation are performed on the point cloud with measurement data; The centerline of the pipe fitting is extracted based on the point cloud with measurement data, and the reference surface patches are divided according to the aggregation trend of the normal on the unit sphere and the arrangement order along the centerline of the pipe fitting. The reference surface pieces are fitted with least squares respectively, and tangential continuous splicing is performed at the transition between adjacent reference surface pieces to obtain the defect-free reference pipe surface. For each sampling point, determine the nearest reference point, reference surface normal, first surface tangent, second surface tangent, first reference curvature and second reference curvature, and use them as the reference surface data.

[0010] Preferably, the calculation of point cloud measurement reliability and pipe fitting surface defect residual judgment value includes: On the surface of the defect-free reference pipe, a range of nearby points is established for each sampling point according to the geodesic distance of the surface, and a surface adjacency diagram is established based on the range of nearby points. Calculate the signed normal residual based on the sampling point, the nearest reference point, and the reference surface normal; Determine the sensitive direction of specular reflection based on the direction of projected illumination and the direction of the camera's line of sight; The reliability of the point cloud measurement is calculated based on the local saturation ratio, fringe gradient intensity, sampling interval along the tangent of the first surface and the tangent of the second surface, and the angle between the reference surface normal and the specular reflection sensitive direction. The defect residual judgment value of the pipe fitting surface is calculated based on the point cloud measurement reliability, the median deviation of the signed normal residual within the range of adjacent points on the surface, the first reference curvature, and the second reference curvature.

[0011] Preferably, generating the set of defect initiation points, the defect connected region, and the defect boundary includes: The sampling points where the residual judgment value of the defect on the pipe fitting surface is greater than the sum of the median value and the median deviation of the judgment value within the range of the adjacent points of the surface are determined as the set of defect starting points. Starting from the set of defect initiation points, region growth is performed on the sampling points along the signed normal residual with the same sign and the point cloud measurement confidence level on the adjacency graph of the surface to form the defect connected region. The set of sampling points on the surface of the defect-free reference pipe that are geodetically adjacent to the defect-connected region but do not belong to the defect-connected region is determined as the defect boundary. Sampling points with different stripe fracture numbers are allowed to be grouped into the same defect connectivity region only when they are adjacent in the surface adjacency diagram.

[0012] Preferably, obtaining the reconstructed point cloud of the defect region includes: Within each defect connectivity region, a sampling point is taken as the center point, and a local fitting point range is established for the center point. The local fitting point range is included by sampling points within the same defect connectivity region and boundary points in the defect boundary according to the surface geodesic distance, until the quadratic surface design matrix reaches six columns of full rank. The range of the local fitting point does not cross other defect connected regions or stripe break segments with different stripe break numbers. Project the support points within the local fitting point range onto a reference tangent plane with the nearest reference point of the center point as the origin and the tangent of the first surface and the tangent of the second surface as the coordinate axes; The point cloud measurement reliability is used as the fitting weight to solve the quadratic surface fitting coefficient, and the defect reconstruction points are generated from the quadratic surface fitting coefficient.

[0013] Preferably, generating a 3D model of a defect in a complex curved pipe fitting and a single-channel defect depth map includes: The surface development coordinates of the defect-free reference pipe surface are generated based on the surface adjacency diagram. According to the surface unfolding coordinates, the defect reconstruction points in each defect connected region are sorted. Within the same defect connectivity region, the sorted defect reconstruction points are subjected to restricted triangulation to generate a defect triangular mesh, and the defect triangular mesh is used to form a three-dimensional model of the defect of the complex curved pipe fitting. For each defect reconstruction point, calculate the defect depth value relative to the nearest reference point along the normal of the reference surface; The defect depth value is rasterized according to the coordinate unfolding of the surface to form the single-channel defect depth map; The system maintains a one-to-one correspondence between the vertex indices of the 3D model of the complex curved pipe fitting defect and the pixel indices of the single-channel defect depth map.

[0014] Secondly, a complex curved surface pipe fitting defect reconstruction system includes a processor, a memory, an image and point cloud unified module, a stripe measurement data recording module, a reference pipe fitting curved surface reconstruction module, a defect residual judgment module, a defect region generation module, a defect curved surface reconstruction module, and a defect model output module. The image and point cloud unification module is used to obtain a unified point cloud; The stripe measurement data recording module is used to obtain a point cloud with measurement data; The reference pipe surface reconstruction module is used to obtain the defect-free reference pipe surface and reference surface data; The defect residual judgment module is used to calculate the point cloud measurement reliability and the defect residual judgment value of the pipe fitting surface; The defect region generation module is used to generate a set of defect starting points, a defect connected region, and a defect boundary. The defect surface reconstruction module is used to obtain the reconstructed point cloud of the defect region; The defect model output module is used to generate a three-dimensional model of defects in complex curved pipe fittings and a single-channel defect depth map; The processor executes instructions in the memory to implement the method for reconstructing defects in complex curved pipe fittings.

[0015] Thirdly, a computer-readable storage medium storing program instructions thereon, which, when executed by a processor, implement the aforementioned method for reconstructing defects in complex curved pipe fittings.

[0016] Compared with existing technologies, the beneficial effects include: This invention retains the source image, pixel neighborhood, camera viewing direction, and projection illumination direction in the sampling points. It incorporates fringe saturation, fringe gradient, fringe breakage, surface curvature, surface sampling interval, and normal residual into the same reconstruction process, thus constraining the pseudo-residuals caused by specular reflection, overexposure, occlusion, and sampling anisotropy in defect judgment and surface fitting. Simultaneously, it generates a 3D defect model and a single-channel defect depth map by using the defect-free reference pipe surface, surface adjacency map, defect boundary, and constrained triangulation. This separates the curvature changes at elbows, reducing diameter sections, and tee junctions from the processing of depressions, corrosion pits, and scratch boundaries, reducing misjudgments, boundary filling, and depth deviations caused by fixed thresholds, fixed neighborhoods, and ordinary unfolding methods. It also ensures that the output results are traceable to the acquired images and point cloud data. Attached Figure Description

[0017] Figure 1 This is a flowchart of a method for reconstructing defects in complex curved pipe fittings according to the present invention. Detailed Implementation

[0018] The subject matter described herein will now be discussed with reference to exemplary embodiments. It should be understood that these embodiments are discussed only to enable those skilled in the art to better understand and implement the subject matter described herein, and changes may be made to the function and arrangement of the elements discussed without departing from the scope of this specification. Various processes or components may be omitted, substituted, or added as needed in the examples. Furthermore, features described in some examples may be combined in other examples.

[0019] Figure 1 Flowchart of a method for reconstructing defects in complex curved pipe fittings. Figure 1 S101 is used to acquire a ring-shaped multi-view structured light image and generate an initial 3D point cloud; S102 is used to convert the initial 3D point cloud into a unified point cloud; S103 is used to extract the fringe center and record the fringe image quality data and the acquired optical path data to obtain a point cloud with measurement data; S104 is used to reconstruct the surface of the defect-free reference pipe fitting and determine the reference surface data; S105 is used to calculate the point cloud measurement reliability and the defect residual judgment value of the pipe fitting surface; S106 is used to generate the defect starting point set, defect connected region and defect boundary; S107 is used to perform secondary surface reconstruction and obtain the defect region reconstruction point cloud; S108 is used to generate a 3D model of the defect in the complex surface pipe fitting and a single-channel defect depth map. Figure 1 The striped image in the image serves as the image source for S103. Figure 1 The unified point cloud, point cloud with measurement data, defect-free reference pipe surface, reference surface data, point cloud measurement reliability, pipe surface defect residual judgment value, defect starting point set, defect connected region, defect boundary, defect region reconstructed point cloud and single-channel defect depth map are passed to the subsequent processing in the direction of the arrow.

[0020] Example 1: In this embodiment, the complex curved pipe fitting includes elbows, reducers, tee junctions, and fittings with transition fillets. The complex curved pipe fitting is fixed to a fixture reference, and the structured light acquisition system uses a camera and projector with calibrated internal and external parameters to perform ring-shaped multi-view acquisition. The structured light acquisition system uses a structured light acquisition head consisting of a single camera and a single projector, which is driven by a precision turntable to perform a ring-shaped motion around the pipe fitting. The rotation axis of the turntable coincides with the Z-axis of the fixture reference coordinate system, and the ring radius is set to be greater than the maximum radial dimension of the pipe fitting. Doubled The number of viewpoints is calculated per to A uniformly distributed sampling perspective.

[0021] Before data acquisition, the alignment of the turntable axis with the fixture reference coordinate system is completed using a standard calibration plate. During acquisition, the pipe is fixed and stationary on the fixture reference. When the acquisition head rotates with the turntable to each viewing position, the projector is triggered to project a stripe pattern, and the camera simultaneously acquires images. During a single scan, the camera intrinsic parameters, projector intrinsic parameters, camera and projector extrinsic parameters, fixture coordinate system, and the rigid body pose of the pipe remain unchanged. The pose matrix from the acquisition device coordinate system to the unified three-dimensional coordinate system under the fixture reference corresponding to each sampling frame is calculated using the turntable angle and a pre-calibrated hand-eye matrix and pre-stored.

[0022] The origin of the fixture reference coordinate system is defined as the center of the fixture positioning end face. The X-axis is along the horizontal reference direction of the fixture positioning end face, the Y-axis is along the vertical reference direction of the fixture positioning end face, and the Z-axis is along the axial reference direction of the pipe installation. The calibration of the acquisition device coordinate system and the fixture reference coordinate system uses a standard ceramic calibration sphere. The calibration sphere is fixed at multiple different positions on the fixture reference. Point clouds of the calibration sphere are acquired using a structured light acquisition system, and the coordinates of the sphere center are fitted. Simultaneously, the nominal coordinates of the calibration sphere center in the fixture reference coordinate system are recorded. The transformation matrix from the acquisition device coordinate system to the fixture reference coordinate system is obtained through multi-point pose registration. The pose matrix corresponding to each viewpoint under multi-view acquisition is calculated in real time using the turntable angle and the reference pose matrix, and is pre-stored in the acquisition system.

[0023] A projector projects a phase-shifted fringe pattern onto the surface of a complex curved pipe fitting, while a camera simultaneously captures the fringe image modulated by the complex curved pipe fitting surface. The phase-shifted fringes are sinusoidal, with the fringe projection direction parallel to the pixel row direction of the projector. The step size for the four-step phase shift is... A single set of phase-shifting fringes contains A striped image with sequentially shifted frame phases. Gray code uses binary-coded stripes in the same direction as the phase-shifted stripes. The number of Gray code bits is determined based on the projector's horizontal resolution and the number of pixels per period stripe, satisfying the following relationship: (Formula 1); in, Gray code bits, This represents the number of pixels corresponding to a single-period phase-shifted fringe. This represents the total number of pixels in the horizontal direction of the projector. Gray code images and phase-shifted fringe images are projected and acquired sequentially according to their encoding order. During decoding, each Gray code image is first binarized to obtain a binary encoded value, and then converted into a decimal fringe level. .

[0024] When calculating the wrap-around phase of a four-step phase-shifted image, the quadrant is determined using a two-parameter arctangent function, expressed as: (Formula 2); in, , , , These are the grayscale values ​​of the four-step phase-shifted images. The phase is wrapped. The fringe order is determined by the Gray code image. Then, the absolute phase is represented as: (Formula 3); in, For absolute phase, This is a stripe-level correction. Error correction at periodic transitions is achieved by checking the phase continuity of adjacent pixels. Based on absolute phase... Find corresponding points in the projector image, and calculate the local point cloud by combining the camera intrinsic matrix, the projector intrinsic matrix, and the rotation matrix and translation vector determined by the camera and projector extrinsic parameters.

[0025] The mapping relationship between projector pixel coordinates and absolute phase is determined through projector intrinsic parameter calibration. The projector's horizontal pixel coordinates... with absolute phase Satisfies a linear mapping relationship: (Formula 4); in, These are the pixel coordinates in the horizontal direction of the projector. For absolute phase, This represents the number of projector pixels corresponding to a single-period phase-shifted fringe. This represents the phase offset corresponding to the origin of the projector image. During matching, the sub-pixel coordinates of the corresponding points on the projector are obtained by interpolation based on the absolute phase of the camera pixels using the mapping relationship described above.

[0026] The triangulation process is achieved through spatial constraints between camera pixel rays and projector pixel rays. The linear DLT method transforms the two ray equations into a system of linear equations, and solves for the coordinates of the 3D points in the camera coordinate system using the least squares method. Then, the coordinates are transformed to the acquisition device coordinate system using extrinsic parameters. The camera pixel rays and projector pixel rays are represented as follows: (Formula 5); (Formula 6); in, These are the coordinates of a 3D point in the camera coordinate system. Let be the homogeneous coordinates of the camera pixels. Let be the homogeneous coordinates of the projector pixels. For the camera intrinsic parameter matrix, This is the intrinsic parameter matrix of the projector. and The rotation matrix and translation vector are determined by the extrinsic parameters of the camera and projector. and is the scale factor. Solving the above two expressions simultaneously yields the local point cloud in the coordinate system of the acquisition device. The local point clouds of all sampled frames together constitute the initial 3D point cloud.

[0027] Example 2: In this embodiment, the coordinates of each point in the local point cloud are expanded to homogeneous coordinates, and then transformed to a unified three-dimensional coordinate system under the fixture reference using the pose matrix corresponding to the sampling frame. The transformation relationship is expressed as: (Formula 7); in, Let these be the homogeneous coordinates of a local point. To unify the homogeneous coordinates of points in a three-dimensional coordinate system, This represents the pose matrix from the coordinate system of the acquisition device to a unified 3D coordinate system under the fixture reference. The local point clouds of all sampled frames are transformed, merged, and downsampled to form... Figure 1 The unified point cloud in [the context of the sentence].

[0028] When merging point clouds from multiple perspectives, a voxel deduplication rule is used to establish a cubic pixel mesh based on a unified 3D coordinate system, with the voxel side length set to... to When multiple sampling points exist within the same grid, the source records of all points are retained, and the grid center coordinates are used as the coordinates of the fused points. Downsampling employs a voxel downsampling method, with the voxel side length set according to the pipe size and detection accuracy requirements. to During downsampling, only one sampling point is retained within each voxel. The coordinates of the sampling point are taken as the average of the coordinates of all points within the voxel, and the corresponding point cloud source record retains the source information corresponding to the point with the highest confidence within the voxel.

[0029] To unify the 3D points in the point cloud, a point cloud source record is established. This record includes point coordinates, the source image, pixel neighborhood, camera view direction, and projection illumination direction. The pixel neighborhood in the point cloud source record is centered on the pixel corresponding to the sampling point. A square pixel window, the window area includes the center pixel and the surrounding pixels. OK The adjacent pixels of the column. When the center pixel is located at the edge of the image, causing the window to extend beyond the image boundary, only pixels within the valid image area of ​​the window are retained for subsequent calculations, and the insufficient parts are not zero-padded or extrapolated.

[0030] After multi-view point clouds are transformed to a unified 3D coordinate system, they are stored sequentially according to viewpoint order, with points within each viewpoint initially indexed in pixel row and column order. During merging and downsampling, a new continuous index is assigned to each output sampled point, and a mapping table is established between the new index and the original viewpoint point indices. The source record corresponding to a point deleted during downsampling is synchronously removed from the storage list. When the same fused point corresponds to multiple sets of source records, they are stored in order of confidence and share the same index number.

[0031] The camera's line of sight is defined by the direction from the camera's optical center to a 3D point, and is normalized as follows: (Formula 8); The projection illumination direction is defined by the direction from the projector's optical center to the three-dimensional point, and is normalized as follows: (Formula 9); in, Let these be the coordinates of a 3D point in a unified 3D coordinate system. Let the coordinates of the camera's optical center be in a unified three-dimensional coordinate system. This represents the coordinates of the projector's optical center in a unified 3D coordinate system. The point cloud source record and the sampling points in the unified point cloud are stored using the same index, enabling each sampling point in the unified point cloud to be traced back to the source image, pixel neighborhood, camera line of sight, and projection illumination direction.

[0032] Example 3: In this embodiment, the source image and pixel neighborhood corresponding to the sampling point are determined based on the point cloud source record. Edge-preserving filtering is performed on each frame of the stripe image. This filtering reduces dark noise and ambient light interference while preserving the stripe edges. Edge-preserving filtering employs bilateral filtering or a filtering process with the same edge-preserving effect. When bilateral filtering is used, the spatial domain standard deviation is set to... 1 pixel, grayscale standard deviation set to 1 Several gray levels, the filtering window uses Pixel window. During filtering, for each pixel, a weighted average gray value of neighboring pixels is calculated by combining spatial distance weight and gray-level difference weight. This suppresses dark noise and ambient light interference while preserving the gray-level abrupt changes at the fringe edges. Alternative filtering methods must meet the edge-preservation criterion, meaning the gray-level gradient magnitude retention rate at the fringe edge positions is not less than a certain percentage of the original image's value. Meanwhile, the noise standard deviation in flat areas decreased by no less than .

[0033] Stripe normal adopted The Sobel operator of the window calculates the local gray-level gradient, and the gradient direction is the fringe normal. The sign of the gradient direction is determined by which side of the normal the bright area of ​​the fringe is located on; it is uniformly stipulated that the bright area is located on the side of the positive normal. The fringe normal cross section is a straight line section passing through the center of the pixel to be processed and extending along the fringe normal, and the total length of the cross section is set to... Each pixel extends in both positive and negative directions from the center pixel as the midpoint. pixels, cross-sectional sampling step size is Each pixel is truncated to the image boundary when the cross-section extends beyond the image boundary, retaining only the valid pixel segment.

[0034] Bright fringe ridges are located using the zero-crossing point determination method based on the first derivative of grayscale. The first-order difference of grayscale is calculated on the fringe normal section; the zero-crossing point where the first-order difference changes from positive to negative is the location of the bright fringe ridge. When multiple zero-crossing points appear on a single section, the zero-crossing point with the largest grayscale value is selected as the valid ridge position. The ridge neighborhood is defined as the area on either side of the ridge position. For pixels within a pixel range, the grayscale centroid correction is performed by calculating the sub-pixel position using a grayscale-weighted average within the cross-section. The grayscale centroid calculation formula is as follows: (Formula 10); in, These are the position coordinates of each pixel in the ridge neighborhood. This represents the grayscale value at the corresponding position. This is the subpixel center position. When... When the gray level is zero, the bright ridge line location does not exist, or the sum of gray levels in the neighborhood is less than a preset gray level threshold, no gray level centroid correction is performed in the ridge line neighborhood, and the corresponding location is treated as a ridge break point. This preset gray level threshold is adaptively adjusted to the average gray level of the image based on the overall gray level of the image. .

[0035] When updating the sampling point coordinates based on the sub-pixel center and the calibration parameters of the structured light acquisition system, the absolute phase value of the corresponding pixel is first corrected according to the sub-pixel fringe center position. The correction method involves substituting the sub-pixel offset into the phase-pixel mapping relationship to calculate the phase correction amount, which is then superimposed on the original absolute phase to obtain the corrected absolute phase. The corrected absolute phase is then used to re-execute the projector corresponding point matching and triangulation calculations to obtain the updated 3D point coordinates. The updated coordinates directly replace the sampling point coordinates corresponding to the original point cloud. Fringe break points are not updated; the original triangulation results are retained, and the corresponding fringe break number is recorded.

[0036] For each sampling point, record the local saturation ratio, fringe gradient intensity, fringe break number, camera viewing direction, and projection illumination direction. Overexposed pixels in the fringe image are included in the local saturation ratio. The preset saturation grayscale value is determined based on the camera's upper bit depth limit. A bit-level grayscale camera, with a grayscale upper limit of 1. The preset saturation grayscale value is set to For cameras with higher bit depth, the preset saturation grayscale value is set to the upper limit of grayscale minus... There are several gray levels. Pixels with gray values ​​greater than or equal to a preset saturation gray value in their neighborhood are considered overexposed and are included in the local saturation ratio statistics. The local saturation ratio is expressed as: (Formula 11); in, This represents the number of pixels in the pixel's neighborhood that are identified as overexposed. The total number of pixels in the neighborhood of a pixel. This represents the local saturation ratio. The fringe gradient intensity is expressed as: (Formula 12); in, For pixel neighborhood, The grayscale gradient along the normal direction of the stripes. The intensity of the fringe gradient is denoted as .

[0037] The criterion for determining the continuity of the fringe center is: the offset of the sub-pixel position of the fringe center corresponding to two adjacent pixels is less than 1. When there are 1 pixel or more, it is considered continuous; greater than or equal to 1 pixel. An interruption is defined as a breakpoint occurring when one or more pixels are broken. The length is less than [a certain value]. A continuous stripe segment of a pixel is uniformly identified as a break segment. Stripe break numbers are encoded sequentially within the image frame; each independent break segment within the same frame is assigned a unique positive integer number. The stripe break number corresponding to a point in a normal stripe region is... The fracture numbers for images from different viewpoints are independent. When merging point clouds, the fracture numbers corresponding to the original viewpoints are retained, and no unified numbering is performed across viewpoints. After completing the above recording, the following is obtained: Figure 1 Point cloud with measurement data.

[0038] Example 4: In this embodiment, local covariance normal and curvature estimation are performed on the point cloud with measurement data. Local covariance normal estimation uses a k-nearest neighbor method to construct a neighborhood point set, with k set to a value of [value missing]. That is, each sampling point selects the closest one. Each neighboring point participates in the covariance calculation. The neighborhood size can be adaptively adjusted according to the point cloud density, with the adjustment rule being that the average spacing between neighboring points remains at a certain percentage of the average spacing of the point cloud. Doubled Within a multiple of [number] times.

[0039] After establishing a neighborhood set for each sampling point, calculate the covariance matrix of the neighborhood set: (Formula 13); in, The number of neighboring points. The coordinates of the neighboring points, Let be the centroid coordinates of the neighborhood points. Perform eigenvalue decomposition on the covariance matrix; the eigenvector corresponding to the smallest eigenvalue is used as the normal vector. During viewpoint orientation consistency adjustment, the viewpoint orientation is defined as the direction from the camera's optical center to the sampling point. Calculate the dot product between the normal vector and the viewpoint orientation vector; if the dot product is negative, reverse the normal vector to ensure that the normal vectors of all points point towards the viewpoint. Curvature estimation is expressed as: (Formula 14); in, , , These are the eigenvalues ​​of the covariance matrix. For the smallest eigenvalue, This is the curvature used in curvature estimation. This curvature is used for judging local changes in the point cloud after local covariance normal estimation, extracting the centerline of the pipe fitting, and dividing the reference surface patch. It is not used as the first or second reference curvature.

[0040] The centerline extraction of the pipe fitting employs a layered slicing fitting method. The point cloud containing the measurement data is sliced ​​along the Z-axis of the fixture's reference coordinate system with a fixed step size. The slicing step size is set to the average spacing of the point cloud. The center line of each slice is obtained by performing circular fitting on the point cloud within each slice. All slice centers are then arranged in Z-axis order to obtain the initial centerline. For elbow sections, a circular centerline fitting method is used to fit the center line of the elbow region slices to obtain the elbow section centerline. For reducing sections, a conical axis fitting method is used to obtain the reducing section centerline. For tee junctions, the centerlines of the three branches are fitted separately, and a smooth spline curve is used to connect them in the junction area. Centerline breakpoints are connected using least-squares linear interpolation to finally obtain a complete and continuous pipe fitting centerline.

[0041] The reference surface patch division adopts a combination of normal clustering and centerline segmentation. First, the normal vectors of all sampling points are mapped onto the unit sphere. Then, the K-means clustering algorithm is used to cluster the normal vectors. The number of clusters is preset according to the pipe fitting structure type. to The system then categorizes the pipes into four types based on their arrangement along the pipe centerline: straight cylindrical plates, elbow annular plates, reducing conical plates, and tee transition plates. The boundaries of these transition regions are determined using a normal angle threshold; adjacent points with a normal angle greater than a certain threshold are considered transition regions. The location is determined by the boundary of the patch, and the boundary is used as... The transition zone of the points achieves smooth connection.

[0042] The straight cylindrical surface patches are least - squares fitted using a cylindrical parameter model. The fitting parameters include the direction of the cylinder axis, the position of the axis, and the radius of the cylinder. The fitting goal is to minimize the sum of the squares of the normal distances from all points to the cylindrical surface. The elbow toroidal surface patches are fitted using a torus parameter model. The fitting parameters include the radius of the central axis of the torus and the radius of the torus cross - section. The conical surface patches with different diameters are fitted using a conical parameter model. The fitting parameters include the cone axis, the position of the cone apex, and the half - cone angle. The three - way transition surface patches are fitted using a bi - quadratic Bezier surface, with the boundary curves of the three branches as the constraint boundaries. Tangential continuous splicing uses the parameter adjustment method. At the common boundary of adjacent surface patches, the parameters of one of the surface patches are adjusted to make the tangent vector directions at the boundary consistent, meeting the continuous requirement, and the normal deviation after splicing is less than and is judged as qualified. The least - squares fitting is respectively performed on the reference surface patches, and tangential continuous splicing is carried out at the transitions of adjacent reference surface patches to obtain Figure 1 the defect - free reference pipe fitting surface in

[0043] For each sampling point, the nearest reference point is determined on the defect - free reference pipe fitting surface. The nearest reference point is solved using the Newton iteration method, with the initial projection point of the sampling point as the iteration starting point. The position of the projection point is iteratively corrected along the surface tangent direction until the line connecting the projection point to the sampling point is parallel to the surface normal. The iteration convergence threshold is set to . The first surface tangent is defined as the unit tangent vector along the tangent direction of the pipe fitting center line, and the second surface tangent is defined as the unit tangent vector along the circumferential direction of the pipe fitting and perpendicular to both the first surface tangent and the reference surface normal. The first reference curvature and the second reference curvature are calculated through the first fundamental form and the second fundamental form of the parametric surface, corresponding to the curvature values in two principal directions respectively. All reference surface data are stored in one - to - one correspondence with the sampling points. The nearest reference point, the reference surface normal, the first surface tangent, the second surface tangent, the first reference curvature, and the second reference curvature together serve as Figure 1 the reference surface data in

[0044] Example Five: In this example, on the defect - free reference pipe fitting surface, a surface neighborhood point range is established for each sampling point according to the geodesic distance on the surface, and a surface adjacency graph is established based on the surface neighborhood point range. The geodesic distance on the surface is calculated using the Dijkstra shortest - path algorithm on the surface adjacency graph. The surface adjacency graph is constructed using geodesic Delaunay triangulation. With the sampling points on the defect - free reference pipe fitting surface as vertices, the three points with the closest geodesic distance are connected to form a triangular mesh, and the mesh edges are the edges of the adjacency graph. The preset geodesic distance threshold is set to The radius of each sampling point is the set of all sampling points whose geodesic distance to that point is less than a preset threshold. The shortest path is calculated by propagating the distance along the edges of the adjacency graph; the total path length is the geodesic distance between the two points.

[0045] The signed normal residual is calculated based on the sampling point, the nearest reference point, and the reference surface normal. The signed normal residual is expressed as: (Formula 15); in, These are the coordinates of the sampling point. The coordinates of the nearest reference point. As the reference surface normal, The sign normal residual. A positive value indicates that the sampling point is located outside the curved surface of the defect-free reference pipe fitting. A negative value indicates that the sampling point is located inside the curved surface of the defect-free reference pipe.

[0046] The specular reflection sensitive direction is determined based on the projection illumination direction and the camera's line-of-sight direction. Since both the camera's line-of-sight direction and the projection illumination direction are defined as directions pointing from the optical center to a 3D point, the effect of specular reflection is subsequently calculated using the absolute angle between the reference surface normal and the specular reflection sensitive direction. The specular reflection sensitive direction is expressed as: (Formula 16); in, Let the unit vector be the direction of the projected illumination. Let the unit vector be the direction of the camera's line of sight. is the unit vector for the direction sensitive to specular reflection.

[0047] Sampling interval along the tangent of the first surface The median of the differences between adjacent projected coordinates of all points within the vicinity of the surface along the tangent of the first surface is taken. During calculation, the coordinates of the vicinity points are first projected onto the tangent of the first surface at the center point to obtain one-dimensional projected coordinates. After sorting, the median of the differences between adjacent coordinates is calculated. Similarly, the sampling interval along the tangent of the second surface... The median value of the difference between adjacent projections is obtained by projecting neighboring points onto the tangent of the second surface.

[0048] Stable terms The value is taken as the median value of the fringe gradient intensity within the range of adjacent points on the surface. This is used to avoid the denominator being zero while ensuring numerical stability. Stability term Based on the average of the tangential sampling interval of the first surface and the tangential sampling interval of the second surface The value is determined and included in the denominator of the squared term of the sampling anisotropy factor, with the corresponding squared value. Stability term. The value is the sum of the absolute values ​​of the first reference curvature and the second reference curvature. Stable terms Based on the absolute deviation of the sign normal residual within the range of adjacent points on the surface The value is determined and included in the calculation as the corresponding squared value in the squared term of the signified normal residual median deviation. Stability term. Based on the median deviation of the residual value of the pipe fitting surface defect judgment value within the range of adjacent points on the surface The squared value is determined and included in the calculation of the squared term of the positional deviation in the judgment value. All stable terms are adaptively adjusted according to the corresponding statistics to ensure that the denominator is always greater than zero during the calculation process.

[0049] (Formula 17); (Formula 18); (Formula 19); (Formula 20); (Formula 21); The reliability of point cloud measurements is calculated based on the local saturation ratio, fringe gradient intensity, sampling intervals along the tangents of the first and second surfaces, and the angle between the reference surface normal and the specular reflection sensitive direction. The reliability of point cloud measurements is expressed as: (Formula 22); in, To measure the reliability of point cloud measurements, This represents the local saturation ratio. For the fringe gradient intensity, The range of points adjacent to the surface. The sampling interval is along the tangent of the first curved surface. The sampling interval is along the tangent of the second surface. and All are stable terms. This represents the median operator.

[0050] Based on the point cloud measurement reliability, the median deviation of the signed normal residual within the range of adjacent points on the surface, and the first and second reference curvatures, the defect judgment value for the pipe fitting surface is calculated. The median absolute deviation of the signed normal residual within the range of adjacent points on the surface is expressed as: (Formula 23); in, This represents the median deviation of the signed normal residual within the range of points near the surface. The curvature correction factor is expressed as: (Formula 24); in, As the first reference curvature, As the second reference curvature, For curvature correction factor, This is a stabilizing term. The curvature correction factor is used to offset the increased residual dispersion caused by changes in the curvature of the reference surface itself. The greater the difference between the first and second reference curvatures, the larger the curvature correction factor, and the stronger its suppressive effect on the residual judgment value of pipe fitting surface defects. The verification rule is: on a defect-free standard surface, the median deviation of the residual judgment value of pipe fitting surface defects in different curvature regions does not exceed [a certain value]. If the deviation exceeds [a certain threshold], then the curvature correction factor is considered valid; if the deviation exceeds [a certain threshold], then [the correction factor is considered valid]. This can be achieved by adjusting the stability parameters. The proportionality coefficient is adjusted, and the adjustment range of the proportionality coefficient is [missing information]. to .

[0051] The residual value for defects on the curved surface of pipe fittings is expressed as follows: (Formula 25); in, This is the residual value for judging defects on the curved surface of the pipe fitting. This is a stability term. This value simultaneously reflects the measurement reliability of the sampling points, the degree of dispersion of local residuals, and the curvature background of the reference surface.

[0052] S106: Generate the set of defect starting points, defect connected regions, and defect boundaries.

[0053] In this embodiment, the median value and median deviation of the judgment value of the defect residual of the pipe fitting surface within the range of the nearest points of the surface at each sampling point are first calculated. The median deviation of the judgment value is expressed as: (Formula 26); in, This refers to the median deviation of the judgment value for the defect residual on the pipe fitting surface within the range of points adjacent to the surface. Sampling points meeting the following conditions are determined as... Figure 1 The set of defect starting points in the data: (Formula 27); in, This is a stable term. The adaptive threshold is set based on local statistical characteristics and can automatically adapt to the residual fluctuation levels in different regions. When the overall residual in a local area is low, the threshold decreases accordingly to ensure the detection capability of weak defects; when the overall residual in a local area is high, the threshold increases accordingly to suppress false defects. When screening defect initiation points, the minimum density screening rule is also executed simultaneously. If the geodetic distance around a single defect initiation point is... If there are no other defect initiation points within the threshold range, they are identified as isolated noise points and removed, while only clustered defect initiation points are retained to participate in subsequent region growth.

[0054] Region growing uses the direct adjacency relationship of the surface adjacency graph as the criterion, only including sampling points with direct adjacent edges to the current region boundary points in the growing candidate list. During growing, the signed normal residual sign of the defect starting point set is used as the unified sign of the current region. A candidate point is included in the current region if its signed normal residual sign is the same as the region sign, its point cloud measurement confidence is not zero, and it has not yet been assigned to another defect-connected region. Points with zero signed normal residual do not change their region sign; they are only included if the signs of adjacent regions are consistent. When multiple defect starting points result in adjacent regions with consistent residual signs, they are merged into the same defect-connected region.

[0055] The defect boundary is a single-layer set of adjacent points, i.e., all sampling points that have a direct geodesic adjacency with the defect-connected region but are not themselves part of the defect-connected region. Defect boundary points only participate in the construction of the support point range for quadratic surface fitting and are not included in the defect-connected region itself. Defect boundaries spanning reference surface patches remain continuous at tangentially continuous joints; at non-tangentially continuous patch boundaries, defect boundaries are automatically truncated and do not extend across patches. Sampling points with different fringe break numbers are only allowed to be included in the same defect-connected region when they are adjacent in the surface adjacency diagram. Therefore, the defect starting point set, the defect-connected region, and the defect boundary together constrain the range of subsequent quadratic surface reconstruction.

[0056] Example 7: In this embodiment, quadratic surface reconstruction is performed independently for each defective connected region. A uniform interval strategy is used for center point selection; center points are selected at fixed intervals along the geodesic direction within the defective connected region. The selection interval is set to the average spacing of the local fitting point range. The center points are selected at multiples of the normal interval to ensure overlap in the fitting range of adjacent center points and guarantee the continuity of the reconstructed point cloud. For defect edge regions with significant curvature changes, the center points can be selected more densely, with the density interval being one-times the normal interval. Each center point corresponds to an independent quadratic surface fitting and reconstruction point calculation.

[0057] The local fitting point range starts from the center point and includes sampling points within the same connected region of the defect and boundary points within the defect boundary, arranged in order of geodesic distance from near to far. The stopping condition simultaneously satisfies two criteria: first, the rank of the quadratic surface design matrix is ​​equal to... First, the number of columns must be at least six; second, the number of points included must be no less than [number missing]. The full-rank criterion is that after the singular value decomposition of the design matrix, the ratio of the sixth largest singular value to the first largest singular value is greater than 1. If the number of included points reaches the total number of points in the defect area. If the result is still not rank-complete, then stop including the data and use a regularization method to solve for the coefficients. The maximum geodesic distance within the local fitting point range does not exceed a preset geodesic distance threshold. To prevent the fitting range from being too large and losing local characteristics, the local fitting point range should not cross other defect connected regions or stripe break segments with different stripe break numbers.

[0058] A reference tangent plane is established with the nearest reference point to the center point as the origin, the tangents of the first and second surfaces corresponding to the center point as the coordinate axes, and the normal of the reference surface corresponding to the center point as the normal. Support points within the local fitting point range are projected onto the same reference tangent plane. For the center point... and support points The coordinates of the tangent plane and the normal height are expressed as follows: (Formula 28); (Formula 29); (Formula 30); in, To support point coordinates, The nearest reference point to the center point, The tangent of the first surface corresponding to the center point. The tangent of the second surface corresponding to the center point. The reference surface normal corresponding to the center point.

[0059] The expression for a quadric surface is: (Formula 31); in, to Let be the fitting coefficients for the quadratic surface to be determined. The fitting coefficients are solved using the point cloud measurement reliability as the fitting weight, and the objective function is expressed as: (Formula 32); in, Center point The corresponding local fitting point range, To enhance the reliability of point cloud measurements, the quadratic surface coefficients are solved using the weighted normal equation method, transforming the objective function into a normal equation form: (Formula 33); in, To design the matrix, This is a diagonal weight matrix with point cloud measurement confidence as its diagonal element. Let be the coefficient vector to be determined. Let be the normal height vector. The quadratic surface fitting coefficients are obtained by solving the normal equations through Cholesky decomposition. When the design matrix is ​​ill-conditioned, a Tikhonov regularization term is added, with the regularization coefficient taking the minimum singular value of the design matrix. This ensures the stability of the solution.

[0060] Each center point corresponds to a defect reconstruction point, which is located on the normal of the reference tangent plane corresponding to the center point. The position is determined by the normal height of the fitted quadratic surface at the center point. Confirmed, the calculation formula is: (Formula 34); in, For the defect reconstruction point, Center point The corresponding quadratic surface fitting coefficients at that location. The nearest reference point corresponding to the center point. The reference surface normal is used. The reconstructed point cloud of the defect region is composed of the defect reconstruction points corresponding to all center points. The number of reconstruction points is the same as the number of center points, and there is no one-to-one correspondence with the original sampling points. The density of the reconstructed point cloud is determined by the center point selection interval.

[0061] Example 8: In this embodiment, the surface development coordinates of the defect-free reference pipe surface are generated based on the surface adjacency diagram. When the straight pipe section is developed along the axial generatrix, one generatrix of the straight pipe section is selected as the origin of the vertical axis of the development coordinates. The direction along the centerline of the pipe is the v-axis of the development coordinates, and the direction along the circumference of the pipe is the u-axis of the development coordinates. After development, the u-axis coordinate corresponds to the circumferential arc length, and the v-axis coordinate corresponds to the axial length. When the elbow section is developed along the centerline of the ring, the arc length of the elbow section's centerline is used as the v-axis of the development coordinates, and the circumferential arc length of the elbow section is used as the u-axis of the development coordinates. During development, the proportion of the centerline arc length remains unchanged, and the circumferential coordinates are mapped proportionally according to the central angle of the section.

[0062] The segments with different diameters and the three-way crossing are expanded using least-squares conformal expansion. A conformal mapping objective function is constructed with tangential continuous boundaries as constraints. The expanded coordinates are solved through nonlinear optimization to ensure that the local angular deformation after expansion is less than [a certain value]. At tangential continuous splicing points, the adjacency relationship is maintained through translational alignment. The unfolded coordinates of cross-surface defect regions are continuously connected through the correspondence of boundary points. Multiple reference surface patches maintain the adjacency relationship of their surface unfolded coordinates at tangential continuous splicing points, ensuring local continuity of defect reconstruction points within the same defect connectivity region in the surface unfolded coordinates. The defect reconstruction points within each defect connectivity region are then sorted according to their surface unfolded coordinates.

[0063] Constrained triangulation employs the constrained Delaunay triangulation algorithm, performed within the surface unfolding plane, with defect boundaries included as hard constraint edges in the triangulation constraints. Triangulation vertices are the sorted defect reconstruction points within the same defect connected region, generating triangular meshes only within the vertex set. During constrained triangulation, only triangles belonging to the same defect connected region, not crossing defect boundaries, not crossing stripe fractures with different numbers and no direct adjacency, and not crossing non-tangential continuous boundaries of different surface patches are retained. After triangulation, the mesh topology is adjusted through minimum interior angle optimization to ensure that the minimum interior angle of each triangle is not less than [value missing]. After the triangular mesh is generated, the two-dimensional unfolded coordinates of each vertex are mapped back to three-dimensional surface coordinates to obtain a three-dimensional defect triangular mesh. During the mapping, the vertex positions are directly transformed through the correspondence between the unfolded coordinates and the three-dimensional coordinates. Constrained triangulation generates defect triangular meshes, and the defect triangular meshes form a three-dimensional model of the defect in the complex curved pipe fitting.

[0064] For each defect reconstruction point, calculate the defect depth value relative to the nearest reference point along the normal to the reference surface. The defect depth value is expressed as: (Formula 35); in, This represents the defect depth value. For the defect reconstruction point, The most recent reference point, The reference surface normal.

[0065] The defect depth value is rasterized according to the coordinates of the surface unfolding, forming... Figure 1 The single-channel defect depth map is used. The grid resolution is set according to the unfolded size and accuracy requirements of the defect region, and the physical size corresponding to a single grid is set to the average spacing of the reconstructed point cloud. The rasterization process employs a bilinear interpolation algorithm, performing interpolation calculations only within the same connected region of the defect, with the interpolation range not crossing the defect boundary. Pixels outside the defect region are uniformly marked as invalid values, and invalid values ​​are identified using specific numerical values. The origin of the depth map's coordinates is located at the upper left corner of the unfolded plane, with the u-axis running horizontally to the right and the v-axis running vertically downward, maintaining a consistent axial relationship with the unfolded coordinates.

[0066] This system maintains a one-to-one correspondence between the vertex indices of the 3D model of a complex curved pipe fitting defect and the pixel indices of the single-channel defect depth map, enabling mutual positioning between the two. The one-to-one correspondence between vertex and pixel indices means that each vertex of the defect triangular mesh corresponds to a unique pixel position in the depth map, and each valid pixel in the depth map corresponds to a unique vertex of the defect mesh. This is achieved by aligning the defect reconstruction points to the grid vertices of the depth map according to their unfolded coordinates, ensuring that each reconstruction point falls exactly at the vertex position of a grid cell, with a one-to-one correspondence between the reconstruction point index and the grid pixel index. The alignment process is achieved by adjusting the depth map resolution and the origin position, ensuring that all reconstruction points within the defect area match the depth map pixels one-to-one, avoiding many-to-one or one-to-many relationships.

[0067] Example 9: In this embodiment, the system includes a processor, a memory, an image and point cloud unification module, a stripe measurement data recording module, a reference pipe surface reconstruction module, a defect residual judgment module, a defect region generation module, a defect surface reconstruction module, and a defect model output module. The processor calls program instructions from the memory and follows... Figure 1 The process shown dispatches each module.

[0068] The interface parameters, processing flow, and method steps of each module in the system are completely consistent. The unified image and point cloud module is used to obtain a unified point cloud and has a built-in pose matrix calculation unit and point cloud index mapping unit, which can complete the coordinate transformation, merging and downsampling, and source record storage of multi-view point clouds. The unified image and point cloud module receives stripe images and initial 3D point clouds, reads the pose matrix, transforms the local point clouds to a unified 3D coordinate system, and establishes point cloud source records.

[0069] The fringe measurement data recording module is used to obtain point clouds with measurement data. It has built-in edge-preserving filtering unit, fringe center extraction unit, and quality parameter calculation unit. It also has built-in default values ​​and adaptive adjustment logic for all preset parameters, and can directly output point clouds with measurement data. The fringe measurement data recording module receives unified point clouds and point cloud source records, and completes fringe center extraction, sampling point coordinate update, local saturation ratio recording, fringe gradient intensity recording, fringe break number recording, camera line-of-sight direction recording, and projection illumination direction recording.

[0070] The reference pipe fitting surface reconstruction module is used to obtain the surface of the defect-free reference pipe fitting and the reference surface data. It has built-in centerline extraction unit, surface patch division unit, and surface fitting and stitching unit, and built-in fitting algorithms and tangential continuous stitching algorithms for various parametric surfaces. The reference pipe fitting surface reconstruction module receives point clouds with measurement data and completes local covariance normal estimation, curvature estimation, pipe fitting centerline extraction, reference surface patch division, least squares fitting, and tangential continuous stitching.

[0071] The defect residual determination module is used to calculate the point cloud measurement reliability and the defect residual determination value of the pipe fitting surface. It has built-in geodesic distance calculation unit and statistical quantity calculation unit, and built-in adaptive calculation logic for all stable terms. The defect residual determination module receives point cloud with measurement data, defect-free reference pipe fitting surface, and reference surface data, establishes a surface adjacency graph, and calculates the signed normal residual, specular reflection sensitive direction, point cloud measurement reliability, and defect residual determination value of the pipe fitting surface.

[0072] The defect region generation module generates a set of defect starting points, defect connected regions, and defect boundaries. It includes a built-in threshold calculation unit and a region growth unit, as well as built-in logic for extracting and verifying defect boundaries. The defect region generation module receives the defect residual judgment value of the pipe fitting surface and the surface adjacency graph, determines the set of defect starting points through a local adaptive threshold, and performs region growth on the surface adjacency graph.

[0073] The defect surface reconstruction module is used to obtain the reconstructed point cloud of the defect region. It has built-in fitting range construction units and weighted least squares solution units, as well as built-in regularization solution logic. The defect surface reconstruction module receives the defect connected region, defect boundary, point cloud measurement confidence level, and reference surface data, establishes the local fitting point range, and performs quadratic surface reconstruction.

[0074] The defect model output module generates 3D models of defects in complex curved pipe fittings and single-channel defect depth maps. It includes built-in surface unfolding units, triangulation units, and depth map generation units, as well as built-in logic for constructing and storing index correspondences. The module receives the reconstructed point cloud of the defect region, the surface of the defect-free reference pipe fitting, and the surface adjacency graph. It generates surface unfolding coordinates, performs restricted triangulation, calculates defect depth values, rasterizes the surface, and saves the one-to-one correspondences.

[0075] Example 10: In this embodiment, program instructions are stored on a computer-readable storage medium. When the program instructions are executed by the processor, the processor sequentially calls the image and point cloud unification module, the stripe measurement data recording module, the reference pipe surface reconstruction module, the defect residual judgment module, the defect region generation module, the defect surface reconstruction module, and the defect model output module to complete the process. Figure 1 The processing steps S101 to S108 are shown, and the output includes a 3D model of the complex curved pipe fitting defect and a single-channel defect depth map. The program instructions stored in the computer-readable storage medium contain the executable code of all the above units. When called by the processor, all steps can be executed sequentially according to the process flow, and the results can be output.

[0076] For nominal diameter Wall thickness The typical implementation parameters for stainless steel elbow pipe fittings are as follows: camera resolution is... pixels, projector resolution is Pixels, the number of pixels per period of phase-shifted fringes is Gray code bit depth is Position, ring acquisition view step size is The side length of the point cloud downsampling voxels is The pixel neighborhood window is The standard deviation of the bilateral filter in the spatial domain is The standard deviation of pixels and grayscale is Gray level, the number of nearest neighbors estimated by normal direction is The slice step size is The preset geodetic distance threshold is The proportionality coefficients of all stable terms are This set of parameters enables high-precision reconstruction of surface defects in pipe fittings, with a depth detection error of less than [value missing]. .

[0077] Defect reconstruction accuracy verification uses standard stepped defect specimens. The specimen surface is machined with standard stepped defects of known depth. This embodiment uses this method to collect and reconstruct the specimens, comparing the reconstructed defect depth values ​​with the standard values ​​to calculate the absolute and relative errors. Defect detection rate verification uses standard pipe fittings with artificial defects of different sizes, statistically analyzing the ratio of the number of defects detected in this embodiment to the actual number of defects. Reconstruction accuracy is quantified using the intersection-union ratio (IUU) of the 3D defect model and the standard defect model. An IUU greater than [value missing] indicates a higher accuracy. The reconstruction was determined to be accurate.

[0078] For highly reflective areas, the degree of high reflectivity is determined by the local saturation ratio. When the local saturation ratio is greater than... When this occurs, the measurement reliability of points in that area is reduced, and multi-view complementary data is used for filling. For point cloud missing areas caused by occlusion, if the area of ​​the missing area is smaller than the minimum defect determination area, it is directly marked as an invalid area; if the missing area is large, it is supplemented by multi-view data fusion. For stripe fracture areas, points within the fracture segments do not participate in the defect initiation point determination. During region growth, points with different fracture numbers can only be classified into the same region if they are directly adjacent in the surface adjacency map and have the same residual sign, thus avoiding false defect connectivity caused by fractures.

[0079] The embodiments of this example have been described above. However, this example is not limited to the specific implementation methods described above. The specific implementation methods described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms based on the guidance of this example, and all of them are within the protection scope of this example.

Claims

1. A method for reconstructing defects in complex curved pipe fittings, applied to the image data processing of three-dimensional reconstruction of defects in complex curved pipe fittings, wherein the complex curved pipe fittings include elbows, reducing sections, tee junctions, and fittings with transition fillets, characterized in that, include: Annular multi-view structured light images of the complex curved surface pipe are acquired to generate stripe images and initial three-dimensional point clouds. The initial three-dimensional point clouds are then converted to a unified three-dimensional coordinate system under the fixture reference to obtain a unified point cloud. Extract the stripe center of the stripe image, and record the stripe image quality data and acquisition optical path data of the sampling points in the unified point cloud to obtain a point cloud with measurement data; Based on the point cloud with measurement data, the surface of the defect-free reference pipe fitting is reconstructed, and the reference surface data is determined. Based on the stripe image quality data, the acquisition optical path data, the reference surface data, and the signed normal residual of the sampling point relative to the defect-free reference pipe surface, the point cloud measurement reliability and the defect residual judgment value of the pipe surface are calculated. Based on the residual judgment value of the defect on the pipe fitting surface, generate a set of defect starting points, a defect connected region, and a defect boundary; Within the defective connected region, a quadratic surface is reconstructed according to the point cloud measurement reliability and the defect boundary to obtain the defective region reconstructed point cloud; Based on the point cloud reconstruction of the defect area, a 3D model of the defect in the complex curved pipe fitting and a single-channel defect depth map are generated.

2. The method for reconstructing defects in complex curved surface pipe fittings according to claim 1, characterized in that, Obtaining the unified point cloud includes: A structured light acquisition system consisting of a camera and a projector with calibrated intrinsic and extrinsic parameters was used to perform ring-shaped multi-view acquisition. The camera intrinsic parameters, projector intrinsic parameters, camera and projector extrinsic parameters, fixture coordinate system, and rigid body pose of the pipe remain unchanged during a single scan. Record the pose matrix of each sampling frame from the coordinate system of the acquisition device to the unified three-dimensional coordinate system; Local point clouds are obtained based on the structured light triangulation results, and the pose matrix is ​​used to convert each local point cloud into the unified point cloud. A point cloud source record is established for the 3D points in the unified point cloud. The point cloud source record includes point coordinates, source image, pixel neighborhood, camera line of sight, and projection illumination direction.

3. The method for reconstructing defects in complex curved surface pipe fittings according to claim 2, characterized in that, Obtaining the point cloud with measurement data includes: The source image and pixel neighborhood corresponding to the sampling point are determined based on the point cloud source record; Perform edge-preserving filtering on each frame of the striped image; The position of the bright ridge line is determined on the fringe normal section, and the sub-pixel center is corrected according to the gray centroid of the ridge line neighborhood. Update the sampling point coordinates according to the sub-pixel center and the calibration parameters of the structured light acquisition system; Record the local saturation ratio, fringe gradient intensity, fringe break number, camera line of sight, and projection illumination direction for each sampling point; In this context, overexposed pixels in the stripe image are included in the local saturation ratio, and the stripe break segments in the stripe image are numbered to form the stripe break number.

4. The method for reconstructing defects in complex curved surface pipe fittings according to claim 3, characterized in that, Determining the reference surface data includes: Local covariance normal and curvature estimation are performed on the point cloud with measurement data; The centerline of the pipe fitting is extracted based on the point cloud with measurement data, and the reference surface patches are divided according to the aggregation trend of the normal on the unit sphere and the arrangement order along the centerline of the pipe fitting. The reference surface pieces are fitted with least squares respectively, and tangential continuous splicing is performed at the transition between adjacent reference surface pieces to obtain the defect-free reference pipe surface. For each sampling point, determine the nearest reference point, reference surface normal, first surface tangent, second surface tangent, first reference curvature and second reference curvature, and use them as the reference surface data.

5. The method for reconstructing defects in complex curved surface pipe fittings according to claim 4, characterized in that, Calculate the point cloud measurement reliability and the residual judgment value of the pipe fitting surface defect, including: On the surface of the defect-free reference pipe, a range of adjacent points is established for each sampling point according to the geodesic distance of the surface, and a surface adjacency diagram is established based on the range of adjacent points. Calculate the signed normal residual based on the sampling point, the nearest reference point, and the reference surface normal; Determine the sensitive direction of specular reflection based on the direction of projected illumination and the direction of the camera's line of sight; The reliability of the point cloud measurement is calculated based on the local saturation ratio, fringe gradient intensity, sampling interval along the tangent of the first surface and the tangent of the second surface, and the angle between the reference surface normal and the specular reflection sensitive direction. The defect residual judgment value of the pipe fitting surface is calculated based on the point cloud measurement reliability, the median deviation of the signed normal residual within the range of adjacent points on the surface, the first reference curvature, and the second reference curvature.

6. The method for reconstructing defects in complex curved surface pipe fittings according to claim 5, characterized in that, Generate the set of defect initiation points, defect connected regions, and defect boundaries, including: The sampling points where the residual judgment value of the defect on the pipe fitting surface is greater than the sum of the median value and the median deviation of the judgment value within the range of the adjacent points of the surface are determined as the set of defect starting points. Starting from the set of defect initiation points, region growth is performed on the sampling points along the signed normal residual with the same sign and the point cloud measurement confidence level on the adjacency graph of the surface to form the defect connected region. The set of sampling points on the surface of the defect-free reference pipe that are geodetically adjacent to the defect-connected region but do not belong to the defect-connected region is determined as the defect boundary. Sampling points with different stripe fracture numbers are allowed to be grouped into the same defect connectivity region only when they are adjacent in the surface adjacency diagram.

7. The method for reconstructing defects in complex curved surface pipe fittings according to claim 6, characterized in that, Obtaining the reconstructed point cloud of the defect region includes: Within each defect connectivity region, a sampling point is taken as the center point, and a local fitting point range is established for the center point. The local fitting point range is included by sampling points within the same defect connectivity region and boundary points in the defect boundary according to the surface geodesic distance, until the quadratic surface design matrix reaches six columns of full rank. The range of the local fitting point does not cross other defect connected regions or stripe break segments with different stripe break numbers. Project the support points within the local fitting point range onto a reference tangent plane with the nearest reference point of the center point as the origin and the tangent of the first surface and the tangent of the second surface as the coordinate axes; The point cloud measurement reliability is used as the fitting weight to solve the quadratic surface fitting coefficient, and the defect reconstruction points are generated from the quadratic surface fitting coefficient.

8. The method for reconstructing defects in complex curved surface pipe fittings according to claim 7, characterized in that, Generate a 3D model of defects in complex curved pipe fittings and a single-channel defect depth map, including: The surface development coordinates of the defect-free reference pipe surface are generated based on the surface adjacency diagram. According to the surface unfolding coordinates, the defect reconstruction points in each defect connected region are sorted. Within the same defect connectivity region, the sorted defect reconstruction points are subjected to restricted triangulation to generate a defect triangular mesh, and the defect triangular mesh is used to form a three-dimensional model of the defect of the complex curved pipe fitting. For each defect reconstruction point, calculate the defect depth value relative to the nearest reference point along the normal of the reference surface; The defect depth value is rasterized according to the coordinate unfolding of the surface to form the single-channel defect depth map; The system maintains a one-to-one correspondence between the vertex indices of the 3D model of the complex curved pipe fitting defect and the pixel indices of the single-channel defect depth map.

9. A defect reconstruction system for complex curved surface pipe fittings, characterized in that, It includes a processor, memory, image and point cloud unified module, stripe measurement data recording module, reference pipe surface reconstruction module, defect residual judgment module, defect region generation module, defect surface reconstruction module, and defect model output module; The image and point cloud unification module is used to obtain a unified point cloud; The stripe measurement data recording module is used to obtain a point cloud with measurement data; The reference pipe surface reconstruction module is used to obtain the defect-free reference pipe surface and reference surface data; The defect residual judgment module is used to calculate the point cloud measurement reliability and the defect residual judgment value of the pipe fitting surface; The defect region generation module is used to generate a set of defect starting points, a defect connected region, and a defect boundary. The defect surface reconstruction module is used to obtain the reconstructed point cloud of the defect region; The defect model output module is used to generate a three-dimensional model of defects in complex curved pipe fittings and a single-channel defect depth map; The processor executes instructions in the memory to implement the method for reconstructing defects in complex curved pipe fittings as described in any one of claims 1 to 8.

10. A computer-readable storage medium having program instructions stored thereon, characterized in that, When the program instructions are executed by the processor, they implement the method for reconstructing defects in complex curved pipe fittings as described in any one of claims 1 to 8.

Images