A Smart Detection Method for Surface Defects in Pipe Fittings Based on Image Recognition

Abstract

This invention discloses an intelligent detection method for pipe surface defects based on image recognition. The method first acquires 3D point cloud data of the pipe to be inspected and calculates the Gaussian curvature and normal vector of each point. Based on the differences in Gaussian curvature, the surface of the pipe is divided into several geometrically homogeneous regions. Multi-view original images are acquired within each region, and a local isometric mapping function is constructed using the normal vector and Gaussian curvature to correct and unfold the image into a 2D planar image. Illumination normalization and learnable scaling and translation operations are performed on the unfolded image. The processed image is input into a defect detection neural network containing a geometric attention module and a context-aware decoder, outputting a pixel-level defect probability map. Finally, the defect result is projected back into 3D space using an inverse mapping function. This method effectively overcomes the imaging distortion and recognition difficulties caused by the complex geometry of pipes and is suitable for high-precision surface defect detection of various irregularly shaped pipes.

Description

Technical Field

[0001] This invention belongs to the interdisciplinary field of industrial visual inspection and artificial intelligence technology, and relates to an intelligent detection method for surface defects of pipe fittings based on image recognition. Background Technology

[0002] In the fields of industrial manufacturing and operation and maintenance, the detection of surface defects in pipe fittings is of great significance for ensuring the safe operation of systems. Currently, commonly used detection methods include manual visual inspection, contact sensor detection, and machine vision methods based on image recognition. Manual visual inspection relies on operator experience, has low efficiency, and is prone to missed detections and false positives. Contact sensor detection methods, such as ultrasonic or eddy current testing, can acquire internal or surface information, but are limited by the physical contact requirements between the probe and the pipe fitting surface, making them unsuitable for pipe fittings with complex geometries such as bends, tees, and reducers, especially in areas with drastic curvature changes or limited space. Traditional machine vision methods based on two-dimensional images typically assume the object being measured is a plane or a regular cylinder. When processing high-curvature pipe fittings, distorted defect morphology in the image is caused by viewpoint obstruction, surface reflection, and projection distortion, making feature extraction difficult.

[0003] In recent years, although some studies have attempted to introduce deep learning models for surface defect identification, most methods have not considered the impact of pipe geometry on the imaging process and lack effective correction for brightness unevenness and texture stretching caused by curved surfaces. This results in low detection rates and high false alarm rates for minute defects in high curvature areas. While some multi-view fusion schemes can improve coverage, they do not incorporate geometric priors for region segmentation and feature guidance, leading to large image registration errors and poor consistency in detection results.

[0004] Therefore, existing technologies are insufficient to meet the automated defect identification needs of various irregular-shaped pipe fittings while ensuring detection accuracy. There is an urgent need for an intelligent detection method that can integrate three-dimensional geometric information and image recognition capabilities. Summary of the Invention

[0005] To address the problems existing in the background technology, this invention proposes an intelligent detection method for surface defects of pipe fittings based on image recognition.

[0006] The first aspect of this application provides an intelligent detection method for surface defects in pipe fittings based on image recognition, including:

[0007] Acquire the three-dimensional point cloud data of the pipe fitting to be inspected, and calculate the Gaussian curvature and normal vector of each point based on the three-dimensional point cloud data;

[0008] Based on the difference in Gaussian curvature, the surface of the pipe fitting is divided into several geometrically homogeneous regions, such that the absolute value of the difference in Gaussian curvature between any two points in each geometrically homogeneous region is less than a preset curvature tolerance threshold.

[0009] Original images from multiple perspectives are acquired within each geometrically homogeneous region, and a local isometric mapping function is constructed using the normal vector and Gaussian curvature to correct and unfold the original images into two-dimensional planar images.

[0010] The two-dimensional planar image is subjected to illumination normalization processing to obtain a normalized image;

[0011] The normalized image is input into a defect detection neural network, which includes a multi-scale feature encoder, a geometric attention module, and a context-aware decoder, wherein the geometric attention module performs channel weighting on the convolutional feature map guided by the Gaussian curvature.

[0012] Output a pixel-level defect probability map and project the defect results back into three-dimensional space using an inverse mapping function.

[0013] Optionally, the Gaussian curvature is calculated using principal curvature, which is the curvature value of each point in the three-dimensional point cloud along two orthogonal principal directions.

[0014] Optionally, the local isometric mapping function is used to map the curved surface region to a two-dimensional plane without stretching, so as to eliminate image distortion caused by the geometry of the pipe fitting.

[0015] Optionally, the illumination normalization process uses the following formula:

[0016] ;

[0017] in, This represents the brightness value of the normalized image at pixel location (u,v); This represents the original brightness value at the same pixel position in a two-dimensional planar image after correction and unfolding using a local isometric mapping function; The angle of incidence of light at pixel position (u,v) is determined by the angle between the direction of the light source and the normal vector of that point; cos represents the cosine function. This represents the numerical stability constant, with a value of 0.001.

[0018] Optionally, when the geometric attention module performs channel weighting on the convolutional feature map, it uses Gaussian curvature and global average pooling to generate attention weights through a learnable weight matrix and a Sigmoid function, and then multiplies them channel by channel with the original convolutional feature map.

[0019] Optionally, the context-aware decoder includes dilated convolutional layers and nonlocal operation layers to capture long-range spatial dependencies of defective regions.

[0020] Optionally, after obtaining the normalized image, the method further includes: introducing two trainable parameters, used to perform a linear transformation on each pixel value of the normalized image; wherein, one parameter controls the scaling factor and the other parameter controls the translation bias, and the operation takes the form of:

[0021] ;

[0022] in, This represents the brightness value at pixel location (u,v) of the image after being adjusted by learnable scaling and translation operations. Indicates the learnable scaling parameter; This represents the brightness value at pixel location (u,v) in the image obtained after illumination normalization. Represents the learnable translation parameters; Corresponding column index of the image The row index of the corresponding image.

[0023] Compared with the prior art, the present invention has the following beneficial effects:

[0024] The intelligent detection method for pipe surface defects based on image recognition described in this invention has demonstrated significant technical effects in practical applications. This method acquires 3D point cloud data of the pipe and calculates the Gaussian curvature and normal vector of each point, achieving accurate modeling of the pipe's geometric characteristics. By dividing geometrically homogeneous regions based on Gaussian curvature differences, subsequent image processing is performed within a local area with consistent geometric characteristics, effectively avoiding mapping distortion caused by processing across curvature regions. A local isometric mapping function is constructed using the normal vector and Gaussian curvature, unfolding the original image into a two-dimensional planar image without stretching, eliminating perspective distortion and texture compression caused by the complex curved surface of the pipe. Furthermore, illumination normalization processing is introduced, combined with learnable scaling and translation operations, to compensate for brightness unevenness caused by differences in incident angle and material reflectivity, thus highlighting the true defect features. The geometric attention module integrated into the defect detection neural network, guided by Gaussian curvature, dynamically enhances the feature response of high-curvature regions, improving sensitivity to minor defects in critical parts such as bends. The context-aware decoder, through joint modeling with dilated convolutional layers and nonlocal operation layers, alleviates the defect breakage or false connection problems caused by the limited receptive field in traditional methods, enhancing the ability to understand the overall structure of slender, meandering, or sparsely distributed defects. Finally, the inverse mapping function accurately traces the two-dimensional detection results back to three-dimensional space, achieving precise location and visualization of defects on the original pipe surface. Overall, this invention is universally applicable to pipe fittings with various complex geometries, such as straight pipes, bends, tees, and reducers, effectively improving the surface defect detection rate, reducing the false alarm rate, and enhancing automation, demonstrating good engineering practicality and industrial deployment value. Attached Figure Description

[0025] Figure 1 This is a flowchart of an intelligent detection method for surface defects of pipe fittings based on image recognition in one embodiment of the present invention. Detailed Implementation

[0026] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0027] In one embodiment, such as Figure 1 As shown, an intelligent detection method for surface defects in pipe fittings based on image recognition is provided, which is then applied to... Figure 1 Taking China as an example, the following specific steps will be used:

[0028] S10: Obtain the three-dimensional point cloud data of the pipe to be inspected, and calculate the Gaussian curvature and normal vector of each point based on the three-dimensional point cloud data.

[0029] Specifically, the process involves acquiring 3D point cloud data of the pipe fitting to be inspected and calculating the Gaussian curvature and normal vector of each point based on the 3D point cloud data. The process includes the following steps: First, the surface of the pipe fitting is scanned using a structured light scanning device to obtain 3D point cloud data representing the geometric shape of the pipe fitting surface. The 3D point cloud data consists of multiple spatial points, each with 3D coordinate values. Then, the 3D point cloud data is preprocessed, including denoising, filtering, and resampling, to improve the point cloud quality. Next, for each spatial point, a local surface model is constructed using several nearest neighbors. The local surface model is fitted with a quadratic surface function using the least squares method to estimate the differential geometric properties at that point. Based on the fitted surface, two principal curvature values ​​are calculated for that point, corresponding to the degree of curvature of the surface in two mutually orthogonal principal directions. Further, the two principal curvature values ​​are multiplied to obtain the Gaussian curvature of that point. Simultaneously, the normal vector of that point is determined based on the gradient direction of the local surface model; the normal vector is perpendicular to the tangent plane of the fitted surface at that point. The Gaussian curvature and normal vector mentioned above are used for subsequent geometric homogeneous region division and image correction processing to ensure that the defect detection process fully considers the geometric characteristics of the pipe surface.

[0030] S20: Based on the Gaussian curvature difference, the surface of the pipe is divided into several geometrically homogeneous regions, such that the absolute value of the difference in Gaussian curvature between any two points in each geometrically homogeneous region is less than the preset curvature tolerance threshold.

[0031] Specifically, first, all spatial points in the 3D point cloud data are traversed to obtain the Gaussian curvature value corresponding to each spatial point. Then, a new geometrically homogeneous region is initialized using any undivided spatial point as the starting point. Next, a region growing algorithm is used to check the neighboring spatial points sequentially from the starting point. For each neighboring spatial point, the absolute value of the difference between its Gaussian curvature and the Gaussian curvature of the spatial points already included in the current geometrically homogeneous region is calculated. If the absolute value of the difference is less than a preset curvature tolerance threshold, the neighboring spatial point is included in the current geometrically homogeneous region, and the region continues to expand using it as a seed point. If the absolute value of the difference is not less than the preset curvature tolerance threshold, the expansion in that direction stops. The above process is repeated until all spatial points are assigned to a geometrically homogeneous region. The resulting geometrically homogeneous regions have similar curvature characteristics in their geometric shape, making them suitable for subsequent unified image correction and unfolding processing. The preset curvature tolerance threshold is a pre-set non-negative real number used to control the fineness of the region division.

[0032] For example, a 90-degree stainless steel elbow to be inspected is processed. The elbow has an outer diameter of 58 mm and a bending radius of 116 mm. After acquiring its three-dimensional point cloud data through structured light scanning, the Gaussian curvature of each point is calculated, and a preset curvature tolerance threshold of 0.05 mm² is set. After region growing processing, the elbow surface is divided into three geometrically homogeneous regions, corresponding to the straight pipe section, the transition bend section, and the high-curvature elbow center section, respectively. The absolute value of the difference in Gaussian curvature between any two points in each region is less than 0.05 mm².

[0033] Based on the difference in Gaussian curvature, the surface of the pipe fitting is divided into several geometrically homogeneous regions. This ensures that the absolute value of the difference in Gaussian curvature between any two points within each region is less than a preset curvature tolerance threshold. This guarantees that subsequent image correction and unfolding operations are performed within regions with similar geometric characteristics, avoiding mapping distortion caused by abrupt changes in curvature. This improves the consistency of defect feature representation in two-dimensional images, providing a geometric basis for neural networks to accurately identify minute defects.

[0034] S30: Acquire original images from multiple perspectives within each geometrically homogeneous region, and construct a local isometric mapping function using the normal vector and Gaussian curvature to correct and unfold the original images into a two-dimensional planar image.

[0035] Specifically, multiple industrial cameras are first deployed around each geometrically homogeneous region. Each camera simultaneously captures images of the pipe surface from different angles, obtaining multi-view original images covering the entire visible surface of the region. Then, for each pixel in the original image, a local coordinate system is established based on the normal vector and Gaussian curvature of its corresponding 3D spatial point. In this local coordinate system, the point is the origin, the normal vector direction is the local z-axis, and the other two orthogonal directions are determined by the principal curvature directions. Based on this local coordinate system, a local isometric mapping function is constructed using the isometric mapping principle from differential geometry. This local isometric mapping function approximately maintains the geodesic distance between adjacent points on the surface as Euclidean distance, thus achieving unstretched unfolding of the surface to a plane. Through this function, the pixel positions distorted by surface deformation in the original image are remapped onto a 2D plane, generating a corrected unfolded image. This unfolded image retains the original texture information while eliminating perspective distortion and scale compression caused by the pipe's geometry.

[0036] For example, a geometrically homogeneous region of a curved section of the aforementioned 90-degree stainless steel elbow is processed. Four industrial cameras are positioned in this region, located above, below, to the left, and to the right of the pipe fitting. After acquiring four original images, a local isometric mapping function is constructed by combining the normal vector and Gaussian curvature of the point cloud in this region. After mapping processing, the four original images are each unfolded into four two-dimensional planar images, which are then stitched together to form a distortion-free texture map that completely covers the surface of the curved section. The scratch features that were originally blurred due to surface reflection, surface distortion, and viewpoint occlusion become clearly discernible.

[0037] Original images from multiple perspectives are acquired within each geometrically homogeneous region. A local isometric mapping function is constructed using the normal vector and Gaussian curvature to correct and unfold the original images into a two-dimensional planar image. This effectively overcomes the texture stretching or compression problems caused by traditional planar projection in high-curvature regions. The unfolded image has a stable geometric structure, which facilitates the accurate extraction of defect morphological features by the subsequent defect detection neural network, significantly improving the robustness of identifying micro-cracks and pit-like defects.

[0038] S40: Perform illumination normalization processing on the two-dimensional planar image to obtain a normalized image.

[0039] Specifically, first, a two-dimensional planar image is acquired after correction and unfolding using a local isometric mapping function. Although geometric distortion has been eliminated, the different directions of the normal vectors at various points on the pipe surface cause inconsistent brightness in areas of the same material due to differences in the incident angle of light. Second, based on the normal vector of the corresponding point in the three-dimensional point cloud and the known light source direction, the incident angle θ(u,v) of light at each pixel position is calculated. The incident angle is the angle between the light source direction vector and the normal vector at that point. Subsequently, Lambert's cosine law is used to compensate for the original pixel values ​​to eliminate the brightness attenuation caused by the incident angle.

[0040] The illumination normalization process uses the following formula:

[0041] ;

[0042] in, This represents the brightness value of the normalized image at pixel location (u,v); This represents the original brightness value at the same pixel position in a two-dimensional planar image after correction and unfolding using a local isometric mapping function; The angle of incidence of light at pixel position (u,v) is determined by the angle between the direction of the light source and the normal vector of that point; cos represents the cosine function. This represents the numerical stability constant, with a value of 0.001.

[0043] In one embodiment, a multi-view image of the surface of a 90-degree stainless steel bend is performed. After the surface is unfolded, one side of the image is too bright due to its proximity to the light source, while the other side is severely darkened due to its incident angle being close to 90 degrees. After normalization, the gray values ​​of the originally smooth area tend to be consistent, while the actual corrosion spots, due to their different reflective properties compared to the substrate, still maintain a lower brightness, thus forming a clear contrast with the background.

[0044] The two-dimensional planar image is subjected to illumination normalization processing to obtain a normalized image. This normalized image is then further processed to generate an adjusted image, effectively suppressing non-defect-related brightness variations caused by the geometric orientation of the pipe fitting, thus preventing the subsequent neural network from misclassifying normal reflective areas as surface defects. Simultaneously, it enhances the visual distinction between real defects and normal areas, improving the accuracy and robustness of defect detection. This method relies solely on the light source direction and normal vector information, requiring no additional calibration equipment and possessing good engineering applicability.

[0045] S50: Input the normalized image into the defect detection neural network, the defect detection neural network comprising a multi-scale feature encoder, a geometric attention module and a context-aware decoder, wherein the geometric attention module performs channel weighting on the convolutional feature map guided by the Gaussian curvature.

[0046] Specifically, first, the generated and adjusted image is fed into a multi-scale feature encoder. This encoder employs a deep convolutional structure to extract multi-level feature maps, from low-level edge texture to high-level semantic information, layer by layer. The output of each layer maintains a balance between spatial resolution and semantic depth, providing a foundation for subsequent defect localization.

[0047] Subsequently, the convolutional feature maps from each layer enter the geometric attention module. This module synchronously receives Gaussian curvature maps corresponding to the spatial locations of the normalized image. The Gaussian curvature maps are compressed into scalars through global average pooling, then transformed by a learnable weight matrix and generated channel attention weights between 0 and 1 using the Sigmoid function. These weights reflect the importance of the current region's geometric curvature for defect identification and are multiplied channel-by-channel with the original convolutional feature maps to achieve adaptive enhancement of the feature response in high-curvature regions.

[0048] Finally, the weighted feature map is input into the context-aware decoder. This decoder fuses dilated convolutional layers and nonlocal operation layers. The former expands the receptive field to capture the overall contour of elongated defects, while the latter establishes long-range dependencies between any two points in the image, thereby strengthening the consistent representation of scattered but semantically related defect regions. The decoder progressively upsamples and outputs a pixel-level defect probability map with the same size as the input image.

[0049] For example, an inspection is performed on a composite stainless steel pipe fitting consisting of a straight section and a high-curvature bend. After normalizing the image input, the multi-scale feature encoder effectively extracts preliminary features of surface scratches and pits; the geometric attention module automatically increases the weight of the corresponding channel based on the high Gaussian curvature value of the bend area, thus enhancing the response to micro-cracks; and the context-aware decoder connects the broken crack segments into a continuous structure. The final output defect probability map has clear boundaries, with no false positives or false negatives.

[0050] By inputting the normalized image into a defect detection neural network, which includes a multi-scale feature encoder, a geometric attention module, and a context-aware decoder, the geometric attention module performs channel weighting on the convolutional feature map guided by the Gaussian curvature. This explicitly incorporates the geometric prior of the pipe surface into the deep learning process, enabling the network to dynamically adjust the feature attention intensity based on the local curvature. Compared to traditional general detection models, this method significantly improves the sensitivity to small defects in high-curvature regions while suppressing false defect responses caused by geometric abrupt changes, resulting in a significant improvement in overall detection accuracy and robustness.

[0051] S60: Outputs a pixel-level defect probability map and projects the defect results back into three-dimensional space through an inverse mapping function.

[0052] Specifically, the first step involves analyzing a two-dimensional planar image using a trained defect detection neural network. In this step, each pixel is assigned a probability value representing the likelihood of a defect, forming a complete pixel-level defect probability map. This map clearly shows the risk level of defects in different areas, providing a foundation for subsequent accurate identification and localization.

[0053] Next, the defect location information in the two-dimensional plane is transformed back to the original three-dimensional spatial coordinate system through the inverse process of the previously constructed local isometric mapping function, i.e., the inverse mapping function. In this process, each defect point marked on the two-dimensional plane can find its corresponding three-dimensional spatial location, ensuring an accurate correspondence between image analysis and the actual physical object.

[0054] For example, a pixel-level defect probability map is automatically generated from an unfolded image of a calibrated stainless steel pipe fitting using a deep learning-based defect detection algorithm. Each pixel in the image is assigned a value between 0 and 1, representing the probability that the point is a defect. Then, using the previously constructed inverse mapping function, these potential defect points marked in the two-dimensional unfolded image are mapped back to the corresponding pipe fitting surface in three-dimensional space. Results show that, compared to traditional methods, this invention can more accurately locate the actual position of defects and reduce misjudgments caused by complex geometry.

[0055] Outputting pixel-level defect probability maps and projecting the defect results back into 3D space using an inverse mapping function not only improves the accuracy of defect detection but also allows the detection results to be directly applied to subsequent repair or replacement decisions. Through detailed analysis at the pixel level, combined with inverse mapping technology, an effective transformation from the image level to the physical object level is achieved, significantly enhancing the ability to assess surface defects on complex-shaped workpieces and providing reliable data support for automated quality control.

[0056] In one embodiment, in step S10, the Gaussian curvature is calculated by principal curvature, which is the curvature value of each point in the three-dimensional point cloud along two orthogonal principal directions.

[0057] Specifically, Gaussian curvature is calculated based on principal curvatures. The curvature values ​​of each point in a 3D point cloud along two orthogonal principal directions are called principal curvatures. Principal curvatures are key parameters describing the local bending degree of a surface. At any given point, there are two mutually perpendicular directions; the curvature values ​​calculated along these two directions are called the first principal curvature and the second principal curvature, respectively. These two values ​​accurately reflect the surface morphology at that point.

[0058] To obtain the principal curvatures, a small neighborhood suitable for the analysis region is first constructed. Then, a fitting algorithm is used to find the best-fitting quadratic surface within this small neighborhood. Next, the principal curvatures of this quadratic surface are calculated. These steps ensure a faithful representation of the original data and also lay the foundation for subsequent Gaussian curvature calculations.

[0059] The calculation formula is as follows: Where K represents the Gaussian curvature; Indicates the first principal curvature; This represents the second principal curvature. The above method can effectively extract the Gaussian curvature values ​​of each point from 3D point cloud data.

[0060] For example, a metal part with a complex geometry is scanned to obtain its 3D point cloud data. Each point in this point cloud is then processed using the method described above. First, a suitable neighborhood size is determined. Then, the least squares method is applied to fit a quadratic surface. Finally, the first and second principal curvatures of each point are calculated, and then... (The sentence is incomplete and requires further context to be fully translated.) The Gaussian curvature was obtained. The results show that this method can accurately identify depressions and protrusions on the surface of parts, as well as other minute deformations, providing higher accuracy than traditional methods.

[0061] The principal curvature is the curvature value of each point in the 3D point cloud along two orthogonal principal directions, making the analysis of the surface morphology of 3D objects more refined and accurate. By calculating the Gaussian curvature, not only obvious defects can be identified, but also subtle morphological changes can be captured. This has significant implications for multiple fields such as quality control, materials research, and product design.

[0062] In one embodiment, the local isometric mapping function in step S30 is used to map the curved surface region to a two-dimensional plane without stretching, so as to eliminate image distortion caused by the geometry of the pipe fitting.

[0063] Specifically, the local isometric mapping function is used to map curved regions onto a two-dimensional plane without stretching, thereby eliminating image distortion caused by the geometry of the pipe fitting. The key is to maintain the geodesic distance between neighboring points on the curved surface as close as possible to their Euclidean distance in the two-dimensional plane after mapping, thus avoiding texture stretching or compression.

[0064] The mapping process begins with 3D point cloud data within each geometrically homogeneous region. A local differential geometry is constructed using the normal vector and Gaussian curvature of each point. Within the neighborhood of each point, two orthogonal tangent vectors are determined by the principal direction, establishing a local tangent plane coordinate system. Subsequently, the pixels in the original image corresponding to this surface region are projected onto the tangent plane along this local coordinate system, forming the initial unfolding result. To further improve isometry, energy optimization is performed on the initial unfolding result, minimizing the error between the distance between adjacent point pairs in the 2D unfolded image and their geodesic distance on the original surface. Finally, a local isometry mapping function is obtained.

[0065] The mapping function is denoted as Its function is to map a point p on a surface in three-dimensional space to a point (u,v) on a two-dimensional plane, that is... The function is continuously differentiable over the entire geometrically homogeneous region and satisfies the local length preservation property.

[0066] For example, a curved branch area of ​​a reducing tee pipe fitting is processed. This area has a gentle change in Gaussian curvature and is divided into a geometrically homogeneous region. After acquiring multi-view images, a local isometric mapping function is constructed using the method described above to correct the elliptical reflective patches caused by the curved surface in the original image into circular textures. This effectively restores the true geometric shape of the surface. In the unfolded image, surface scratches appear as continuous straight lines, rather than the distorted curves in the original image.

[0067] The local isometric mapping function is used to map curved surface regions onto a two-dimensional plane without stretching, thereby eliminating image distortion caused by the geometry of the pipe fitting. Local isometric mapping significantly reduces visual distortion introduced by the complex geometry of the pipe fitting, maintaining the true proportions and continuity of defect morphology in the two-dimensional image, providing high-quality input for subsequent neural network recognition. Furthermore, this method is not dependent on a specific pipe type and can be applied to pipe surfaces with arbitrarily varying curvatures, exhibiting good versatility and engineering applicability.

[0068] In one embodiment, in step S40, the illumination normalization process uses the following formula:

[0069] ;

[0070] in, This represents the brightness value of the normalized image at pixel location (u,v); This represents the original brightness value at the same pixel position in a two-dimensional planar image after correction and unfolding using a local isometric mapping function; The angle of incidence of light at pixel position (u,v) is determined by the angle between the direction of the light source and the normal vector of that point; cos represents the cosine function. This represents the numerical stability constant, with a value of 0.001.

[0071] The physical basis of this formula stems from Lambert's cosine law, which states that the intensity of reflected light from an ideal diffuse surface is proportional to the cosine of the angle of incidence. During the imaging process of pipe surfaces, the different directions of the normal vectors at various points on the curved surface cause areas of the same material to exhibit inconsistent brightness in the image. Illumination normalization is achieved by dividing by... This compensates for the brightness differences caused by geometric orientation, so that regions of the same material have a uniform grayscale representation in the normalized image.

[0072] In one embodiment, a multi-view imaging of the surface of a stainless steel bent pipe is performed. The side closer to the light source appears brighter because its normal vector is close to the direction of the light source, while the side farther from the light source appears darker due to its larger angle of incidence. After the above illumination normalization processing, the same smooth area that was originally uneven in brightness and darkness becomes uniform in grayscale in the normalized image. The actual corrosion spots, due to their different reflective properties compared to the substrate material, retain their low reflectivity, thus forming a clear contrast with the background.

[0073] The use of the illumination normalization formula effectively suppresses non-defect-related brightness variations caused by the curved geometry of the pipe fittings, preventing the neural network from misclassifying normal reflective areas as surface defects. Simultaneously, it enhances the contrast between real defects and normal areas, improving the accuracy and robustness of subsequent defect detection. This method does not rely on additional light source calibration; it only requires known light source direction and point cloud normal vectors, demonstrating good practicality and deployability.

[0074] In one embodiment, in step S50, when the geometric attention module performs channel weighting on the convolutional feature map, it uses Gaussian curvature and global average pooling to generate attention weights through a learnable weight matrix and a Sigmoid function, and then multiplies them channel by channel with the original convolutional feature map.

[0075] Specifically, firstly, the Gaussian curvature values ​​at corresponding locations in the unfolded 2D image are used to construct a Gaussian curvature map with the same spatial dimensions as the input image. This Gaussian curvature map reflects the degree of curvature in different regions of the pipe surface. Next, a global average pooling operation is performed on this Gaussian curvature map to obtain a scalar value. This scalar value represents the overall curvature level of the geometrically homogeneous region corresponding to the current input image. Then, this scalar value is input into a learnable weight matrix, linearly transformed, and then mapped to a value between 0 and 1 using the Sigmoid function. This value is the geometric attention weight, used to adjust the neural network's attention to the current feature channel. Finally, this geometric attention weight is multiplied channel-by-channel with the original convolutional feature map along the channel dimension to achieve adaptive enhancement or suppression of feature responses in regions with different curvature.

[0076] The mathematical expression for this process is:

[0077] ;

[0078] in, This represents the weighted feature map obtained after processing by the geometric attention module. The weighted feature map is the result of adaptive adjustment of the original convolutional feature map according to its corresponding geometric attributes in the geometric attention module, reflecting the different degrees of attention the defect detection neural network pays to features in different regions. This represents the Sigmoid function, which maps input values ​​to a range between 0 and 1, forming a numerical range suitable for use as attention weights. This mapping ensures that the output values ​​can smoothly represent different weight levels while effectively controlling the channel activation intensity of the feature map. This represents the learnable weight matrix. It is automatically optimized during the training process. Its function is to perform a linear transformation on the Gaussian curvature scalar after global average pooling, thereby generating appropriate attention weights for each feature channel. This mechanism allows the network to automatically learn the optimal weight allocation strategy according to different task requirements. This indicates that a global average pooling operation is performed on the Gaussian curvature map K, and the result is a scalar value. Essentially, this operation calculates the average value of the entire Gaussian curvature map to reflect the overall curvature level of the current image or feature map. This step simplifies subsequent calculations and helps improve the efficiency of the defect detection neural network in utilizing overall structural information. K represents the Gaussian curvature; This represents the convolutional feature map output by the multi-scale feature encoder. This represents a channel-wise multiplication operation, where the generated attention weights are applied to each corresponding channel of the original convolutional feature map, thereby enhancing or suppressing the feature response of a specific region. This operation allows the defect detection neural network to focus on the most informative regions while ignoring relatively unimportant parts.

[0079] In one embodiment, in step S50, the context-aware decoder includes a dilated convolutional layer and a nonlocal operation layer for capturing long-range spatial dependencies of defective regions.

[0080] Specifically, the context-aware decoder includes dilated convolutional layers and nonlocal operation layers to capture long-range spatial dependencies in defective regions. Its specific structure and operation process are as follows.

[0081] Dilated convolutional layers expand the receptive field of the convolutional kernel without increasing the number of parameters or computational cost. By inserting zero values ​​at fixed intervals between the weights of a standard convolutional kernel, the convolutional operation can skip some input pixels, thus covering a wider spatial range. Dilated convolutional layers set up multiple branches with different dilation rates to capture contextual information at different scales, and fuse these multi-scale features to enhance adaptability to the diversity of defect morphologies.

[0082] The nonlocal operation layer establishes global dependencies by calculating the similarity between any two locations in the feature map. Specifically, for each location in the input feature map, the dot product of its feature vectors with all other locations is calculated to obtain the attention weight matrix. Then, the features of all locations are weighted and summed to generate a new feature representation that includes global contextual information. This operation does not rely on local neighborhoods and can effectively model the relationships between semantically related regions that are far apart.

[0083] In one embodiment, an image of the surface of a curved pipe with multiple scattered microcracks is processed. After preliminary features are extracted by a multi-scale feature encoder, the image is input to a context-aware decoder. The dilated convolutional layer effectively expands the perception range of the overall contour of the slender cracks, while the non-local operation layer associates similar crack features located at opposite ends of the image, enhancing the network's ability to consistently identify similar defects. In the final output defect probability map, the boundaries of each crack region are clear, the continuity is good, and there are no breaks or false connections.

[0084] The context-aware decoder includes dilated convolutional layers and nonlocal operation layers to capture long-range spatial dependencies in defect regions, significantly improving the neural network's ability to understand the spatial structure of defects. This is particularly suitable for long, winding, or sparsely distributed defects on pipe surfaces. By fusing local details with global contextual information, it effectively avoids the defects fragmentation, missed detections, or false merging problems caused by the limited receptive field of traditional decoders. This improves overall detection accuracy and result reliability.

[0085] In one embodiment, after performing illumination normalization processing on the two-dimensional planar image to obtain a normalized image in step S40, the method further includes performing learnable scaling and translation operations on the normalization result.

[0086] Specifically, the normalization process further includes a learnable scaling and translation operation on the normalization result. Specifically, after completing the illumination normalization process, two trainable parameters are introduced to perform a linear transformation on each pixel value of the normalized image. One parameter controls the scaling factor, and the other controls the translation bias. The operation takes the following form:

[0087] ;

[0088] in, This represents the brightness value at pixel position (u,v) of the image after being adjusted by learnable scaling and translation operations. This value serves as the input to the subsequent defect detection neural network and has optimized grayscale distribution characteristics. This represents the learnable scaling parameter, which is a scalar and is actually a trainable weight in the neural network. It is automatically updated through backpropagation during training and its function is to adaptively stretch or compress the overall brightness range of the normalized image. This represents the brightness value at pixel position (u,v) of the image obtained after illumination normalization processing. This value has eliminated the brightness difference caused by different incident angles of light and reflects the true reflection characteristics of the pipe surface material and defects. The learnable translation parameter is a scalar and is actually a trainable bias term in the neural network. It is jointly optimized with γ during training. Its function is to shift the overall gray level of the normalized image up and down to match the optimal input range of the subsequent neural network. Representing pixel coordinates in a two-dimensional image, respectively Corresponding column index of the image The row index of the corresponding image is used to locate the specific spatial position in the image.

[0089] This operation does not change the spatial structure of the image, but only performs adaptive calibration on the brightness dynamic range, making the normalized image more suitable for the input distribution of the subsequent defect detection neural network. It plays a role similar to the affine transformation in batch normalization, but its target is an image with clear physical meaning after geometric and illumination correction, rather than an intermediate feature map. Therefore, it has stronger interpretability and generalization ability.

[0090] In one embodiment, images of stainless steel and carbon steel pipe fittings with different surface finishes were acquired and subjected to illumination normalization. While the initial normalization eliminated the influence of the incident angle, the difference in material reflectivity still resulted in a systematic shift in the grayscale mean of the two types of fittings. By introducing learnable scaling and translation operations, the network automatically shifts the overall grayscale of the carbon steel fittings upwards during the training phase, while moderately compressing the dynamic range of the stainless steel fittings. This ensures that the two materials have similar statistical characteristics before being input into the defect detection neural network.

[0091] By applying learnable scaling and translation operations to the normalization results, the method's adaptability to different materials and imaging conditions is improved, avoiding performance degradation caused by the normalized grayscale distribution deviating from the optimal input range of the defect detection neural network. Furthermore, since the scaling and translation parameters are globally shared and can be trained end-to-end, there is no need to manually set thresholds or gain coefficients, enhancing the system's automation and robustness.

[0092] Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for intelligent detection of surface defects in pipe fittings based on image recognition, characterized in that, include: Acquire the three-dimensional point cloud data of the pipe fitting to be inspected, and calculate the Gaussian curvature and normal vector of each point based on the three-dimensional point cloud data; Based on the difference in Gaussian curvature, the surface of the pipe fitting is divided into several geometrically homogeneous regions, such that the absolute value of the difference in Gaussian curvature between any two points in each geometrically homogeneous region is less than a preset curvature tolerance threshold. Original images from multiple perspectives are acquired within each geometrically homogeneous region, and a local isometric mapping function is constructed using the normal vector and Gaussian curvature to correct and unfold the original images into two-dimensional planar images. The two-dimensional planar image is subjected to illumination normalization processing to obtain a normalized image; wherein the illumination normalization processing adopts the following formula: ； in, This represents the brightness value of the normalized image at pixel location (u,v); This represents the original brightness value at the same pixel position in a two-dimensional planar image after correction and unfolding using a local isometric mapping function; The angle of incidence of light at pixel position (u,v) is determined by the angle between the direction of the light source and the normal vector of the pixel position; cos represents the cosine function. This represents the numerical stability constant, with a value of 0.

001. The normalized image is input into a defect detection neural network, which includes a multi-scale feature encoder, a geometric attention module, and a context-aware decoder, wherein the geometric attention module performs channel weighting on the convolutional feature map guided by the Gaussian curvature. Output a pixel-level defect probability map and project the defect results back into three-dimensional space using an inverse mapping function.

2. The intelligent detection method for pipe surface defects based on image recognition according to claim 1, characterized in that, The Gaussian curvature is calculated using principal curvature, which is the curvature value of each point in the three-dimensional point cloud along two orthogonal principal directions.

3. The intelligent detection method for pipe surface defects based on image recognition according to claim 1, characterized in that, The local isometric mapping function is used to map the curved surface region to a two-dimensional plane without stretching, so as to eliminate image distortion caused by the geometry of the pipe fitting.

4. The intelligent detection method for pipe surface defects based on image recognition according to claim 1, characterized in that, When the geometric attention module performs channel weighting on the convolutional feature map, it uses Gaussian curvature and global average pooling to generate attention weights through a learnable weight matrix and a Sigmoid function, and then multiplies them with the original convolutional feature map channel by channel.

5. The intelligent detection method for surface defects of pipe fittings based on image recognition according to claim 1, characterized in that, The context-aware decoder includes dilated convolutional layers and nonlocal operation layers to capture long-range spatial dependencies in defective regions.

6. The intelligent detection method for surface defects of pipe fittings based on image recognition according to claim 1, characterized in that, After obtaining the normalized image, the process further includes: introducing two trainable parameters, used to perform a linear transformation on each pixel value of the normalized image; one parameter controls the scaling factor, and the other controls the translation bias, with the specific operation as follows: ； in, This represents the brightness value at pixel location (u,v) of the image after being adjusted by learnable scaling and translation operations. Indicates the learnable scaling parameter; This represents the brightness value at pixel location (u,v) in the image obtained after illumination normalization. Represents the learnable translation parameters; Corresponding column index of the image The row index of the corresponding image.

Images