A real-time monitoring method for a punching line for pipe machining

Abstract

The application relates to the technical field of image processing, in particular to a stamping line real-time monitoring method for pipeline processing, which comprises the following steps: acquiring a surface image of a pipeline, calculating the gradient amplitude and local structure main direction of a pixel point, generating a candidate path, determining the path structure continuity by accumulating the gradient amplitude and the main direction consistency of all pixel points on the path; determining the gradient direction consistency by analyzing the gradient direction of the symmetric sampling points on the normal line of the local structure main direction of the pixel point, calculating the fusion saliency of the pixel point by combining the path structure continuity and the gradient direction consistency, generating a fusion saliency atlas, carrying out double-threshold segmentation processing to extract all edge contours of the pipeline, and determining whether the pipeline has defects according to the morphological characteristics of the edge contours, so as to realize the real-time monitoring of the pipeline quality in the stamping production line. The method improves the accuracy of pipeline processing monitoring.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing. Specifically, it relates to a stamping line real-time monitoring method for pipe processing. BACKGROUND

[0002] In modern industrial manufacturing, especially in the field of pipe processing, stamping forming is a key process. In order to ensure product quality, a real-time monitoring system based on machine vision is usually deployed to monitor the geometric size and surface quality of the pipe. In such a system, edge contour is the core basis for realizing subsequent size measurement, defect identification and other advanced functions. An ideal edge detection algorithm needs to accurately and completely extract the physical contour of the pipe and the real boundary of the surface defect from an image containing complex background texture, providing reliable data support for the stable operation of the entire monitoring system.

[0003] At present, the edge detection technology commonly used in the industry is mostly based on local gradient operators, of which the most representative is the Canny edge detection algorithm and its various improved methods. The core idea of this kind of method is to judge whether each pixel point in the image is an edge contour by calculating its gradient.

[0004] However, after the stamping process of the pipe, the surface of the metal material will form a large number of stretching textures with different directions and dense distribution. These surface textures produced by normal process will also produce strong gray gradient in local scale, and their gradient response may even be stronger than that of some early micro cracks. When the traditional local gradient operator faces these pseudo-edges caused by dense textures and true edges caused by real defects, it cannot effectively distinguish between the two because its analysis range is limited to the neighborhood of several pixels and lacks consideration of the macroscopic continuity of the structure, resulting in the contours of real defects being submerged in a large number of meaningless and chaotic texture edges in the final recognized edge contour, which reduces the accuracy of pipe defect identification. SUMMARY

[0005] To solve the problem that the existing technology adopts an edge detection method based on local gradient operators, which cannot effectively distinguish between the normal stretching texture edges on the surface of the pipe after stamping and the real defect edges, thereby reducing the accuracy of pipe defect identification, the present application proposes a stamping line real-time monitoring method for pipe processing, comprising:

[0006] Obtaining the surface image of any one pipe in the stamping production line, calculating the gradient amplitude and local structure main direction of each pixel point in the surface image based on the local structure main direction of each pixel point, generating a candidate path for the pixel point, and the tangent direction of each pixel point passing through the candidate path is consistent with the local structure main direction of the pixel point;

[0007] performing path integration along each candidate path of the pixel point, calculating path structure continuity of the pixel point by accumulating gradient amplitudes of all pixel points passed on the candidate path and consistency of unit vectors of local structure principal directions; performing bidirectional offsetting based on normal directions of the local structure principal direction of each pixel point to obtain two symmetrical sampling points, and determining gradient direction consistency on both sides of the pixel point according to dot product results of unit vectors of gradient directions of the two symmetrical sampling points;

[0008] combining path structure continuity of each pixel point and gradient direction consistency on both sides of the pixel point, determining fusion saliency of the pixel point to obtain a fusion saliency map composed of all pixel points, performing adaptive double-threshold segmentation processing on the fusion saliency map to obtain all edge contours of the pipeline, and monitoring whether the pipeline has defects based on morphological features of all edge contours.

[0009] The technical scheme first calculates local structure principal directions of pixel points and generates candidate paths, thereby providing a direction basis for analyzing line structures in an image, calculates path structure continuity through path integration, thereby evaluating geometric morphologies of structures from a macroscopic scale, not only considers gradient strengths of points on a path, but more importantly quantifies smoothness and continuity of the path by evaluating consistency of unit vectors of local structure principal directions of the pixel points, thereby effectively preliminarily distinguishing long and continuous real contours from short and chaotic texture noises, then analyzes texture features of pixel points on both sides of a normal direction of the local structure principal direction of a pixel point from a microscopic scale, gradient directions of symmetrical pixel points on both sides of the normal direction of a real contour boundary line are all directed to the same direction, while gradient directions of symmetrical pixel points on both sides of a normal direction of a line texture on a pipeline surface are opposite, the gradient direction consistency is calculated to reflect the difference between the texture features, thereby identifying contours with real physical boundary characteristics, finally, the macroscopic analysis result and the microscopic analysis result are fused, thereby performing double verification from two orthogonal dimensions of geometric morphologies and physical properties, thereby clearly separating real and continuous defect contours from a complex texture background, and improving accuracy of a pipeline processing monitoring result in a complex texture background.

[0010] As a preferred scheme, the gradient amplitude of the pixel point and the local structure principal direction are determined based on the following manner: performing Gaussian filtering denoising processing on the surface image to generate a denoised image; calculating the gradient amplitude of each pixel point of the denoised image by using a Sobel operator; constructing a covariance matrix of gradient information in a preset neighborhood window of each pixel point; and determining a feature vector corresponding to a maximum eigenvalue of the covariance matrix as the local structure principal direction of the pixel point.

[0011] As a preferred approach, a candidate path for each pixel is generated based on the principal orientation of its local structure. This includes: treating the orientation field formed by the principal orientations of all pixels as a continuous vector field; defining the candidate path as a solution to an ordinary differential equation, whereby the ordinary differential equation constrains the tangent direction at any point on the path to be equal to the direction of that point in the vector field; and solving the ordinary differential equation using numerical integration starting from each pixel to generate a series of discrete path points to constitute the candidate path for that pixel.

[0012] As a preferred option, the path structure continuity of pixels is determined based on the following relationship:

[0013]

[0014] in, It is the first Path structure continuity of individual pixels For the first The pixels traversed on the candidate path of the i-th pixel are related to the pixels of the i-th pixel. The arc length between pixels Let [0,L] be the upper limit of the arc length for the path integral, and [0,L] be the integration interval. For candidate paths and the first The arc length between pixels is The gradient magnitude of the pixel at that location. For candidate paths and the first The arc length between pixels is The unit vector of the principal direction of the local structure of the pixel at that location. It is a tiny differential element of the arc length. For candidate paths and the first The arc length between pixels is The unit vector of the principal direction of the local structure of the pixel at that location. For the dot product operation of vectors, It is a preset exponential parameter.

[0015] This technical solution provides an objective evaluation standard for assessing whether a pixel belongs to a macroscopic continuous structure by giving a path structure integral formula that integrates gradient magnitude and direction consistency. It can endow long and smooth real contours with high path structure continuity, while suppressing short and broken surface textures, thus achieving a preliminary and effective distinction between the two.

[0016] As a preferred approach, the consistency of gradient directions on both sides of a pixel is determined based on the following relationship:

[0017]

[0018] wherein, is the gradient direction consistency of the two sides of the pixel point, is the unit vector of the gradient direction of the sampling point obtained by forward offsetting the normal direction of the main direction of the local structure of the pixel point, is the unit vector of the gradient direction of the sampling point obtained by reverse offsetting the normal direction of the main direction of the local structure of the pixel point, is the dot product operation of vectors.

[0019] This technical solution is based on the difference in texture characteristics between the pixel points on both sides of the real physical boundary and the pixel points on both sides of the linear texture, and constructs a gradient direction consistency based on the gradient direction of the normal direction on both sides, which provides a basis for judging whether a continuous linear structure has the characteristics of a real physical boundary, and can accurately distinguish the real physical boundary with opposite gradients on both sides, thereby improving the accuracy of the final edge extraction.

[0020] As a preferred solution, the fusion saliency of the pixel point is determined based on the following relationship:

[0021]

[0022] wherein, is the fusion saliency of the pixel point, is the path structure continuity of the pixel point, is the gradient direction consistency of the two sides of the pixel point, is a preset influence weight coefficient, used to adjust the influence degree of on .

[0023] This technical solution realizes the effective fusion of the logical AND operation of the information of the path structure continuity and the gradient direction consistency in two independent dimensions through a multiplication fusion formula. Only the pixel points that simultaneously satisfy the macro-continuity and the local boundary characteristics can obtain the final high saliency response, thereby highlighting the real defect profile to the greatest extent and suppressing all other types of interference.

[0024] As a preferred solution, the upper limit of the arc length of the path integral is determined based on the following manner: a plurality of standard pipeline images conforming to the processing standard are pre-acquired; for each pixel point in each standard pipeline image, a candidate path of the pixel point is obtained, the arc lengths of the candidate paths of all pixel points of all standard pipeline images are counted, and the arc lengths of all candidate paths are sorted in ascending order, and then the upper limit of the arc length of the path integral is determined by using the percentile method.

[0025] As a preferred solution, the distance moved by the forward offset and the reverse offset respectively is determined based on the following manner: a plurality of standard pipeline images conforming to the processing standard are pre-acquired; for each pixel point in each standard pipeline image, the texture width in the normal direction of the main direction of the local structure of the pixel point is obtained, and the maximum texture width is determined according to all pixel points of all standard pipeline images; the distance moved by the forward offset and the reverse offset respectively is set to be not less than the maximum texture width.

[0026] This technical solution associates the offset distance with the maximum texture width observed in the sample, providing a sampling scale that can effectively cross the texture itself for the calculation of gradient direction consistency, ensuring that the two sampling points and respectively fall on both sides of the texture when calculating the gradient direction consistency, rather than inside the texture, thereby ensuring that the factor can correctly reflect the characteristics of the true physical boundary and avoiding the problem of misjudging strong texture as a boundary due to too small sampling distance.

[0027] As a preferred solution, the morphology features of all edge contours are used to monitor whether the pipeline has defects in real time, including: using Hough transform or RANSAC algorithm to perform linear fitting on all edge contours to obtain a group of straight line segments, and traversing all possible combinations of the group of straight line segments; according to the length, parallelism and spacing of the group of straight line segments and other geometric features, a pair of straight line segments most consistent with the physical features of the pipeline body is identified, which is determined as the pipeline body contour; all edge contours outside the pipeline body contour are determined as potential defect contours; and the defect recognition result of the pipeline is determined based on the pipeline body contour and the potential defect contour.

[0028] As a preferred solution, the defect recognition result of the pipeline is determined based on the pipeline body contour and the potential defect contour, including: for the pipeline body contour, the average spacing and mutual parallelism between the straight line segments contained in the pipeline body contour are calculated, and are compared with the pre-acquired tolerance range respectively to determine whether the pipeline has structural defects;

[0029] For the potential defect contour, the geometric features of the potential defect contour are extracted, and the geometric features are matched with the pre-constructed defect feature library to determine whether the pipeline has surface defects.

[0030] The present application has the following effects:

[0031] The present application firstly aims at the problem that the existing edge detection is easily disturbed by the complex stretch texture of the pipe surface, and through constructing an evaluation mechanism based on non-local path integration, the continuity of the edge is evaluated from the macro scale, which can effectively distinguish the long and continuous real defect profile from the short and broken surface texture, and improve the anti-texture interference ability of the algorithm, and further introduce a physical property discrimination factor based on the consistency of the gradient direction to preferentially enhance the profile line with the real physical boundary characteristics, which can effectively distinguish the real physical boundary from the surface defect, and improve the accuracy of the pipe processing monitoring. BRIEF DESCRIPTION OF DRAWINGS

[0032] Figure 1 is a flowchart of the present application. DETAILED DESCRIPTION

[0033] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application.

[0034] Referring to Figure 1 A stamping line real-time monitoring method for pipe processing, specifically comprising the following steps:

[0035] S1: obtaining a surface image of a pipe on a stamping production line, calculating the gradient amplitude and local structure main direction of a pixel point to generate a candidate path of the pixel point.

[0036] In a complex stamping production line environment, it is unreliable to directly identify defects from the surface image of the pipe, and this step calculates two core basic attributes for each pixel point in the image: one is the gradient amplitude of the pixel point, which is used to reflect the degree of gray scale change, and the other is the local structure main direction of the pixel point, which is used to reflect the main direction of change. These two attributes are the basis for all subsequent advanced analysis, especially path tracking and structure identification.

[0037] Firstly, the surface image of any one pipe in the stamping production line is obtained and gray scale processing is performed, considering that random noise (such as sensor thermal noise) may exist in the industrial field and will interfere with subsequent calculation, the surface image needs to be denoised by Gaussian filtering to generate a denoised image, and the following operations are performed for each pixel point in the denoised image to calculate its gradient amplitude and local structure main direction:

[0038] The Sobel operator or equivalent gradient operator is used to calculate the gradient amplitude of the pixel point, and the gradient amplitude is a scalar, which reflects the degree of gray scale change in the neighborhood of the pixel point, and is the basic basis for judging whether it is an edge or a surface stretch texture.

[0039] In order to determine the local texture direction of the pixel point, 5 Within a neighborhood window of size 5, a covariance matrix of gradient information is constructed. This covariance matrix describes the distribution of all gradient vectors within the neighborhood. According to the principles of linear algebra, the eigenvector corresponding to the largest eigenvalue of this covariance matrix points to the direction in which the gradient energy distribution is most concentrated, i.e., the direction in which the structural changes within the neighborhood are most consistent. Therefore, this eigenvector is determined as the dominant local structural direction of that pixel.

[0040] This step transforms the original surface image into two feature maps rich in structural information: one is a gradient magnitude map composed of the magnitudes of the gradient vectors of all pixels, indicating the intensity of grayscale changes; the other is a direction field composed of the principal directions of the local structure of all pixels, indicating the direction of grayscale changes. This processing method allows subsequent algorithms to no longer rely solely on isolated pixel grayscale values, but to make judgments based on more stable structural features. In particular, the introduction of the principal directions of the local structure provides reliable guidance for tracing along the real texture and contour to generate candidate paths, which is a key step in solving problems of traditional algorithms.

[0041] After obtaining the principal direction of the local structure of each pixel, a candidate path can be generated for that pixel. This path is not a simple pixel connection, but is defined as a smooth curve. The tangent direction at any point on the curve is consistent with the principal direction of the local structure at that point. This is usually achieved by iteratively tracking the local structure from the current pixel with a step size of one pixel, based on its principal direction of local structure.

[0042] Specifically, in order to find the path for each pixel, an arbitrary pixel in the image is given as the starting point. To find a path starting from this pixel and extending along the main direction of the image texture, this path is defined as the solution to the following ordinary differential equation:

[0043]

[0044] in: This represents any pixel in the image, with that pixel as the starting point. This represents the arc length from the starting point to any point on the main direction of the local structure of the pixel. Indicates the distance from the starting point on the path. Arc length is Location coordinates, Is the path in The direction of the tangent is the direction in which the path moves. In position The unit vector of the local structure principal direction of the pixel at that location.

[0045] Since the direction of the path at each pixel must be equal to the principal direction of the image texture at that pixel, by solving this equation, a smooth curve that always follows the local texture of the image can be obtained, which is the path of the texture found initially, and is used to distinguish the texture of the pipe surface and the defect profile later.

[0046] In actual calculation, since the image exists in the form of a pixel grid, it is impossible to truly realize completely continuous calculation, so a numerical method (such as Euler method or Runge-Kutta method) is used to start from the starting point and gradually advance forward with a very small step size to generate a series of discrete path points to constitute a candidate path, and the generated path will not deviate from the texture direction, and even if the texture is curved, it can naturally fit the shape.

[0047] In summary, the core operation of this step is feature extraction, that is, to promote the original pixel information to structured features, which provides high-quality and reproducible input data for subsequent path structure continuity calculation and gradient direction consistency analysis through a standardized image processing process, ensuring the accuracy and reliability of the entire monitoring method.

[0048] S2: Perform integration operation along the candidate path of the pixel point to calculate the path structure continuity.

[0049] The fundamental defect of the traditional edge detection algorithm is its local decision-making, which leads to its inability to distinguish between macroscopically continuous real defects (such as long cracks, scratches, etc.) and surface textures with locally strong gradient responses. Therefore, simply generating a candidate path is not enough, and it is also necessary to determine whether this path represents a real and stable texture. For this purpose, an integration process is designed to accumulate the structure support degree along the path, and the gradient amplitude of the next pixel point along the path is used as the weight of the current integration term. Even if the gradient amplitude is high, if the path itself frequently turns or is unstable in direction, it cannot be considered as a highly coherent texture.

[0050] By introducing this global macroscopic analysis perspective, path integration is performed along the candidate path of the pixel point, thereby evaluating the macroscopic continuity of the structure in which each pixel point is located. Specifically:

[0051] For each pixel point, along the candidate path of the pixel point, the path structure continuity of the pixel point is calculated by accumulating the gradient amplitudes of all the pixel points passed along the candidate path and the consistency of the unit vector of the local structure principal direction.

[0052]

[0053] wherein, is the path structure continuity of the pixel point at the i-th step of the integration process. The path structure continuity of each pixel reflects the cumulative gradient strength on the candidate path. The higher the value, the more likely the pixel is to lie on a long and continuous structure. For the first The pixels traversed on the candidate path of the i-th pixel are related to the pixels of the i-th pixel. The arc length between pixels, in pixels. This represents the upper limit of the arc length for the path integral. For the integration interval, For candidate paths and the first The arc length between pixels is The gradient magnitude of a pixel reflects the edge or texture intensity at that point on the path. For candidate paths and the first The arc length between pixels is The unit vector of the local structure of the pixel at that location (containing only direction information). It is a tiny differential element of the arc length, which can be set to a pixel length in discrete calculations. For candidate paths and the first The arc length between pixels is The unit vector of the principal direction of the local structure of the pixel at that location. For the dot product operation of vectors, It is a preset exponential parameter.

[0054] in, As the gradient magnitude term, it forms the basis of integration. It ensures that the integral value accumulates effectively only when the path continuously traverses high-gradient regions. If the path enters a flat region (where the gradient magnitude decreases), the growth of the integral value stagnates. This constitutes a directional consistency factor. The dot product of the direction vectors of two adjacent points on the path is calculated. If the path is straight, the two vectors have the same direction, and the dot product is 1. If the path bends, the dot product decreases. If the path makes a 180-degree turn, the dot product is -1. Adding 1 and then dividing by 2 expands the range of the dot product from... Linear mapping to Thus, a straight path corresponds to a factor value of 1, a 90-degree turn corresponds to 0.5, and a 180-degree U-turn corresponds to 0. Typically, a value greater than 1 is chosen; here, it's set to 2.0 to serve as a penalty amplification. Even if the direction deviates slightly, that is When the value is small, the directional consistency factor will be significantly compressed, thus more strongly suppressing the growth of the integral value. This means that only textures with stable orientation and almost no curvature can output a large path structure continuity.

[0055] It is worth mentioning that the path structure continuity calculated for the pixel points on the same path is also the same, so the path structure continuity not only indicates whether the path is continuous, but also can evaluate the possibility of the pixel point being on a real path, so the path structure continuity can be regarded as a property of the pixel point.

[0056] wherein the upper limit of the arc length of the path integral is determined based on a data-driven calibration method: first, a plurality of standard pipeline images conforming to the processing standard and free of defects are pre-collected; then, for each pixel point in these images, a candidate path thereof is generated and the arc lengths of all the paths are counted; finally, all the arc lengths are sorted from small to large, and the value at the 99.9% percentile point is determined by using the percentile method. This method ensures that is statistically obtained based on real data, and can represent the maximum path length that can be formed by normal surface texture, thereby providing an objective basis for distinguishing normal texture and abnormal defects, because the candidate path length of the pixel point of the abnormal defect will be greater than that of the normal texture.

[0057] In summary, the path structure continuity of the pixel point is obtained by multiplying the gradient intensity and the direction consistency, and is accumulated on the path. The linkage effect is that only when a path simultaneously satisfies the two strict conditions that the gradient amplitudes of the points on the path are generally high and the overall trend of the path is smooth and almost without bending, the final path structure continuity will be high. The destruction of any one condition (such as the path entering a low gradient area or the path bending) will significantly inhibit the integral value, and the path structure continuity will be low.

[0058] A micro-crack appears as a thin and continuous dark line on the image. The candidate path generated from an arbitrary point on the crack will extend along the crack itself, the path is long and the direction is stable, and during the integration process, the value is continuously high, the direction consistency factor is continuously close to 1, and the finally calculated path structure continuity will be very high; although the local gradient of the normal stretch texture formed by stamping is also high, they are usually short, broken and different in direction, and the path from a point on the texture will soon stop or cause the direction consistency factor to drop sharply due to the interruption or dramatic change of the direction of the texture, so the final path structure continuity will be low.

[0059] By simulating a process of walking along the structure path and evaluating the path quality, the analysis scale is expanded from the neighborhood of a single pixel to the macro scale of the entire structure, thereby effectively quantifying the continuity of the structure and providing a decisive basis for subsequent separation of the real contour from the complex texture background.

[0060] S3: Determine the gradient direction consistency of the two sides of the pixel point based on the physical boundary attribute.

[0061] After obtaining the path structure continuity of each pixel, macroscopically continuous linear structures can be filtered out. However, these structures may include both the actual outline of the pipe and some continuous linear textures, such as scratches, cracks, and other defects formed on the pipe surface during processes like drawing or stretching. These two types of structures are similar in continuity, but their physical causes are completely different. Specifically:

[0062] The true outline of a pipe is the boundary line between different regions, typically characterized by a significant step in brightness. On one side of the boundary line, there is a brighter area, and on the other side, a darker area. In this case, the gradient directions of symmetrical pixels on both sides of the boundary line's normal direction all point in the same direction, i.e., from the darker area to the brighter area. The better the gradient consistency on both sides of the boundary, the better. However, for linear textures on the pipe surface, such as a scratch, both sides are darker areas. In this case, the gradient directions of symmetrical pixels on both sides of the boundary line's normal direction all point from the sides to the center of the scratch, exhibiting opposite gradient direction characteristics, i.e., the worse the gradient direction consistency.

[0063] Therefore, by examining whether the gradient directions of the adjacent regions on both sides of a pixel are more or less the same or more opposite, we can determine whether the pixel is located on a real boundary or on a linear texture. Specifically:

[0064] For each pixel in the image, a bidirectional offset is performed based on the normal direction of its local principal direction, and the offset distance is... Two symmetrical sampling points are obtained. Then, the consistency of the gradient direction on both sides of the pixel is determined based on the dot product of the unit vectors of the gradient directions of these two symmetrical sampling points. This calculation is based on the following relationship:

[0065]

[0066] In this relation, For the first The gradient directions on both sides of each pixel are consistent, and the value range is [value range missing]. The closer the value is to 1, the more the point conforms to the characteristics of a real physical boundary. For the first The sampling points are obtained by positively offsetting the normal direction of the local structure of each pixel. For the first The sampling points are obtained by inverse offsetting the normal direction of the local structure of each pixel in the principal direction. For sampling points The unit vector of the gradient direction at that point. For sampling points The unit vector of the gradient direction at that point. For vector dot product operation.

[0067] Formula composition and principle:

[0068] The core of this formula is This term, by calculating the dot product of two gradient direction vectors at symmetric sampling points, detects the relationship between the gradient directions on both sides of the edge. For a real physical boundary contour, the gradient directions of the symmetric sampling points on both sides of the normal direction are consistent, and at this time, the dot product of the two unit vectors is 1. For a linear texture on the surface of the pipe, the gradient directions of the symmetric sampling points on both sides of the normal direction are opposite, and at this time, the dot product of the two unit vectors is -1.

[0069] Subtracting 1 from And dividing by 2 is to map the value range of the dot product from To the value range of gradient direction consistency When the dot product is 1, The first Pixel point is more likely to be located on the real contour, and when the dot product is -1, The first Pixel point is more likely to be located on the linear texture.

[0070] This relationship converts an intuitive physical phenomenon into an evaluation index, The value of which directly reflects the possibility of a pixel point being located on the real contour boundary, achieving preliminary differentiation of different continuity structures.

[0071] Wherein, when performing positive offset / negative offset, the offset distance, i.e. the distance from the first Pixel point sampling point And Is determined based on the following manner:

[0072] A plurality of standard pipe images are collected in advance, all surface textures caused by normal process are measured, and the maximum width in the normal direction of the main direction of the local structure is measured. Then, the offset distance is set to a value not less than the maximum texture width, and the offset distance is set to the maximum texture width plus 1 here. The scientific basis for this is to ensure that the two sampling points And Can cross the widest normal texture and really fall on both sides of the texture instead of inside the texture. If It is too small, for a relatively wide bright texture, the two sampling points may fall inside it, resulting in a wrong judgment of the gradient direction, and thus the strong texture is incorrectly identified as a boundary.

[0073] In summary, the core of this step lies in attribute identification based on physical models. It no longer focuses solely on whether the geometric shape of the structure is continuous, but delves into its underlying image physics features, providing a physical basis for distinguishing between real defect contours and continuous textures with similar shapes.

[0074] S4: Integrate the continuity of the path structure and the consistency of the gradient direction to generate a fusion saliency map.

[0075] After performing macroscopic and microscopic analyses on each pixel, we have obtained two independent and orthogonal attribute indicators for each pixel. Now, a final decision step is needed to combine the information from these two dimensions to generate a saliency map. In this map, only pixels that simultaneously satisfy macroscopic continuity and microscopic boundary characteristics should have the highest response value and thus be highlighted.

[0076] The saliency of a pixel is determined by combining the path structure continuity of each pixel with the consistency of the gradient directions on both sides of that pixel. This calculation is based on the following relationship:

[0077]

[0078] In this relation, It is the first The saliency of pixel fusion is a dimensionless final score; the higher the value, the greater the likelihood that the point is a true defect contour point. It is the first Path structure continuity of individual pixels It is the first The gradient directions on both sides of each pixel are consistent. The preset influence weighting coefficients are dimensionless and used for adjustment. right The extent of the impact As a trade-off parameter, its setting requires qualitative trade-off analysis: The value of determines the degree of importance attached to physical boundary properties. In practical applications, testing is conducted on a validation set containing various typical defects to select a value that maximizes the defect detection rate while minimizing the false alarm rate. The value, 1.5, is an empirical value that performs well in most scenarios; it will be used here. Set to 1.5.

[0079] In this relation, This is an enhancement weight term, a regulator based on physical boundary properties. When a point does not possess the characteristics of a true contour boundary, The weighting term is approximately equal to 1. This means that for those structures which are continuous but not real boundary (such as continuous scratch), their fusion saliency is basically equal to their path structure continuity, and will not be enhanced additionally; when a pixel has strong real contour boundary characteristics, This term becomes At this time This means that for those pixels which are both continuous and real boundary, their original path structure continuity will be significantly amplified by a coefficient greater than 1. The term is multiplied by a basic term and an enhancement weight term to provide a logical and operation. If a pixel's is very low (i.e. it does not lie on a continuous structure), then no matter how high its value is, the final will be close to zero, which ensures that only the pixels that pass the path structure continuity preliminary screening are qualified to enter the fusion saliency.

[0080] The formula realizes an intelligent weighting mechanism through the product of a basic term and an enhancement term, which gives priority to preserving the path structure continuity of all pixels, and gives greater enhancement weight to those pixels that are proven to have real physical boundary properties, so that in the final fusion saliency map, the real contour of the pipe is clearly separated from other linear textures.

[0081] Finally, after obtaining the fusion saliency of each pixel, the saliency map composed of the fusion saliency of all pixels is obtained. The saliency map and the image size of the pipe are consistent, the only difference is that the image of the pipe stores the gray value of each pixel, while the saliency map stores the fusion saliency of each pixel.

[0082] In summary, the core of this step is the multi-source information fusion decision, which simulates the way the human visual system recognizes objects, i.e. not only looking at whether the contour of the object is continuous, but also judging whether the contour meets the physical boundary characteristics. By logically fusing the geometric continuity and the physical boundary properties, this step generates the final fusion saliency map with extremely high signal-to-noise ratio, laying a solid foundation for subsequent accurate contour extraction.

[0083] S5: Perform double-threshold segmentation on the fusion saliency map to extract the edge contour.

[0084] In the fusion saliency map, only the pixel points satisfying both macro-continuity and local boundary characteristics have higher fusion saliency, so as to highlight the target contour and suppress various noises and textures. Therefore, for the generated fusion saliency map, a standard adaptive double-threshold segmentation algorithm is used for binaryzation processing to obtain all edge contours of the pipeline surface image. The adaptive double-threshold segmentation algorithm compares the fusion saliency threshold and the double threshold of each pixel point by traversing each pixel point, and marks each pixel point as a strong edge pixel point or a weak edge pixel point, and finally obtains an edge contour composed of strong edge pixel points and an edge contour composed of weak edge pixel points, which together constitute all edge contours of the pipeline surface image.

[0085] A specific adaptive double-threshold segmentation method is to count the mean value of the fusion saliency of all pixel points in the entire fusion saliency map and the standard deviation Then, based on the two statistics, the high and low thresholds are dynamically determined, according to the principle of 1.5 times the standard deviation, the high threshold , and the low threshold To ensure the non-negativity of the threshold, when the calculated is negative, it is corrected to 0, and then the double-threshold binaryzation segmentation is performed: all pixel points in the fusion saliency map with a fusion saliency greater than are marked as strong edge pixel points, and these pixel points have a higher possibility of being real contours, and all pixel points with a fusion saliency between and are marked as weak edge pixel points, which have a higher possibility of being potential linear defects, and all weak edge points are traversed, and only those weak edge points that can be connected to at least one strong edge point through 8-neighborhood are retained. Through this method, all edge contours of the complete and continuous pipeline are finally extracted, which are composed of all strong edge points and retained weak edge points.

[0086] This method determines the threshold by using the statistical characteristics of the fusion saliency map itself, and the double-threshold mechanism utilizes the spatial continuity of the contour, which can accurately locate the strong edge points and connect the broken parts through the weak edge points, so as to obtain more complete and continuous binaryzation edge contours while ensuring low false alarm rate.

[0087] S6: Defect recognition based on edge contour for real-time monitoring of pipeline processing.

[0088] After the foregoing steps, an accurate binaryzation edge contour has been obtained. However, this is only an intermediate result of image processing, and in order to achieve the final industrial application goal of real-time monitoring of whether the pipeline has defects, it is necessary to interpret, analyze and distinguish these pixel-level contours.

[0089] Real-time monitoring of the presence of defects in the pipeline based on the morphological features of all edge contours, including:

[0090] First step: using Hough transform or RANSAC (Random Sample Consensus) algorithm, linear fitting is performed on all edge contours to obtain a set of straight line segments. Since the main body of the pipeline is cylindrical, its contour in the 2D image mainly shows as parallel straight line segments. These algorithms can robustly identify and separate these straight lines from the contour point set, i.e. all straight lines belonging to the pipeline main body contour.

[0091] Traverse all possible combinations of the set of straight line segments, and according to the length, parallelism and spacing of the set of straight line segments, identify a pair of straight line segments that best match the physical characteristics of the pipeline main body contour, and determine them as the pipeline main body contour. The edge contours outside the pipeline main body contour are determined as potential defect contours, which usually correspond to linear defects such as cracks, depressions, scratches, etc. on the surface.

[0092] Second step: determining the defect recognition result of the pipeline based on the pipeline main body contour and the potential defect contour.

[0093] The defects of the pipeline mainly include structural defects and surface defects, the structural defects include: uneven thickness, deformation, etc., and the surface defects include: scratches, cracks, etc.

[0094] Therefore, for the pipeline main body contour, the average spacing and mutual parallelism between the straight line segments contained in the pipeline main body contour are calculated, and the tolerance range of the thickness of the pipeline and the tolerance range of the mutual parallelism of the main body contour are obtained. If the average spacing exceeds the tolerance range of the thickness, it is determined that the pipeline has uneven thickness defect, if it does not exceed, it is determined that the pipeline does not have uneven thickness defect, if the mutual parallelism exceeds the tolerance range of the mutual parallelism, it is determined that the pipeline has deformation defect, if it does not exceed, it is determined that the pipeline does not have deformation defect.

[0095] For the potential defect contour, the geometric features (such as curvature, length, position, etc.) of the potential defect contour are extracted, and the geometric features are matched with the pre-constructed defect feature library. If the matching is successful, the surface defect type corresponding to the potential defect contour is output, if the matching is not successful, the potential defect contour is not a surface defect.

[0096] This step upgrades the pixel-level edge information to structure and defect measurement with clear physical meaning and industrial standard through geometric fitting and parameter comparison, which completes the closed loop from defect recognition to abnormality determination, and realizes real-time, reliable and automatic monitoring of the processing quality of the pipeline.

[0097] The above merely provides the preferred embodiment of the present application, and is not used to limit the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the protection scope of the present application.

Claims

1. A method for real-time monitoring of stamping lines used in pipe processing, characterized in that, include: Obtain the surface image of any pipe in the stamping production line. For each pixel in the surface image, calculate the gradient magnitude and the main direction of the local structure of the pixel. Based on the main direction of the local structure of each pixel, generate a candidate path for the pixel. The tangent direction of each pixel passed through the candidate path is consistent with the main direction of the local structure of the pixel. Path integration is performed along the candidate path of each pixel. The path structure continuity of the pixel is calculated by accumulating the gradient magnitude of all pixels passed through the candidate path and the consistency of the unit vector of the principal direction of the local structure. Based on the normal direction of the principal direction of the local structure of each pixel, a bidirectional offset is performed to obtain two symmetrical sampling points. The consistency of the gradient direction on both sides of the pixel is determined by the dot product of the unit vectors of the gradient directions of the two symmetrical sampling points. By combining the path structure continuity of each pixel and the consistency of the gradient directions on both sides of the pixel, the fusion saliency of the pixel is determined to obtain the fusion saliency map composed of all pixels. Adaptive double threshold segmentation is performed on the fusion saliency map to obtain all edge contours of the pipeline. Based on the morphological features of all edge contours, the pipeline is monitored in real time for defects.

2. The real-time monitoring method for a stamping line according to claim 1, characterized in that, The gradient magnitude and principal direction of the local structure of a pixel are determined based on the following method: Gaussian filtering is applied to the surface image to reduce noise, generating a denoised image; the Sobel operator is used to calculate the gradient magnitude of each pixel in the denoised image. Within a preset neighborhood window of each pixel, a covariance matrix of gradient information is constructed; the eigenvector corresponding to the largest eigenvalue of the covariance matrix is ​​determined as the main local structural direction of that pixel.

3. The real-time monitoring method for a stamping line according to claim 1, characterized in that, Based on the main direction of the local structure of each pixel, candidate paths for that pixel are generated, including: The direction field formed by the main local structural directions of all pixels is regarded as a continuous vector field; the candidate path is defined as a solution of an ordinary differential equation, which constrains the tangent direction at any point on the path to be equal to the direction of that point in the vector field. Starting from each pixel, the ordinary differential equation is solved using numerical integration to generate a series of discrete path points to form the candidate path for that pixel.

4. The real-time monitoring method for a stamping line according to claim 1, characterized in that, The continuity of the path structure of pixels is determined based on the following relationship: ;in, It is the first Path structure continuity of individual pixels For the first The pixels traversed on the candidate path of the i-th pixel are related to the pixels of the i-th pixel. The arc length between pixels Let be the upper limit of the arc length for the path integral. For the integration interval, For candidate paths and the first The arc length between pixels is The gradient magnitude of the pixel at that location. For candidate paths and the first The arc length between pixels is The unit vector of the principal direction of the local structure of the pixel at that location. It is a tiny differential element of the arc length. For candidate paths and the first The arc length between pixels is The unit vector of the principal direction of the local structure of the pixel at that location. For the dot product operation of vectors, It is a preset exponential parameter.

5. The real-time monitoring method for a stamping line according to claim 1, characterized in that, The consistency of gradient directions on both sides of a pixel is determined based on the following relationship: ;in, For the first The gradient directions on both sides of each pixel are consistent. For the first The unit vector of the gradient direction of each sampling point is obtained by positively offsetting the normal direction of the principal direction of the local structure of each pixel. For the first The unit vector of the gradient direction of each sampling point is obtained by offsetting the normal direction of the principal direction of the local structure of each pixel in the opposite direction. This is the dot product operation for vectors.

6. The real-time monitoring method for a stamping line according to claim 1, characterized in that, The saliency of pixel fusion is determined based on the following relationship: ;in, For the first The saliency of pixel fusion For the first Path structure continuity of individual pixels For the first The gradient directions on both sides of each pixel are consistent. These are preset influence weighting coefficients used for adjustment. right The extent of the impact.

7. The real-time monitoring method for a stamping line according to claim 4, characterized in that, The upper limit of the arc length for path integrals is determined as follows: Pre-acquire images of multiple standard pipes that meet processing standards; For each pixel in each standard pipeline image, obtain the candidate path for that pixel, count the arc length of the candidate paths for all pixels in all standard pipeline images, sort the arc lengths of all candidate paths in ascending order, and then use the percentile method to determine the upper limit of the arc length for path integration.

8. The real-time monitoring method for a stamping line according to claim 5, characterized in that, The distances traveled by the forward and reverse offsets are determined based on the following method: Pre-acquire images of multiple standard pipes that meet processing standards; For each pixel in each standard pipeline image, the texture width in the normal direction of the local structure principal direction of that pixel is obtained, and the maximum texture width is determined based on all pixels of all standard pipeline images; the distances moved by the forward offset and the reverse offset are set to be no less than the maximum texture width.

9. The real-time monitoring method for a stamping line according to claim 1, characterized in that, Real-time monitoring of pipeline defects based on the morphological features of all edge contours, including: Using Hough transform or RANSAC algorithm, straight lines are fitted to all edge contours to obtain a set of straight line segments, and all possible combinations of the set of straight line segments are traversed. Based on the geometric characteristics such as the length, parallelism, and spacing of the straight line segments, the pair of straight line segments that best match the physical characteristics of the main body of the pipeline is identified and determined as the main body contour of the pipeline; all edge contours outside the main body contour of the pipeline are determined as potential defect contours. The defect identification results of the pipeline are determined based on the main pipeline profile and the potential defect profile.

10. The real-time monitoring method for a stamping line according to claim 9, characterized in that, The defect identification results of the pipeline are determined based on the main pipeline profile and the potential defect profile, including: For the main profile of the pipeline, the average spacing and parallelism between the straight segments contained in the main profile of the pipeline are calculated and compared with the pre-obtained tolerance range to determine whether there are structural defects in the pipeline. For potential defect profiles, the geometric features of the potential defect profiles are extracted, and the geometric features are matched with a pre-built defect feature library to determine whether there are surface defects in the pipeline.

Images