A pipe cutout recognition method and system based on machine vision
Patent Information
- Application Number
- CN202611159177.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-08-01
- Publication Date
- 2026-10-27
Smart Images
Figure CN122989614A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of image recognition technology, and discloses a method and system for pipe cut recognition based on machine vision. Background Technology
[0002] Existing machine vision recognition solutions for pipe cuts use fixed point or area light sources for illumination and industrial cameras to capture two-dimensional images of the pipe cuts. In the image processing stage, current technologies typically input the captured two-dimensional images directly into conventional convolutional neural network models, or use edge detection operators based on pixel gradient calculations to extract features in a Cartesian coordinate system. Conventional convolutional network models contain multiple layers of two-dimensional convolutional and pooling layers. By sliding the convolutional kernels in the horizontal and vertical directions of the Cartesian coordinate system, they capture local textures and gray-level transitions in the image, thereby outputting the contour segmentation results of the pipe cuts.
[0003] The cut surface of a metal pipe exhibits specular reflection, and the interior of the pipe contains deep holes. In the existing technology that uses a fixed light source for illumination and a conventional convolutional network for feature extraction, light at a fixed angle creates locally highly reflective overexposed areas on the pipe cut surface, while simultaneously creating underexposed shadow areas inside the pipe. These highly reflective and shadow areas manifest as extreme distortions in grayscale values in a two-dimensional image, directly causing a break in the pixel-level gradient information at the cut edge. Conventional convolutional networks rely on local neighboring pixels to calculate gradients in a Cartesian coordinate system. When highly reflective or shadow areas disrupt the continuity of edge pixels, the network cannot establish long-distance edge dependencies across the grayscale distortion areas, resulting in fragmented extracted edge features and an inability to generate a topologically continuous cut contour. Summary of the Invention
[0004] The purpose of this invention is to provide a solution that can effectively address the problems described in the background section.
[0005] To achieve the above objectives, the technical solution adopted by the present invention is as follows: A machine vision-based method for pipe cut recognition includes: controlling a multi-angle ring light source array to alternately trigger light source combinations of different phases according to a preset time sequence, and acquiring a multi-frame original image sequence of the pipe cut; The original image sequences of multiple frames are transformed to the frequency domain by Fourier transform. The frequency domain coefficients of each frame of the original image sequences are weighted by a preset high reflectivity band suppression mask and shadow band compensation mask, and then multi-frame illumination equalization images are generated by inverse Fourier transform. The multi-frame illumination equalization image is input into a dual-branch feature extraction network. The first branch performs polar coordinate transformation on the multi-frame illumination equalization image and uses one-dimensional convolution to extract continuous edge features along the circumferential direction. The second branch uses dilated spatial convolution pooling pyramid in Cartesian coordinate system to extract local geometric abrupt features. The continuous edge features and the local geometric mutation features are dimensionally aligned and adaptively weighted and fused through the feature cross-attention mechanism, and the fused features are then passed through the pixel-level binarization mask of the output pipeline of the fully convolutional decoder.
[0006] Preferably, the multi-angle ring light source array includes multiple independent light source partitions evenly distributed in a circle, and the preset timing alternately triggers light source combinations of different phases, including: dividing the multiple independent light source partitions into non-overlapping odd-numbered sequence light source partitions and even-numbered sequence light source partitions. In the first exposure period, only the odd-numbered sequence light source partitions are illuminated, and in the second exposure period, only the even-numbered sequence light source partitions are illuminated, and so on, the multi-frame original image sequence is acquired alternately and cyclically. The method for generating the high reflectivity frequency band suppression mask includes: calculating the average amplitude spectrum of the multi-frame original image sequence in the frequency domain, dividing the coordinates of frequency points in the average amplitude spectrum whose amplitude exceeds a preset dynamic threshold into high reflectivity frequency bands, and setting the frequency domain coefficients corresponding to the high reflectivity frequency bands to zero to generate the high reflectivity frequency band suppression mask.
[0007] Preferably, the process of constructing the shadow band compensation mask includes: performing polar coordinate unfolding transformation on the multi-frame original image sequence, and calculating the mean gray-level gradient of the unfolded image in the radial direction; Image regions whose mean gray-level gradient in the radial direction is lower than the preset dark area gradient are marked as shadow candidate regions, and the shadow candidate regions are mapped back to the frequency domain space to extract the corresponding low-frequency shadow bands. Extract the average frequency domain coefficients of the adjacent high-frequency bands of the low-frequency shadow band, and use the average frequency domain coefficients to perform linear interpolation compensation on the frequency domain coefficients of the low-frequency shadow band. Mark the compensated frequency domain space as the shadow band compensation mask.
[0008] Preferably, the first branch performs polar coordinate transformation on the multi-frame illumination equalization image and uses one-dimensional convolution to extract continuous edge features along the circumferential direction, including: taking the physical center of the pipe cut as the origin of polar coordinates, mapping the multi-frame illumination equalization image from the Cartesian coordinate system to the polar coordinate system, and generating a polar coordinate expansion matrix with angle as the horizontal axis and radial distance as the vertical axis. A one-dimensional causal convolution kernel with a preset receptive field size is applied along the horizontal axis of the polar coordinate unfolded matrix. The one-dimensional causal convolution kernel slides unidirectionally along the horizontal axis to extract the gray-level jump sequence between adjacent pixels in the horizontal axis as the continuous edge features along the circumferential direction.
[0009] Preferably, the local geometric abrupt change features are extracted by using a dilated spatial convolution pooling pyramid in the Cartesian coordinate system through the second branch, including: in the Cartesian coordinate system, parallel multi-branch dilated convolutional layers perform convolution sampling on the multi-frame illumination equalization images with different dilation rates. The specific values of the different void ratios are adaptively configured according to the nominal wall thickness parameter of the pipeline to be identified, so that the receptive field size corresponding to the largest void ratio in the multi-branch void convolutional layer is equivalent to the pixel span occupied by the nominal wall thickness parameter in the image. The output feature maps of the multi-branch dilated convolutional layer are concatenated by channels and then generated by global average pooling to produce the local geometric mutation features.
[0010] Preferably, the continuous edge features and the local geometric mutation features are dimensionally aligned and adaptively weighted and fused through a feature cross-attention mechanism, including: using the local geometric mutation features as a query matrix and the continuous edge features as both a key matrix and a value matrix; Calculate the product of the query matrix and the transpose of the key matrix to obtain the spatial similarity matrix across coordinate systems; Normalize each row of the spatial similarity matrix to generate an adaptive weight map. Multiply the adaptive weight map with the value matrix to recalibrate the local geometric abrupt change features according to the spatial location distribution of the continuous edge features, and output the fused features.
[0011] Preferably, after generating the shadow band compensation mask, the method further includes performing a time-series dynamic update of the shadow band compensation mask: extracting three initial shadow band compensation masks corresponding to three consecutive frames of illumination equalization images, calculating the first difference region between the current frame's initial shadow band compensation mask and the previous frame's initial shadow band compensation mask, and calculating the second difference region between the current frame's initial shadow band compensation mask and the next frame's initial shadow band compensation mask; Perform an intersection operation on the first difference region and the second difference region, and remove the non-static shadow frequency bands caused by dynamic drifting objects in the intersection. The removed mask is superimposed on the initial shadow band compensation mask of the current frame to generate the final shadow band compensation mask after time-updated sequence.
[0012] Preferably, after generating the polar coordinate unfolded matrix and before applying the one-dimensional causal convolution kernel, the method further includes injecting radial position encoding into the polar coordinate unfolded matrix: obtaining the radial distance value corresponding to each pixel in the polar coordinate unfolded matrix, and mapping the radial distance value to a fixed-dimensional radial position vector through a sine position encoding function; The radial position vector and the image feature vector of the polar coordinate expansion matrix are concatenated in the channel dimension to generate an enhanced polar coordinate matrix carrying radial depth information; The one-dimensional causal convolution kernel specifically acts on the enhanced polar coordinate matrix, so that the extracted continuous edge features along the circumferential direction include radial depth position attributes.
[0013] Preferably, after outputting the fused features and before outputting them via the fully convolutional decoder, the method further includes applying a priori boundary constraint of the physical cutting trajectory to the fused features: generating a standard tangent direction field of the ideal circular trajectory in the image coordinate system based on the preset cutting trajectory parameters of the pipe cutting device; Calculate the predicted gradient direction corresponding to each feature point in the fused features, and calculate the cosine value of the angle between the predicted gradient direction and the standard tangent direction at the corresponding position in the standard tangent direction field; The negative of the cosine of the included angle is used as a boundary compliance penalty term. This boundary compliance penalty term is superimposed on the total loss function of the network, and backpropagation is used to constrain the pixel-level binary mask output by the fully convolutional decoder to fit the standard tangent direction field.
[0014] Preferably, it includes: a multi-angle ring light source array, used to alternately trigger light source combinations of different phases according to a preset timing sequence; An image acquisition device, synchronized with the multi-angle ring light source array, is used to acquire a multi-frame original image sequence of the pipe cut. A frequency domain illumination modulation device is used to convert the multi-frame original image sequence to the frequency domain by performing Fourier transform on each frame, and to perform weighted processing on the frequency domain coefficients of each frame of the multi-frame original image sequence by using a preset high reflectivity frequency band suppression mask and shadow frequency band compensation mask, and to generate multi-frame illumination equalization images by inverse Fourier transform. The dual-branch feature extraction and fusion device is used to perform polar coordinate transformation on the multi-frame illumination equalization image through the first branch and extract continuous edge features along the circumferential direction using one-dimensional convolution; and to extract local geometric abrupt features in the Cartesian coordinate system using dilated spatial convolution pooling pyramid through the second branch. The continuous edge features and the local geometric abrupt features are then dimensionally aligned and adaptively weighted and fused through a feature cross attention mechanism. A decoding output device for transmitting pixel-level binarization masks of fused features through the output pipeline of a fully convolutional decoder.
[0015] Compared with the prior art, the beneficial effects of the present invention are as follows: 1. This invention acquires multiple frames of original image sequences by controlling a multi-angle ring light source array to alternately trigger light source combinations of different phases. In the frequency domain, a high-reflectivity frequency band suppression mask and a shadow frequency band compensation mask are used to weight the frequency domain coefficients of each frame, eliminating extreme gray-level distortion in a single frame image. The first branch of the dual-branch feature extraction network transforms the illumination-equalized image to a polar coordinate system and uses one-dimensional convolution to extract continuous edge features along the circumferential direction, transforming the identification of the cut contour from local pixel gradient calculation to global sequence feature extraction along the circumferential direction, thus bridging the gray-level break areas caused by high reflectivity or shadows in the original image. The second branch extracts local geometric abrupt change features in the Cartesian coordinate system using dilated spatial convolution. Combined with a feature cross-attention mechanism, the features of the two branches are dimensionally aligned and adaptively weighted and fused, ensuring that the fused features simultaneously contain global circumferential continuity and local geometric accuracy. A topologically complete pipe cut mask is output through a fully convolutional decoder.
[0016] 2. By dividing the multi-angle ring light source into odd-sequence and even-sequence light source zones and illuminating them alternately, and constructing a high-reflectivity frequency band suppression mask in conjunction with the coordinates of frequency points exceeding a preset dynamic threshold in the frequency domain average amplitude spectrum, fixed reflection interference caused by specific lighting angles is eliminated. By calculating the mean gray-level gradient in the radial direction of the polar coordinate expansion matrix to define the shadow candidate region and performing linear interpolation compensation on the low-frequency shadow band, the edge details inside the cavity are restored. In the feature fusion stage, local geometric abrupt features in the Cartesian coordinate system are used as the query matrix, and continuous edge features in the polar coordinate system are used as the key matrix and value matrix to calculate the cross-coordinate spatial similarity matrix. This achieves feature recalibration based on the spatial distribution of continuous edges on the circumference, avoiding feature confusion caused by directly splicing features from different coordinate systems. Attached Figure Description
[0017] Figure 1 This is a flowchart of the overall method for pipe cut recognition based on machine vision according to the present invention; Figure 2 This is a flowchart of the multi-angle light source timing triggering and image acquisition process of the present invention; Figure 3 This is a flowchart illustrating the construction process of the high-reflectivity frequency band suppression mask of the present invention; Figure 4 This is a flowchart illustrating the construction process of the shadow band compensation mask in this invention. Figure 5 This is a flowchart of the dual-branch feature extraction network processing of the present invention; Figure 6 This is a flowchart illustrating the cross-attention mechanism fusion process of the present invention. Detailed Implementation
[0018] Please refer to Figure 1 This embodiment provides a machine vision-based pipe cut recognition system, including a multi-angle ring light source array, an image acquisition device, a frequency domain illumination modulation device, a dual-branch feature extraction and fusion device, and a decoding output device. The multi-angle ring light source array is communicatively connected to the image acquisition device, and the frequency domain illumination modulation device, the dual-branch feature extraction and fusion device, and the decoding output device are communicatively connected sequentially. The multi-angle ring light source array is configured as multiple independent light source partitions evenly distributed around the circumference. Specifically, an even number of light source emission nodes are arranged at equal intervals around the central axis of the pipe on the array's circuit board, and each light source emission node is controlled by an independent constant current drive circuit. The image acquisition device is configured as an industrial area scan camera, and the optical axis of its lens coincides with the physical center normal of the pipe cut. During system operation, the multi-angle ring light source array is controlled to alternately trigger light source combinations of different phases according to a preset timing sequence. The image acquisition device maintains a hardware-level synchronization signal connection with the multi-angle ring light source array, and acquires multiple frames of original image sequences of the pipe cut at the instant the light source combination is triggered. A multi-frame original image sequence is transmitted to a frequency-domain illumination modulation device. This device performs Fourier transforms on each frame of the original image sequence to the frequency domain. It then uses a preset high-reflectivity frequency band suppression mask and a shadow frequency band compensation mask to weight the frequency domain coefficients of each frame of the original image sequence, and generates multi-frame illumination-equalized images through inverse Fourier transform. These multi-frame illumination-equalized images are input to a dual-branch feature extraction and fusion device. This device performs polar coordinate transformation on the multi-frame illumination-equalized images through the first branch and extracts continuous edge features along the circumferential direction using one-dimensional convolution. The second branch extracts local geometric abrupt features in the Cartesian coordinate system using dilated spatial convolution pooling pyramids. Subsequently, a feature cross-attention mechanism is used to dimensionally align and adaptively weight and fuse the continuous edge features and local geometric abrupt features. The decoding output device passes the fused features through a pixel-level binarization mask cut into the output pipeline of a fully convolutional decoder.
[0019] Specifically, the multiple independent light source sections of the multi-angle ring light source array are divided into non-overlapping odd-numbered sequence light source sections and even-numbered sequence light source sections. All light source nodes on the ring array are arranged in a clockwise direction from... arrive Perform index numbering, where Even numbers. Odd-numbered sequence light source partitions contain a set of nodes with odd indices. Even-numbered sequence light source partitions contain a set of nodes with even indices. During the first exposure period, the system controller sends... The constant current drive circuit within the set sends a high-level trigger signal to... The system sends a low-level shutdown signal, at which point only the odd-numbered sequence light source zones are illuminated; during the second exposure period, the system controller sends... The collection sends a high-level trigger signal to... The system sends low-level off signals alternately to acquire a multi-frame raw image sequence. In this alternating timing control, a preset dark interval exists between adjacent exposure periods. The duration of this dark interval is greater than the sum of the turn-off response time of the light source drive circuit and the shutter reset time of the image acquisition device. Through this alternating odd-even lighting method, the incident angle of illumination on the metal mirror reflection point at the same physical location shifts discretely in different frames. This causes a spatial displacement of the highly reflective overexposed area at a fixed location between frames, providing multi-angle illumination redundancy information for subsequent frequency domain processing. Please refer to [reference needed]. Figures 1 to 2 .
[0020] In one embodiment, the high-reflectivity band suppression mask is generated by performing statistical operations on a multi-frame sequence of original images in the frequency domain. For the acquired... Original image sequence of frames ,in Indicates the first Frame image in Cartesian coordinate system The grayscale value at each position is used to perform a two-dimensional discrete Fourier transform on each frame of the image. The formula for the two-dimensional discrete Fourier transform is:
[0021] in, and These are the width and height in pixels, respectively. and For frequency domain coordinates, The unit is the imaginary unit. This is used when obtaining the frequency domain coefficients for each frame. Then, the average amplitude spectrum of the multi-frame original image sequence in the frequency domain is calculated:
[0022] In the average amplitude spectrum, the highly reflective region formed by specular reflection from the cut surface of the metal pipe exhibits extreme energy concentration at specific spatial frequencies. To accurately locate these reflective frequency bands, a preset dynamic threshold is set. This dynamic threshold is not a fixed constant but is calculated based on local statistical characteristics in the frequency domain. Specifically, in the frequency domain coordinates... Local neighborhood window Calculate the local mean. With local standard deviation Dynamic threshold The calculation formula is:
[0023] in, The preset sensitivity coefficient has a range of values. to Average amplitude spectrum The amplitude exceeds the preset dynamic threshold. The frequency coordinates were divided into high-reflectivity bands. Subsequently, a high-reflectivity band suppression mask was constructed. The mask is a binary matrix with the same size as the frequency domain coefficient matrix, and its generation logic is as follows:
[0024] When weighting the frequency domain coefficients of each frame, the frequency domain coefficients corresponding to the high reflectivity band are directly set to zero, i.e. This operation is physically equivalent to filtering out directional high-frequency interference fringes caused by strong light incident at a fixed angle in the frequency domain. It prevents these high-energy frequency bands from superimposing into extremely high grayscale values in the spatial domain after the inverse Fourier transform. Please refer to [reference needed]. Figure 3 .
[0025] Furthermore, the construction process of the shadow frequency band compensation mask involves a cross-domain mapping from shadow localization in the spatial domain to low-frequency compensation in the frequency domain. Inside the metal pipe cavity, underexposed areas are formed due to the inability of light to penetrate directly. These areas appear as large low-grayscale connected regions in the image, with extremely smooth edge grayscale transitions. Polar coordinate expansion transformation is performed on single frames from a multi-frame original image sequence. The physical center of the pipe cut is used as the coordinates. As the origin of polar coordinates, spatial domain coordinates to polar coordinates The mapping relationship is as follows:
[0026]
[0027] Image in Cartesian coordinate system Resampling as an expansion matrix in polar coordinates The horizontal axis represents the angle. The vertical axis represents the radial distance. In the polar coordinate unfolded matrix, the shadowed area inside the cavity appears as a vertical dark band extending radially. Calculate the mean gray-level gradient of the unfolded image in the radial direction. For a certain angle... The corresponding radial gray-level gradient mean The calculation formula is:
[0028] in, This represents the maximum radial pixel span of the polar coordinate expansion matrix. Due to extremely low grayscale values and lack of texture, the mean radial grayscale gradient of the shadow region is much lower than the texture gradient of a normal metal surface. Set the preset dark area gradient. Image regions with a mean gray-level gradient in the radial direction below a certain threshold are marked as shadow candidate regions. The set of angles that meet the condition is as follows: The candidate shadow region is mapped back to the frequency domain. Since the shadow appears as a large, slowly varying area in the spatial domain, it corresponds to a low-frequency shadow band in the frequency domain. The mean frequency domain coefficients of the adjacent high-frequency bands of this low-frequency shadow band are extracted. Specifically, the radial frequency boundary of the low-frequency shadow band is determined in the frequency domain. Take the high-frequency band interval immediately adjacent to this boundary. , The mean amplitude of the frequency domain coefficients within The mean frequency domain coefficients of the high-frequency band are used to perform linear interpolation compensation on the frequency domain coefficients of the low-frequency shadow band. Let a certain frequency point within the low-frequency shadow band be... The original coefficients are The coefficients after linear interpolation compensation for:
[0029] in, To compensate for the strength coefficient, For frequency point Euclidean distance to the center of the frequency domain The frequency domain radius is used. The compensated frequency domain space is marked as the shadow band compensation mask. During the weighted processing stage, the final output of the frequency domain coefficients is: The image is then transformed using an inverse Fourier transform to generate multiple frames of illumination-equalized images. This compensation mechanism utilizes the energy of adjacent high-frequency bands in the frequency domain to directionally fill the low-frequency energy loss caused by shadows, enabling the inverse-transformed image to recover weak edge contrast in the shadowed region of the cavity. This overcomes the problem of invisible edge information caused by gray-scale collapse in the shadowed region in a single-frame image. Please refer to [reference needed]. Figure 4 .
[0030] In one embodiment, due to dynamic interference such as cutting sparks and dust flow in actual industrial environments, these dynamic objects will also form motion shadows in the image. To prevent dynamic shadows from being mistakenly treated as static shadows and overcompensated, the shadow frequency band compensation mask is dynamically updated in time after its generation. A sliding window buffer queue containing three consecutive frames of illumination equalization images is established, and three initial shadow frequency band compensation masks corresponding to the current frame, the previous frame, and the next frame of illumination equalization images are extracted from this queue, denoted as . Calculate the first difference region between the initial shadow band compensation mask of the current frame and the initial shadow band compensation mask of the previous frame. The calculation method is a pixel-level logical XOR operation:
[0031] Similarly, calculate the second difference region between the initial shadow frequency band compensation mask of the current frame and the initial shadow frequency band compensation mask of the next frame. :
[0032] The difference region represents the set of pixels where the mask changes over time. For shadows caused by dynamic drifting objects, since their positions randomly shift over three consecutive frames, these shadow pixels must exist simultaneously in both the first and second difference regions. By performing an intersection operation on the first and second difference regions, the non-static shadow frequency bands caused by dynamic drifting objects are extracted. :
[0033] Remove pixels from the intersection of the initial shadow band compensation mask of the current frame to generate a mask after removing dynamic interference. :
[0034] The removed mask is superimposed on the initial shadow frequency band compensation mask of the current frame to generate the final shadow frequency band compensation mask after temporal update. This temporal dynamic update mechanism effectively separates static structural shadows from dynamic drifting object interference by examining the pixel-level consistency of the mask on the time axis. This ensures that the frequency domain compensation operation only applies to the real cavity geometric shadows, avoiding the amplification of dynamic noise and its introduction into the subsequent feature extraction network.
[0035] Furthermore, the multi-frame illumination equalization images are input into a two-branch feature extraction network. The first branch is configured to extract continuous edge features along the circumferential direction. Geometrically, the pipe cutout presents as a closed ring or quasi-ring structure, which is represented as continuous straight lines along the horizontal axis in the polar coordinate unfolded image. The first branch performs a polar coordinate transformation on the multi-frame illumination equalization images, using the physical center of the pipe cutout as the origin, mapping the multi-frame illumination equalization images from the Cartesian coordinate system to the polar coordinate system, generating a polar coordinate unfolded matrix with angle as the horizontal axis and radial distance as the vertical axis. During the mapping process, due to the misalignment between the Cartesian coordinate grid and the polar coordinate grid, a bilinear interpolation algorithm is used to calculate the grayscale values at the polar coordinate grid nodes. For any point in the polar coordinate space... Its reverse mapping to the Cartesian coordinate system is: Find its four nearest integer coordinates in the Cartesian grid. Calculate based on the gray values and distance weights of the four points. The interpolated grayscale values at the specified locations are used to generate a hole-free polar coordinate expansion matrix. Please refer to Figure 5 .
[0036] In one embodiment, radial position encoding is injected into the polar coordinate unfolded matrix after generation and before applying a one-dimensional causal convolution kernel. Because the polar coordinate transformation alters the spatial adjacency relationships between pixels, the network loses its absolute positional awareness of radial depth from the original image when processing the polar coordinate matrix. The radial distance value corresponding to each pixel in the polar coordinate unfolded matrix is then obtained. The radial distance values are mapped to fixed-dimensional radial position vectors using a sinusoidal position encoding function. Let the dimension of the position vector be... For the first dimension radial position vector The calculation formula is:
[0037]
[0038] This sinusoidal position encoding function maps a one-dimensional radial distance scalar to a unique trajectory in a high-dimensional space, where pixels at different radial distances obtain different position vectors. The generated radial position vectors are then concatenated with the image feature vectors of the polar coordinate expansion matrix along the channel dimension. Specifically, assuming the image feature vector after the initial convolutional layer has a polar coordinate expansion matrix, the dimension is... The radial position vector dimension is The dimension of the concatenated enhanced polar coordinate matrix in the channel dimension is... After generating an enhanced polar coordinate matrix carrying radial depth information, the one-dimensional causal convolution kernel acts specifically on this enhanced polar coordinate matrix. Causal convolution means that when the convolution kernel slides unidirectionally along the horizontal axis (angular direction), the output of the current pixel depends only on the current pixel and its predecessor pixels in the angular history direction, and does not depend on subsequent pixels. Let the size of the one-dimensional causal convolution kernel be... Its angle The convolution operation at point is:
[0039] in, For convolution kernel weights, To enhance the polar coordinate matrix in terms of angle The feature vector at the location is obtained. This one-dimensional causal convolution kernel slides unidirectionally along the horizontal axis, extracting the gray-level jump sequence between adjacent pixels along the horizontal axis as continuous edge features along the circumferential direction. Since the convolution operation has translation invariance in the horizontal axis of polar coordinates and is superimposed with radial position encoding, the extracted continuous edge features along the circumferential direction not only contain continuous topological information of the cut edges, but also contain radial depth position attributes. Even if there are gray-level break areas in the original image caused by local high reflectivity or shadows, the one-dimensional causal convolution kernel can rely on the contextual information within its preset receptive field size to establish long-distance edge dependencies across gray-level distortion regions, connecting fragmented edges into complete sequence features.
[0040] In the second branch, local geometric abrupt changes are extracted in a Cartesian coordinate system. Due to the physical cutting action of the cutting tool, pipe cuts leave tiny geometric steps, chipped edges, or burrs at the cut edges; these local details manifest as high-frequency geometric abrupt changes in the image. The second branch extracts these local geometric abrupt changes using dilated spatial convolution pooling pyramids in a Cartesian coordinate system. In the Cartesian coordinate system, parallel multi-branch dilated convolutional layers perform convolution sampling on multiple frames of illumination-equalized images with different dilation rates. Dilated convolution expands the receptive field without increasing the number of parameters or computational cost by injecting holes between adjacent elements of the standard convolutional kernel. Let the standard convolutional kernel size be... The void ratio is The actual equivalent convolution kernel size is... for:
[0041] The specific values for different void ratios are adaptively configured based on the nominal wall thickness parameters of the pipe to be identified. Nominal wall thickness parameters (Unit: millimeters) Converted to pixel span in the image using the camera's calibration matrix. Configure the maximum void ratio in a multi-branch dilated convolutional layer. This ensures that the corresponding receptive field size is equivalent to the pixel span occupied by the nominal wall thickness parameter in the image, i.e., satisfies... The maximum void ratio can be calculated by working backwards from this constraint. The configuration value is:
[0042] The void ratio of the remaining branches is configured to decrease in a geometric progression, for example, set to... This adaptive configuration ensures that the receptive field of the dilated spatial convolutional pooling pyramid precisely matches the physical size of the pipe cut, guaranteeing that the network can fully cover the geometrically abrupt regions of the cut edge when extracting features without introducing excessive background noise. The output feature maps of the multi-branch dilated convolutional layers are then concatenated via channels. Assuming there are... There are parallel branches, and the number of channels in the output feature map of each branch is . The number of channels in the concatenated feature map is then... The concatenated feature maps are then processed through a global average pooling layer to compress them spatially, resulting in a matrix with dimensions of [dimensionality missing]. The feature vector of local geometric mutations. This feature vector aggregates the local geometric details of the incision at different receptive field scales, and physically represents the roughness and mutation intensity distribution of the incision edge at multiple scales.
[0043] In one embodiment, a feature cross-attention mechanism is used to perform dimension alignment and adaptive weighted fusion of continuous edge features and local geometrical aberration features. Since the continuous edge features output by the first branch are in polar coordinate space and are two-dimensional matrices, while the local geometrical aberration features output by the second branch are global vectors after global average pooling, there is a significant misalignment between the two in terms of spatial dimension and coordinate system. The local geometrical aberration features are then used as the query matrix. Continuous edge features are simultaneously used as the key matrix Sum matrix Before performing matrix operations, dimension alignment is first performed. The feature vectors of local geometric abrupt changes are projected through a fully connected layer into a query matrix that is consistent with the continuous edge feature matrix in the channel dimension. ,in This represents the total number of pixels in the polar coordinate expansion matrix. This represents the number of feature channels. Simultaneously, the continuous edge feature matrix is flattened and projected into a key matrix. AND-value matrix Calculate the product of the query matrix and the transpose of the key matrix to obtain the spatial similarity matrix across coordinate systems. :
[0044] Elements in the spatial similarity matrix Describes the first in the Cartesian coordinate system The local geometrical abrupt change features and the first local geometrical change in polar coordinates The correlation strength of continuous edge features in the feature space. Normalization is performed on each row of the spatial similarity matrix, and an adaptive weight map is generated using the Softmax function. :
[0045] in, The channel dimension of the key matrix is used to control gradient stability. Multiplying the adaptive weight map by the value matrix recalibrates local geometrical abrupt changes according to the spatial distribution of continuous edge features, outputting fused features. :
[0046] In this operation, local geometric mutation features (query matrix) actively address topologically continuous edge features (key matrix) in polar coordinates. The adaptive weight graph determines which local geometric mutation details should be enhanced and assigned to which circumferentially continuous positions. This mechanism avoids feature confusion and semantic conflicts caused by directly splicing features from different coordinate systems along the channel dimension, achieving precise injection of local geometric accuracy into global circumferential continuity.
[0047] Furthermore, after outputting the fused features and before outputting them via the fully convolutional decoder, a priori boundary constraint of the physical cutting trajectory is applied to the fused features. In industrial pipe cutting, the cutting tool's trajectory is ideally a standard circle. Due to noise or occlusion in the image, the cut mask edge predicted by the fully convolutional decoder may exhibit local convexities or depressions, deviating from the physical cutting trajectory. Based on the preset cutting trajectory parameters of the pipe cutting equipment, a standard tangent direction field of the ideal circular trajectory in the image coordinate system is generated. Let the center coordinates of the ideal circular trajectory be... For any coordinate point in the image coordinate system The standard tangent direction vector of this point on the ideal circular trajectory. for:
[0048] Calculate the predicted gradient direction for each feature point in the fused feature map. This predicted gradient direction is obtained by applying a 3D Sobel operator to the fused feature map for each point. The direction in which the gradient magnitude is largest in the channel dimension is defined as the predicted gradient direction. Calculate the cosine of the angle between the predicted gradient direction and the standard tangent direction at the corresponding position in the field:
[0049] The negative of the cosine of the included angle is used as the boundary compliance penalty. Accumulate at the pixel level:
[0050] The boundary compliance penalty term is added to the total network loss function. Due to binary cross-entropy loss , loss Penalties for Boundary Compliance Composed of weighted combinations:
[0051] in, These are preset weight coefficients. Through backpropagation, the gradients generated by the boundary compliance penalty term flow back to the fully convolutional decoder, forcibly constraining the gradient directions of edge pixels in the pixel-level binary mask output by the decoder to conform to the standard tangent direction field. This prior constraint defines the geometry of the mask edges from a physical kinematic perspective, eliminating false edge interference caused by non-cutting trajectories.
[0052] In one embodiment, the fully convolutional decoder receives fused features and outputs a pixel-level binarized mask of the pipe cutouts. The fully convolutional decoder contains multiple stages of transposed convolutional layers, which progressively restore the spatial resolution of the feature map by inserting zeros between feature map elements and performing standard convolution operations. After each stage of transposed convolution, skip connection features from the bi-branch feature extraction network at the corresponding resolution are concatenated to compensate for spatial details lost during pooling or downsampling. The final output layer of the fully convolutional decoder employs a single-channel convolution, mapping the feature dimensions to... Then, each pixel value in the output feature map is input into the Sigmoid activation function, which maps it to... The probability value of the interval. The formula for the Sigmoid activation function is:
[0053] in, For single-channel convolution in coordinates The output value at that location, This represents the probability that the pixel belongs to the edge of the pipe cut. A binarization threshold is set. (The range of values is) to Thresholding is applied to the probability map to generate the final pixel-level binarized mask. :
[0054] This binarized mask accurately outlines the continuous closed contour of the pipe cut, which can be directly used in subsequent robot guidance or dimensional measurement processes.
[0055] Existing technologies use fixed point or area light sources for illumination. Light at a fixed angle creates locally highly reflective, overexposed areas on the surface of the pipe cut, while simultaneously creating underexposed, shadowed areas inside the pipe cavity. These highly reflective and shadowed areas manifest as extreme grayscale distortion in a two-dimensional image, directly causing a break in pixel-level gradient information at the cut edge. Conventional convolutional networks rely on local neighboring pixels to calculate gradients in Cartesian coordinates. When highly reflective or shadowed areas disrupt the continuity of edge pixels, the network cannot establish long-distance edge dependencies across the grayscale distortion areas, resulting in fragmented extracted edge features. This embodiment addresses this by controlling a multi-angle ring light array, dividing it into odd-sequence and even-sequence light source zones that are alternately illuminated, thus breaking down the fixed reflective path at the physical light source level. In the frequency domain processing stage, a high-reflective frequency band suppression mask is constructed by calculating the coordinates of frequency points exceeding a preset dynamic threshold in the frequency domain average amplitude spectrum. The corresponding frequency domain coefficients are then set to zero, eliminating the reconstruction of fixed reflective interference caused by specific lighting angles in the spatial domain. By calculating the mean gray-level gradient in the radial direction of the polar coordinate expansion matrix to define the candidate shadow region, and mapping this candidate region back to the frequency domain to extract the low-frequency shadow band, linear interpolation compensation is performed on the low-frequency shadow band using the mean frequency domain coefficients of adjacent high-frequency bands. This restores the edge details inside the cavity at the frequency domain energy level, thus eliminating extreme gray-level distortion in the illumination-equalized image generated by the inverse transform. Please refer to [reference needed]. Figure 6 .
[0056] In the feature extraction stage, the first branch of this embodiment transforms the illumination-equalized image to a polar coordinate system. By injecting radial position encoding with a sinusoidal function mapping and using a one-dimensional causal convolution kernel sliding along the horizontal axis, the recognition of the cut contour is transformed from local pixel gradient calculation to global sequence feature extraction along the circumferential direction. This allows the extracted continuous edge features along the circumferential direction to carry radial depth position attributes, directly bridging the grayscale break areas caused by high reflectivity or shadows in the original image. The second branch, in the Cartesian coordinate system, uses a dilated spatial convolution pooling pyramid with an adaptively configured void ratio based on the pipe's nominal wall thickness parameter to extract local geometric abrupt features that precisely match the receptive field and physical cut size. In the feature fusion stage, the local geometric abrupt features in the Cartesian coordinate system are used as the query matrix, and the continuous edge features in the polar coordinate system are used as the key and value matrices. A cross-coordinate system spatial similarity matrix is calculated and normalized to generate an adaptive weight map. The local geometric abrupt features are recalibrated according to the spatial position distribution of the continuous edge features, achieving precise injection of local geometric accuracy into global circumferential continuity and avoiding feature confusion caused by direct splicing of features from different coordinate systems. Before the output of the fully convolutional decoder, a standard tangent direction field based on the cutting trajectory parameters is introduced. The cosine value of the angle between the predicted gradient direction and the standard tangent direction is calculated and superimposed on the total loss function as a penalty term. The pixel-level binary mask output is constrained to fit the physical cutting trajectory through backpropagation, eliminating the false edge interference caused by non-cutting trajectories, and finally outputting a topologically complete pipe cut mask.
Claims
1. A method for pipe cut recognition based on machine vision, characterized in that, include: The multi-angle ring light source array is controlled to alternately trigger light source combinations of different phases according to a preset timing sequence to acquire multiple frames of original image sequences of the pipe cut. The original image sequences of multiple frames are transformed to the frequency domain by Fourier transform. The frequency domain coefficients of each frame of the original image sequences are weighted by a preset high reflectivity band suppression mask and shadow band compensation mask, and then multi-frame illumination equalization images are generated by inverse Fourier transform. The multi-frame illumination equalization image is input into a dual-branch feature extraction network. The first branch performs polar coordinate transformation on the multi-frame illumination equalization image and uses one-dimensional convolution to extract continuous edge features along the circumferential direction. The second branch uses dilated spatial convolution pooling pyramid in Cartesian coordinate system to extract local geometric abrupt features. The continuous edge features and the local geometric mutation features are dimensionally aligned and adaptively weighted and fused through the feature cross-attention mechanism, and the fused features are then passed through the pixel-level binarization mask of the output pipeline of the fully convolutional decoder.
2. The pipe cut recognition method based on machine vision according to claim 1, characterized in that, The multi-angle ring light source array includes multiple independent light source partitions that are evenly distributed around the circumference. The preset timing for alternately triggering light source combinations of different phases includes: dividing the multiple independent light source partitions into non-overlapping odd-numbered sequence light source partitions and even-numbered sequence light source partitions. In the first exposure period, only the odd-numbered sequence light source partitions are illuminated, and in the second exposure period, only the even-numbered sequence light source partitions are illuminated, and so on, the multi-frame original image sequence is acquired alternately and cyclically. The method for generating the high reflectivity frequency band suppression mask includes: calculating the average amplitude spectrum of the multi-frame original image sequence in the frequency domain, dividing the coordinates of frequency points in the average amplitude spectrum whose amplitude exceeds a preset dynamic threshold into high reflectivity frequency bands, and setting the frequency domain coefficients corresponding to the high reflectivity frequency bands to zero to generate the high reflectivity frequency band suppression mask.
3. The pipe cut recognition method based on machine vision according to claim 1, characterized in that, The process of constructing the shadow band compensation mask includes: performing polar coordinate unfolding transformation on the multi-frame original image sequence, and calculating the mean gray-level gradient of the unfolded image in the radial direction; Image regions whose mean gray-level gradient in the radial direction is lower than the preset dark area gradient are marked as shadow candidate regions, and the shadow candidate regions are mapped back to the frequency domain space to extract the corresponding low-frequency shadow bands. Extract the average frequency domain coefficients of the adjacent high-frequency bands of the low-frequency shadow band, and use the average frequency domain coefficients to perform linear interpolation compensation on the frequency domain coefficients of the low-frequency shadow band. Mark the compensated frequency domain space as the shadow band compensation mask.
4. The pipe cut recognition method based on machine vision according to claim 1, characterized in that, The first branch performs polar coordinate transformation on the multi-frame illumination equalization image and uses one-dimensional convolution to extract continuous edge features along the circumferential direction, including: taking the physical center of the pipe cut as the origin of polar coordinates, mapping the multi-frame illumination equalization image from the Cartesian coordinate system to the polar coordinate system, and generating a polar coordinate expansion matrix with angle as the horizontal axis and radial distance as the vertical axis. A one-dimensional causal convolution kernel with a preset receptive field size is applied along the horizontal axis of the polar coordinate unfolded matrix. The one-dimensional causal convolution kernel slides unidirectionally along the horizontal axis to extract the gray-level jump sequence between adjacent pixels in the horizontal axis as the continuous edge features along the circumferential direction.
5. The pipe cut recognition method based on machine vision according to claim 1, characterized in that, The second branch extracts local geometric abrupt features in the Cartesian coordinate system by using dilated spatial convolution pooling pyramids, including: in the Cartesian coordinate system, parallel multi-branch dilated convolutional layers use different dilation rates to perform convolution sampling on the multi-frame illumination equalization images. The specific values of the different void ratios are adaptively configured according to the nominal wall thickness parameter of the pipeline to be identified, so that the receptive field size corresponding to the largest void ratio in the multi-branch void convolutional layer is equivalent to the pixel span occupied by the nominal wall thickness parameter in the image. The output feature maps of the multi-branch dilated convolutional layer are concatenated by channels and then generated by global average pooling to produce the local geometric mutation features.
6. The pipe cut recognition method based on machine vision according to claim 1, characterized in that, The continuous edge features and the local geometric mutation features are dimensionally aligned and adaptively weighted and fused through a feature cross-attention mechanism, including: using the local geometric mutation features as a query matrix and the continuous edge features as both a key matrix and a value matrix. Calculate the product of the query matrix and the transpose of the key matrix to obtain the spatial similarity matrix across coordinate systems; Normalize each row of the spatial similarity matrix to generate an adaptive weight map, and multiply the adaptive weight map with the value matrix.
7. The pipe cut recognition method based on machine vision according to claim 3, characterized in that, After generating the shadow band compensation mask, the method further includes performing a time-series dynamic update of the shadow band compensation mask: extracting three initial shadow band compensation masks corresponding to three consecutive frames of illumination equalization images, calculating the first difference region between the current frame's initial shadow band compensation mask and the previous frame's initial shadow band compensation mask, and calculating the second difference region between the current frame's initial shadow band compensation mask and the next frame's initial shadow band compensation mask. Perform an intersection operation on the first difference region and the second difference region, and remove the non-static shadow frequency bands caused by dynamic drifting objects in the intersection. The removed mask is superimposed on the initial shadow band compensation mask of the current frame to generate the final shadow band compensation mask after time-updated sequence.
8. The pipe cut recognition method based on machine vision according to claim 4, characterized in that, After generating the polar coordinate unfolded matrix and before applying the one-dimensional causal convolution kernel, the method further includes injecting radial position encoding into the polar coordinate unfolded matrix: obtaining the radial distance value corresponding to each pixel in the polar coordinate unfolded matrix, and mapping the radial distance value to a fixed-dimensional radial position vector through a sine position encoding function. The radial position vector and the image feature vector of the polar coordinate expansion matrix are concatenated in the channel dimension to generate an enhanced polar coordinate matrix carrying radial depth information; The one-dimensional causal convolution kernel specifically acts on the enhanced polar coordinate matrix, so that the extracted continuous edge features along the circumferential direction include radial depth position attributes.
9. The pipe cut recognition method based on machine vision according to claim 6, characterized in that, After outputting the fused features and before outputting them via the fully convolutional decoder, the method further includes applying a priori boundary constraint of the physical cutting trajectory to the fused features: generating a standard tangent direction field of the ideal circular trajectory in the image coordinate system based on the preset cutting trajectory parameters of the pipe cutting device. Calculate the predicted gradient direction corresponding to each feature point in the fused features, and calculate the cosine value of the angle between the predicted gradient direction and the standard tangent direction at the corresponding position in the standard tangent direction field; The negative of the cosine of the included angle is used as a boundary compliance penalty term. This boundary compliance penalty term is superimposed on the total loss function of the network, and backpropagation is used to constrain the pixel-level binary mask output by the fully convolutional decoder to fit the standard tangent direction field.
10. A pipe cut recognition system based on machine vision, characterized in that, include: A multi-angle ring light source array is used to alternately trigger combinations of light sources with different phases according to a preset timing sequence; An image acquisition device, synchronized with the multi-angle ring light source array, is used to acquire a multi-frame original image sequence of the pipe cut. A frequency domain illumination modulation device is used to convert the multi-frame original image sequence to the frequency domain by performing Fourier transform on each frame, and to perform weighted processing on the frequency domain coefficients of each frame of the multi-frame original image sequence by using a preset high reflectivity frequency band suppression mask and shadow frequency band compensation mask, and to generate multi-frame illumination equalization images by inverse Fourier transform. The dual-branch feature extraction and fusion device is used to perform polar coordinate transformation on the multi-frame illumination equalization image through the first branch and extract continuous edge features along the circumferential direction using one-dimensional convolution; and to extract local geometric abrupt features in the Cartesian coordinate system using dilated spatial convolution pooling pyramid through the second branch. The continuous edge features and the local geometric abrupt features are then dimensionally aligned and adaptively weighted and fused through a feature cross attention mechanism. A decoding output device for transmitting pixel-level binarization masks of fused features through the output pipeline of a fully convolutional decoder.