A cable production anomaly detection method fusing multi-view visual features

Abstract

This application provides a method for detecting cable production anomalies by fusing multi-view visual features, relating to the fields of computer vision and industrial defect detection technology. It includes: simultaneously acquiring multi-view images using multiple industrial cameras arranged along the cable circumference; extracting the effective area of ​​the cable surface; using a convolutional neural network to extract multi-scale feature maps and construct a differentiable cylindrical projection layer; establishing an initial geometric mapping based on calibration parameters and theoretical diameter; performing additive correction using the offset correction amount output by the convolutional neural network; generating aligned cylindrical feature maps through differentiable sampling; stitching them into a fused feature tensor; inputting an axial sequence modeling network to extract axial dependencies and outputting axial enhancement features; and outputting the anomaly category, confidence level, and location information through an anomaly discrimination network. This application solves the problems of missed detection of circumferential defects, dynamic feature misalignment, and difficulty in identifying periodic defects in existing technologies, achieving accurate full-surface detection, dynamic adaptive alignment, and effective identification of axially continuous defects.

Description

Technical Field

[0001] This application relates to the fields of computer vision and industrial defect detection technology, and in particular to a method for detecting cable production anomalies by integrating multi-view visual features. Background Technology

[0002] In industrial production, the quality of cables directly affects the safety and reliability of equipment operation. Traditional cable manufacturing anomaly detection mainly relies on manual visual inspection or simple machine vision methods. These methods have the following main problems: (1) Low detection accuracy Cables may exhibit anomalies such as radial runout, diameter fluctuations, surface scratches, and bubbles during the manufacturing process. Manual visual inspection is susceptible to factors like fatigue and lack of experience, resulting in a high rate of missed defects. Traditional single-view machine vision systems struggle to fully cover the circumferential surface of the cable, making it difficult to detect localized defects.

[0003] (2) Difficulty in fusing information from multiple perspectives To comprehensively inspect cable surfaces, multiple cameras are often deployed in industrial settings. However, images from different perspectives exhibit geometric distortion, viewing angle differences, and variations in lighting. Traditional methods for multi-view images typically employ simple stitching or frame-by-frame inspection, failing to fully utilize the geometric relationships and complementary information between the multiple perspectives, resulting in insufficient detection accuracy and robustness.

[0004] (3) Difficulty in handling changes in cable geometry Cables are typically cylindrical objects, and their radial runout and minute diameter fluctuations can cause shifts in the location of defects in images. Traditional geometric unfolding methods generally employ fixed cylindrical mapping or manual correction, lacking differentiable mapping mechanisms and thus unable to be optimized during end-to-end learning.

[0005] (4) Axial spatial dependence is difficult to model As slender cylinders, cables often exhibit a certain spatial distribution pattern in their surface defects along the axial direction (e.g., scratches may extend continuously, and periodic defects may reflect process oscillations). Most existing methods treat the unfolded cylindrical image as a regular two-dimensional image and use general convolutional neural networks to extract local features, failing to fully model the long-range dependence of defects along the axial direction, thus limiting the accuracy of anomaly detection.

[0006] (5) Traditional methods lack automation capabilities Most existing industrial vision systems rely on manually set detection templates or thresholds, resulting in poor adaptability to novel defects or changing environments. Furthermore, the geometric correction of multi-view images and the anomaly detection network are independent of each other, making joint optimization during end-to-end training impossible, thus hindering the adaptive correction of geometric errors by the detection task. In addition, existing methods struggle to simultaneously output pixel-level anomaly localization and category classification, failing to meet the demands for refined detection.

[0007] In summary, existing technologies for multi-view cable surface anomaly detection suffer from limitations such as limited detection accuracy, insufficient fusion of multi-view information, inability to adaptively correct geometric errors, underutilization of axial spatial dependencies, and insufficient automation. Therefore, there is an urgent need for a high-precision cable anomaly detection method that can integrate multi-view features, automatically correct geometric errors, model axial dependencies, and support end-to-end training. Summary of the Invention The purpose of this application is to provide a method for detecting cable manufacturing anomalies by integrating multi-view visual features. This addresses the technical problems in existing technologies, such as large multi-view projection alignment errors caused by cable radial runout and diameter fluctuations, the inability to optimize geometric correction and detection networks independently to the end, insufficient utilization of the axial spatial dependence of defects, and the difficulty in simultaneously outputting anomaly category and pixel-level location information.

[0008] In view of the above technical problems, this application provides a method for detecting cable production anomalies by integrating multi-view visual features.

[0009] A first aspect of this application provides a method for detecting cable production anomalies by fusing multi-view visual features, the method comprising: Multiple industrial cameras are arranged along the circumference of the cable and synchronously triggered by an encoder. During the cable's travel, a set of synchronous multi-view images are acquired at fixed spatial intervals. The calibration parameters of each industrial camera and the theoretical diameter of the cable are obtained. The calibration parameters include the intrinsic parameter matrix, extrinsic parameter rotation matrix, and translation vector of each camera. The multi-view images are preprocessed, and the effective area of ​​the cable surface in each view image is extracted based on the theoretical diameter and calibration parameters of the cable. The preprocessed multi-view images are input into a convolutional neural network to extract multi-scale feature maps of each view. A differentiable cylindrical projection layer is constructed. The differentiable cylindrical projection layer is used to establish an initial geometric mapping from the pixel coordinates of each view image to a unified cylindrical coordinate system based on the calibration parameters and the theoretical diameter of the cable. The output coordinates of the initial geometric mapping are corrected by adding the offset correction amount output by the convolutional neural network. Then, the multi-scale feature maps of each view are transformed to a unified cylindrical coordinate system through a differentiable sampling operation to generate aligned cylindrical feature maps. The aligned cylindrical feature maps are concatenated along the channel dimension to form a fused feature tensor, wherein the aligned cylindrical feature maps have circumferential properties. Each sampling point has, along the axial direction One location, The number of circumferential sampling points. This refers to the number of axial positions; The fused feature tensor is input into the axial sequence modeling network to extract axial sequence dependencies and output axial enhancement features; the axial enhancement features are input into the anomaly discrimination network to output anomaly detection results, which include anomaly category, anomaly confidence score, and anomaly location information in the unified cylindrical coordinate system.

[0010] One or more technical solutions provided in this application have at least the following technical effects or advantages: By constructing a differentiable cylindrical projection layer, based on the initial geometric mapping established according to the theoretical cable diameter and camera calibration parameters, an offset correction amount predicted by a convolutional neural network is introduced to additively correct the projected coordinates. This adaptively compensates for dynamic coordinate deviations caused by cable radial runout and diameter fluctuations, significantly improving the projection alignment accuracy of multi-view feature maps in a unified cylindrical coordinate system. Simultaneously, the differentiable cylindrical projection layer is jointly optimized end-to-end with the axial sequence modeling network and the anomaly detection network. Cylindrical projection alignment loss and offset correction amount regularization loss are designed to make geometric correction serve the detection task, overcoming the shortcomings of existing technologies where geometric preprocessing and detection networks are independent. The aligned cylindrical feature map is then unfolded along the axial dimension and... By employing bidirectional long short-term memory networks or Transformer networks for sequence dependency modeling, the spatial distribution patterns and long-range dependencies of cable surface defects along the axial direction are fully utilized, improving the robustness of detection for anomaly types such as slender scratches and periodic defects. Furthermore, the anomaly discrimination network adopts a dual-path parallel structure of classification and segmentation heads, coupled with an output integration unit, enabling simultaneous output of anomaly category, confidence score, and pixel-level location information of the anomaly in a unified cylindrical coordinate system, meeting the requirements for refined detection. Moreover, by introducing theoretical cylindrical feature maps as geometric supervision signals and spatial smoothness constraints for offset correction, a deep fusion of physical geometric priors and data-driven learning is achieved, enhancing the model's interpretability and generalization ability.

[0011] The above description is merely an overview of the technical solution of this application. In order to more clearly explain the technical means of this application, and to enable its implementation in accordance with the contents of the specification, and to make the above and other objectives, features and advantages of this application more apparent and understandable, specific embodiments of this application are described below. Attached Figure Description

[0012] To more clearly illustrate the technical solutions of the embodiments of this disclosure, the accompanying drawings of the embodiments of this disclosure will be briefly described below. Flowcharts are used in this application to illustrate the operations performed by the system according to the embodiments of this application. It should be understood that the preceding or following operations are not necessarily performed precisely in sequence. Instead, various steps can be processed in reverse order or simultaneously as needed. Furthermore, other operations can be added to these processes, or one or more steps can be removed from these processes.

[0013] Figure 1 This is a flowchart illustrating a cable production anomaly detection method that integrates multi-view visual features, as provided in an embodiment of this application. Detailed Implementation

[0014] This application provides a method for detecting cable production anomalies by integrating multi-view visual features, which solves the technical problems in the prior art, such as large multi-view projection alignment errors caused by radial runout and diameter fluctuations of cables, the inability to optimize geometric correction and detection networks independently without end-to-end optimization, the failure to fully utilize the axial spatial dependence of defects, and the difficulty in simultaneously outputting anomaly category and pixel-level position information.

[0015] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0016] It should be noted that the terms “comprising” and “having”, and any variations thereof, are intended to cover non-exclusive inclusion, for example, a process, method, system, product, or server that includes a series of steps or units is not necessarily limited to those steps or units that are explicitly listed, but may include other steps or modules that are not explicitly listed or that are inherent to such processes, methods, products, or devices.

[0017] Example 1, as Figure 1 As shown, this application provides a method for detecting cable production anomalies by fusing multi-view visual features, wherein the method includes: Multiple industrial cameras are arranged along the circumference of the cable and synchronously triggered by an encoder. During the cable's travel, a set of synchronous multi-view images are acquired at fixed spatial intervals. The calibration parameters of each industrial camera and the theoretical diameter of the cable are obtained. The calibration parameters include the intrinsic parameter matrix, extrinsic parameter rotation matrix, and translation vector of each camera. Specifically, four industrial cameras are arranged at equal angles along the circumference of the cable to be inspected (for example, for a 10mm diameter cable, the angle between adjacent cameras is 90°; those skilled in the art can adjust the number of cameras according to the cable diameter, for example, six or eight cameras can be added for larger diameters, as long as the field of view of each camera completely covers the entire circumference of the cable after being stitched together). The optical axes of each camera intersect at the cable axis, and the front end of the lens is set at a fixed distance from the cable surface (for example, 50mm, this distance can be selected in the range of 30mm to 80mm according to the working distance of the camera and the imaging clarity), and the focal length of the lens is adjusted so that the circumferential imaging of the cable occupies 70% to 90% of the image brightness. An incremental encoder (for example, Omron E6B2-CWZ6C type, resolution 1000 pulses / revolution; such as 500, 2000 pulses / revolution, determined according to the required sampling interval) is installed at the end of the cable traction wheel. The A-phase signal of the encoder is connected to the high-speed counting module of the programmable logic controller. The circumference of the traction wheel is denoted as... (For example, 200mm, the actual value is determined by the diameter of the traction wheel), then the cable travel distance corresponding to each encoder pulse. For example, when At a resolution of 1000 pulses / revolution, / pulse; those skilled in the art can select based on the minimum detectable defect size. Typically, the value is no larger than half the minimum defect size (e.g., 0.2mm for a 0.5mm defect and 0.1mm for a 0.2mm defect). The encoder pulse output is directly connected to the hardware trigger input of each camera. Each time the encoder generates a pulse, it simultaneously triggers all cameras to acquire an image, thus capturing images at fixed spatial intervals. A set of synchronized multi-view images was obtained. During the installation and debugging phase, each camera was calibrated independently. For example, Zhang Zhengyou's calibration method was used: a checkerboard calibration board (squares with a side length of 10mm and 7×10 corner points) was placed at the cable axis, and 20 to 30 images were acquired from different angles. The intrinsic parameter matrix of each camera was calculated using OpenCV's calibrateCamera function. (in This is the focal length in pixels. (optical center coordinates), extrinsic rotation matrix (3×3 orthogonal matrix) and translation vector (3×1 vector). During calibration, the world coordinate system is defined as follows: the origin is located at the intersection of the cable axis and the central plane of the camera group perpendicular to the axis; the X-axis is along the cable's travel direction; the Y-axis is vertically upward; and the Z-axis is determined by a right-hand rule. The theoretical diameter of the cable... The values ​​can be read from production specifications or configuration files. For example, 10.0 mm can be used for PVC sheathed cables, and 20.0 mm or more for power cables. Those skilled in the art can set the corresponding values ​​according to the actual cable model. The above-mentioned number of cameras, trigger interval, calibration plate parameters, focal length range, etc. are only examples and do not constitute a limitation on the scope of protection of this invention. In actual applications, they can be adjusted according to factors such as cable diameter, production speed, and detection accuracy requirements (for example, for a 20 mm diameter cable, six cameras can be arranged, the encoder pulse interval can be 0.5 mm, and the calibration plate size and shooting distance can be adjusted accordingly), as long as synchronous acquisition and calibration parameter acquisition are achieved.

[0018] The multi-view images are preprocessed, and the effective area of ​​the cable surface in each view image is extracted based on the theoretical diameter and calibration parameters of the cable. Specifically, the four multi-view images acquired simultaneously are preprocessed, each with a resolution of 2448×2048 pixels. The preprocessing includes the following steps: First, median filtering for noise reduction: The filtering window size is set to 3×3 pixels. For each pixel in the image, all pixel values ​​within its 3×3 neighborhood are extracted, sorted in ascending order, and the median value is taken as the new pixel value. This process is repeated for all pixels in the image to obtain the denoised image. In another example, for cables with coarse surface texture, the filtering window size is set to 5×5 pixels, and the median of the neighborhood is also taken as the new pixel value. Second, adaptive histogram equalization to enhance contrast: The denoised image from the first step is divided into 32×32 pixel rectangular blocks. The contrast limit parameter is set to 2.0. For each image block, its grayscale histogram is calculated, and the number of pixels at each grayscale level (0-255) is counted. The average number of pixels per level is calculated as: total number of pixels in the block / 256 = 1024 / 256 = 4. If the number of pixels at a certain gray level exceeds the contrast limit parameter multiplied by the average number of pixels per level (i.e., 2.0 × 4 = 8), the excess portion of that gray level is cropped and evenly distributed across all gray levels to obtain the corrected histogram. The corrected histogram is then subjected to an equalization transformation: the cumulative distribution function is calculated to map the original gray levels to new gray values. The final pixel values ​​are calculated between adjacent blocks using bilinear interpolation to eliminate block boundaries. The enhanced image is then output. In another example, the contrast limit parameter is set to 1.5 (in this case, the limit is 1.5 × 4 = 6). The third step is to extract the effective area of ​​the cable surface: the Canny edge detection algorithm is used to extract the cable boundaries from the enhanced image from the second step. First, a Gaussian filter (kernel size 3 × 3, standard deviation σ = 1.0) is used to smooth the image. Then, the gradient magnitude and direction of each pixel are calculated: the first derivative is calculated using the Sobel operator (horizontal and vertical directions) to obtain the gradient magnitude and direction. Non-maximum suppression is applied to the gradient magnitude: if the gradient magnitude of the current pixel is not a local maximum in the gradient direction, it is set to 0. Finally, dual threshold detection is applied: the low threshold is set to 50 and the high threshold to 150. Pixels with gradient magnitudes higher than the high threshold are marked as strong edges, pixels with gradient magnitudes lower than the low threshold are discarded, pixels with gradient magnitudes between the two and connected to strong edges are marked as weak edges, and the remaining pixels are discarded, outputting a binarized edge map. In another example, the low threshold is set to 40 and the high threshold to 120. Hough line detection is applied to the above binarized edge map. Based on the theoretical diameter of the cable D = 10.0 mm and the camera calibration parameters Ki, Ri, ti, the theoretical imaging width of the cable in the image is calculated. Specific calculation method: Using the pinhole projection model, the three-dimensional points (Y=0, Z=0, X=±D / 2 in world coordinates) of the left and right boundaries of the cable cylinder are projected onto the image plane of the i-th camera to obtain the pixel horizontal coordinates of the left and right boundaries. The difference between them is the theoretical imaging width.The distance between the left and right boundaries detected in step three is compared with the theoretical width to calculate the relative error. If the relative error is greater than 10%, feedback adjustment is performed: if the detected width is less than the theoretical width, the low and high thresholds of Canny are each reduced by 10 (but not less than 20); if the detected width is greater than the theoretical width, the low and high thresholds are each increased by 10 (but not more than 250). The maximum number of iterations is set to 5. After each threshold adjustment, edge detection and Hough line detection are re-executed, and the relative error is recalculated. If the relative error is less than 10% within 5 iterations, the iteration is stopped and the current detection result is used; if the relative error is still not less than 10% after reaching the maximum number of iterations, the detection result of the last iteration is used, and a warning message of threshold adjustment failure is recorded in the log. Edge detection and Hough line detection are repeated until the relative error is less than 10%. In this embodiment, the relative error of the first detection is 3%, which is less than 10%, so no adjustment is triggered. The accumulator threshold of the Hough transform is set to 100, the resolution of the polar coordinate parameter ρ is 1 pixel, and the resolution of θ is π / 180 radians. All straight lines in the image are detected, and lines located on the left and right sides of the image (i.e., the x-coordinate of the center point of the line is less than 1 / 3 or greater than 2 / 3 of the image width) and have an angle of less than 5 degrees with the horizontal axis of the image are selected. The two lines closest to the image center are taken as the left and right boundaries of the cable. Based on the equations of these two lines, the x-coordinates of the left and right boundary points of each row of pixels in the image are calculated, and all pixels between the left and right boundaries are cropped as the effective area of ​​the cable surface. If the angle between the cable axis and the horizontal axis of the image is greater than 1 degree, rotation correction is performed: the angle θ between the axis and the horizontal axis is calculated, and the cropped effective area image is rotated by -θ angle with the image center as the rotation center. After rotation, the pixels outside the boundary are filled with 0. The rotated effective area image is scaled to 512×512 pixels. In another example, the original cropping size is kept unchanged; in yet another example, it is scaled to 256×256 pixels. In this embodiment, the scaling uses a bilinear interpolation method: the target pixel value is calculated by bilinear interpolation of the four integer coordinate pixel values ​​around the corresponding floating-point coordinate in the source image. After the above three preprocessing steps, four preprocessed multi-view images are obtained. Each image contains only the surface area of ​​the cable, and all images are of uniform size (512×512 pixels in this embodiment). The filter window size, CLAHE block size and contrast limit value, Canny threshold, Hough transform parameters, scaling size, etc. mentioned above are specific choices in this embodiment and do not constitute a limitation on the scope of protection of the claims.

[0019] The preprocessed multi-view images are input into a convolutional neural network to extract multi-scale feature maps of each view. A differentiable cylindrical projection layer is constructed. The differentiable cylindrical projection layer is used to establish an initial geometric mapping from the pixel coordinates of each view image to a unified cylindrical coordinate system based on the calibration parameters and the theoretical diameter of the cable. The output coordinates of the initial geometric mapping are corrected by adding the offset correction amount output by the convolutional neural network. Then, the multi-scale feature maps of each view are transformed to a unified cylindrical coordinate system through a differentiable sampling operation to generate aligned cylindrical feature maps. Furthermore, the construction of the differentiable cylindrical projection layer includes: A virtual cylindrical template is created, the perimeter of which is the theoretical perimeter of the cable. ,in The theoretical diameter of the cable is given, and the height H of the virtual cylindrical template is given as the field of view coverage of the industrial camera in the axial direction. The virtual cylindrical template is discretized in the circumferential direction as follows: The sampling points are discretized into L positions along the axial direction, and the spatial resolution of the aligned cylindrical feature map is [value missing]. × ; Based on the calibration parameters of the industrial camera and the theoretical diameter of the cable, the pixel coordinates of the image from the i-th viewpoint are established. To cylindrical coordinates The initial geometric mapping relationship, where the cylindrical surface coordinates include circumferential angles. and axial position Let the output coordinates of the initial geometric mapping be ( , ).

[0020] Furthermore, the differentiable cylindrical projection layer also includes: The convolutional neural network includes a feature extraction branch and a shift regression branch. The feature extraction branch is used to extract multi-scale feature maps from various viewpoints. The shift regression branch consists of fully connected layers or convolutional layers and is used to regress the shift correction amount based on the multi-scale feature maps. The trainable parameters in the shift regression branch are denoted as follows: ; The offset correction amount output by the offset regression branch is used to additively correct the output coordinates of the initial geometric mapping to obtain the corrected geometric mapping relationship: ; ; in, and The offset regression branch pairs pixel coordinates respectively. The predicted circumferential and axial offsets are obtained, and the offset correction output by the offset regression branch has the same spatial resolution as the multi-view image; the offset correction is used to compensate for coordinate deviations caused by radial runout and diameter fluctuations of the cable.

[0021] Furthermore, the modified geometric mapping relationship achieves feature transformation through differentiable sampling operations, including: The coordinates of the cylindrical surface are obtained based on the corrected geometric mapping relationship. , ), the coordinates of the cylindrical surface ( , The pixel coordinates mapped to the multi-scale feature map are calculated using the following formula: ; in, For the first The intrinsic parameter matrix of each camera. For the first The extrinsic rotation matrix and translation vector of each camera. Based on the intrinsic parameter matrix and external references ( The determined inverse projection function is used to map the cylindrical surface coordinates back to the first... A pixel coordinate system for an image from a single perspective; With the pixel coordinates As the sampling location, feature values ​​are sampled from multi-scale feature maps at various viewpoints using bilinear interpolation, and then filled into the virtual cylindrical template at coordinates (...). , At the corresponding positions, generate aligned cylindrical feature maps.

[0022] Specifically, the four preprocessed images of the effective area of ​​the cable surface (scaled to 512×512 pixels in this embodiment; this scaling is only for adapting to the fixed input size of the subsequent convolutional network and does not constitute a limitation on the scope of the claims) are input into a convolutional neural network to extract multi-scale feature maps. In this embodiment, a ResNet-50 convolutional neural network is used: the input image sequentially passes through a 7x7 convolution (stride 2, output 64 channels), batch normalization, ReLU, and 3×3 max pooling (stride 2), followed by four residual block groups, outputting feature maps of 64×64×256, 32×32×512, 16×16×1024, and 8×8×2048 respectively; 16×16×1024 and 8×8×2048 are selected as multi-scale feature maps. In another example, VGG-16 can be used to extract feature maps before and after the fourth pooling layer; in yet another example, MobileNetV3 is used to reduce computational cost. Next, a differentiable cylindrical projection layer is constructed. First, create a virtual cylindrical template: theoretical cable diameter. mm, perimeter mm; Camera axial field of view coverage height mm (corresponding to 512 pixels x 0.1 mm / pixel). Discretize the template circumferentially. Each sampling point (one sampling point every 0.1 mm) is discretized axially. The spatial resolution of the aligned cylindrical feature map is (one sampling point per 0.1 mm). In another instance, for 20mm cable, if the same sampling interval is maintained , It can be maintained at 512 or adjusted according to the field of view. Then establish from the first... Image pixel coordinates from each perspective To cylindrical coordinates The initial geometric mapping. This embodiment uses the analytical method of ray-cylindrical surface intersection: defining a world coordinate system (origin at the intersection of the cable axis and the central plane of the camera group), The shaft runs along the direction of cable travel. The axis is vertically upward. The axis is determined by the right-hand rule. (Emitted from the camera's optical center, passing through the image point...) The ray, the coordinates of its optical center in the world coordinate system (in (External parameter rotation matrix and translation vector), ray direction The calculation formula is: ;in For the first The intrinsic parameter matrix of each camera. The ray parameter equation is as follows: The equation for the cylindrical surface of the cable is: The components of the ray equation are free variables. Substituting into the equation of the cylindrical surface, we obtain about The quadratic equation of : Solve the equation by setting the discriminant... .like If the ray intersects the cylindrical surface, then the ray will not intersect the cylindrical surface (theoretically, this will not happen because the camera is already aligned with the line); if Take the smaller positive root (If both roots are positive, take the smallest positive root), substitute into the ray equation to obtain the world coordinates of the intersection point. Then, the coordinates of the cylindrical surface are calculated: Let the output coordinates of the initial geometric mapping be... In addition to the feature extraction branch, convolutional neural networks also include a offset regression branch: the output of the fourth residual block group in ResNet-50... On the feature map, add two transposed convolutional layers (the kernel size of the first layer is...). Step size 2, 1024 output channels; second-layer core size Step length The feature map is upsampled to the same spatial resolution of 512×512 as the input image, and two channels are output, corresponding to the circumferential offset of each pixel position (ui, vi) in the input image. and axial offset The spatial resolution of the output offset field is 512×512 (the same as the input image). All trainable parameters in the offset regression branch are denoted as... (Including the weights and biases of the transposed convolutional layer), initially all offsets are 0. The offset corrections are additively added to the initial mapped coordinates to obtain the corrected geometric mapping: Subsequently, feature transformation is achieved through differentiable sampling operations: based on the corrected... Calculate the pixel coordinates of the corresponding source image using the inverse projection function According to the corrected geometric mapping relationship Through the inverse projection function Directly calculate the sampling position: , The corrected geometric mapping already includes the offset correction, so there's no need to add the offset again. The inverse projection function follows the reverse process of the initial mapping: first, it uses the cylindrical coordinates... According to the world coordinate formula , , We obtain the three-dimensional points, and then use the pinhole projection formula: ;in The depth factor (taking the third component of the result) is used to obtain floating-point pixel coordinates. This embodiment uses... Feature map, needs to be sampled before The image is scaled from its original resolution (512×512) to the feature map resolution (8×8) with a scaling factor of 1 / 64, meaning the sampling coordinates on the feature map are... Then, bilinear interpolation sampling is used: Let... Take the integer part Decimal part Extract the feature values ​​of the four corner points from the feature map. Calculate the sampled values: Repeat the above process for the four perspectives to obtain four aligned cylindrical feature maps. Each size is In another instance, it can be used simultaneously. The feature map is first upsampled to a value using bilinear interpolation. , and again The upsampled results are concatenated along the channels to enrich the feature representation. Finally, the four cylindrical feature maps are concatenated along the channel dimension to form a fused feature tensor with a size of [size missing]. The convolutional neural network structure, discretization parameters, transposed convolutional regression branch, ray-cylindrical analytical intersection, and bilinear interpolation sampling mentioned above are all specific choices made in this embodiment.

[0023] The aligned cylindrical feature maps are concatenated along the channel dimension to form a fused feature tensor, wherein the aligned cylindrical feature maps have circumferential properties. Each sampling point has, along the axial direction One location, The number of circumferential sampling points. This refers to the number of axial positions; Specifically, for example, the four aligned cylindrical feature maps are denoted as F1, F2, F3, and F4, respectively. In this embodiment, to reduce computational resource consumption, W=100 and L=256 are used as the default parameter configuration (corresponding to 100 sampling points in the circumferential direction and 256 sampling points in the axial direction), and the number of channels C=1024 (obtained from the third residual block group of ResNet-50). In another example, to pursue higher detection accuracy, a high-resolution configuration of W=314 and L=512 can be used, in which case the number of channels C=2048 (obtained from the fourth residual block group). The following explanation uses the default parameter configuration (W=100, L=256) as an example. Perform a concatenation operation along the channel dimension on these four feature maps: For each fixed circumferential angle index w∈[1,W] and axial position index l∈[1,L] on the virtual cylindrical template, extract the C-dimensional feature vectors v1, v2, v3, v4 located at (w,l) from F1, F2, F3, F4 respectively, and concatenate these four vectors in sequence into a new vector of length 4C. =[v1;v2;v3;v4]. Repeat this operation for all W×L spatial locations to obtain a three-dimensional tensor of size W×L×4C, which is the fused feature tensor. In this embodiment, 4C=4096, so the spatial resolution of the fused feature tensor is 100×256, and the number of channels is 4096. At the implementation level, this can be accomplished using `torch.cat([F1,F2,F3,F4],dim=2)` in a deep learning framework (such as PyTorch), where `dim=2` represents the channel dimension. This concatenation operation does not introduce any trainable parameters; it only rearranges the data. In another example, if the number of channels in the four cylindrical feature maps is different (e.g., C1, C2, C3, C4 respectively), then each feature map needs to be projected to the same number of channels C′ using an independent 1×1 convolutional layer before concatenation; in this embodiment, the four feature maps have the same number of channels, so projection is unnecessary. After concatenation, the fused feature tensor... It is output to the axial sequence modeling network.

[0024] The fused feature tensor is input into the axial sequence modeling network to extract axial sequence dependencies and output axial enhancement features; the axial enhancement features are input into the anomaly discrimination network to output anomaly detection results, which include anomaly category, anomaly confidence score, and anomaly location information in the unified cylindrical coordinate system.

[0025] Furthermore, the axial sequence modeling network includes: The input layer is used to receive the fused feature tensor, wherein the dimension of the fused feature tensor is... ,in The number of channels in the fused feature tensor. The number of spatial positions along the axial direction of the cable. This represents the number of sampling points along the circumference of the cable. An axial sequence construction unit is used to expand the fused feature tensor along the axial dimension of the cable, and concatenate the features of the circumferential and channel dimensions corresponding to each axial position to form a feature vector for that axial position, generating an axial sequence feature matrix. The dimension of the axial sequence feature matrix is... ; The sequence modeling unit is used to model the sequence dependencies along the axial dimension of the axial sequence feature matrix and output an axial enhancement feature matrix, wherein the dimension of the axial enhancement feature matrix is... in The dimension of the axial enhancement feature; The sequence modeling unit is a bidirectional long short-term memory network or a Transformer network; The output layer is used to output the axial enhancement feature matrix as the axial enhancement feature to the anomaly discrimination network.

[0026] Specifically, the fused feature tensor of the above output Input axial sequence modeling network. This embodiment uses the default parameter configuration (W=100, L=256) as an example. To reduce computational resource consumption or adapt to different detection accuracy requirements, this configuration is as follows: feature maps are taken from the third residual block group of ResNet-50 (channel number C=1024), the number of circumferential sampling points is set to W=100 (corresponding to one sampling point every 0.314 mm), and the number of axial positions is set to L=256 (corresponding to one sampling point every 0.2 mm). Accordingly, the discretization parameters W and L of the virtual cylindrical template mentioned above should also be adjusted synchronously. 100 The spatial resolution of the fused feature tensor becomes 100×256, and the number of channels is 4C=4096. For a high-resolution configuration (W=314, L=512, C=2048), the size of the fused feature tensor is 314×512×8192. At this time, the length of the feature vector after the axial sequence construction unit is flattened is 8192×314=2,572,288. The embedding dimension of the dimensionality reduction layer can be adjusted accordingly to E=512. The rest of the network structure (dimensionality reduction layer, Bi-LSTM / Transformer, output dimension, etc.) remains unchanged. Only the upsampling interpolation factor needs to be adjusted, which will not be elaborated further. Those skilled in the art can choose between the above two sets of parameters according to actual computing resources and detection accuracy requirements, or set other parameter combinations themselves. The axial sequence modeling network processes in the following order: First, the input layer receives the tensor. Then, the axial sequence construction unit performs the following operations: The fused feature tensor is arranged along the axial dimension ( Expanding by dimension, for each axial position index Extract all circumferential sampling points and features of all channels corresponding to that position, i.e., extract a feature of size 4. The matrix is ​​flattened into a matrix of length 4. One-dimensional feature vector All The feature vectors at each position are arranged in axial order to generate an axial sequence feature matrix. In this embodiment, 4 100 = 409600 The size is Next, a linear dimensionality reduction layer is first set up inside the sequence modeling unit to reduce each... Mapping from 409,600 dimensions to embedded dimensions The dimensionality-reduced sequence matrix is ​​obtained. The size is Then, the sequence modeling unit... Sequence dependency modeling is performed along the axial dimension. In this embodiment, the sequence modeling unit uses a bidirectional long short-term memory network (Bi-LSTM), and the hidden layer dimension is set to... .Will Each row serves as input for a time step. The Bi-LSTM processes the entire sequence along both the forward and backward directions, outputting the hidden state at each time step to obtain the axial augmentation feature matrix. The size is In another example, the sequence modeling unit uses a Transformer network. The Transformer encoder also first reduces the input dimension from 409600 to 256 using a linear dimensionality reduction layer, resulting in... The encoder then consists of four layers, each with eight attention heads. The feedforward network has a dimension of 1024, the input feature dimension is 256, and the output is also... The matrix. Finally, the output layer will... The axial enhancement feature is output to the anomaly detection network. Those skilled in the art should understand that the above parameters ( (This is just an example, where) This refers to the number of channels in a single-view cylindrical feature map; the actual number of fused feature channels used for axial sequence modeling is 4. In practical applications, other parameter combinations can be selected based on computing resources (e.g.) ), as long as the dimension of the axial sequence feature matrix is Furthermore, the sequence modeling unit models the sequence dependencies of this matrix.

[0027] Furthermore, the anomaly detection network includes: A feature receiving layer is used to receive the axial enhancement feature matrix, wherein the dimension of the axial enhancement feature matrix is... ,in The number of spatial positions along the axial direction of the cable. The dimension of the axial enhancement feature; A segmentation head, connected to the feature receiving layer, is composed of a transposed convolutional layer and an upsampling layer, used to upsample the axially enhanced feature matrix to the same spatial resolution as the aligned cylindrical feature map. Each pixel is classified and the anomaly category and anomaly confidence score of each pixel are output, thereby generating a pixel-level position mask of the anomaly in the unified cylindrical coordinate system. The unified cylindrical coordinate system is discretized in the circumferential direction as follows: Each sampling point is discretized along the axial direction. One location, The number of circumferential sampling points and the number of axial positions are the same; The output integration unit is used to integrate the pixel-level anomaly categories and confidence scores output by the segmentation head to form an anomaly detection result containing the category label and location information of each anomaly pixel.

[0028] Specifically, the axial enhancement feature matrix output above (This embodiment is) , Input anomaly detection network. Feature receiving layer receives... And it is directly transmitted to the segmentation head. The segmentation head first... Reshape into shape The three-dimensional tensor, in which Height (corresponding to the number of axial positions), width is The number of channels is The segmentation head performs the following operations in sequence: First transposed convolutional layer: Number of input channels Number of output channels The kernel size is (Operate only in the width direction), step size set to (Step size in height direction) Width direction step size ), fill with The height of the output feature map of this layer remains constant. The width becomes 4, and the number of channels becomes... Second transposed convolutional layer: Number of input channels Number of output channels The kernel size is Step size set to Fill with The input width of this layer is The output width becomes The number of channels becomes Bilinear interpolation upsampling layer: reduces the width from... Upsampling Maintain height Unchanged, number of channels The output feature map size is . Convolutional layer: Number of input channels Number of output channels (Representing normal and abnormal classes respectively), the kernel size is... The output shape of this layer is The fractional graph. For each spatial location in the fractional graph. (in For axial index, Using a circumferential index, the category with the higher score from the two channels is taken as the predicted anomaly category for that pixel, and this score is used as the anomaly confidence score. This generates a pixel-level location mask for the anomaly in a unified cylindrical coordinate system: the mask is a... The matrix stores the anomaly category label and confidence score for each corresponding pixel. The output integration unit receives the pixel-level anomaly category and confidence score matrix output from the segmentation head and performs the following integration operation: For each pixel in the mask that is judged as an anomaly (i.e., the category is "anomaly"), According to axial index and circumferential index Calculate its physical coordinates in a unified cylindrical coordinate system: ;in To detect the starting axial position (taken in this embodiment) ), The axial sampling interval (taken in this embodiment) (millimeters). The output integration unit combines the category label, confidence score, and physical coordinates of each anomalous pixel. These are combined into an anomaly record, and all anomaly records constitute the final anomaly detection result. Those skilled in the art should understand that the parameters such as the number of channels in the transposed convolutional layer, kernel size (1×4), stride (1×2), and upsampling interpolation method are merely examples. In practical applications, these parameters can be adjusted based on the axial feature dimension F, the target resolution W×L, and computational resources. As long as the segmentation head can upsample the axial enhancement feature matrix to the same spatial resolution W×L as the aligned cylindrical feature map, classify each pixel, output a pixel-level position mask, and generate an anomaly detection result containing physical coordinates from the output integration unit, the anomaly detection result will be achieved.

[0029] Furthermore, the convolutional neural network, the axial sequence modeling network, and the anomaly detection network are obtained through end-to-end joint optimization training. The joint optimization adopts a multi-task joint loss function that includes cylindrical projection alignment loss and offset correction regularization loss. Among them, the cylindrical projection alignment loss is used to constrain the geometric structure consistency between the aligned cylindrical feature map and the theoretical cylindrical feature map. The theoretical cylindrical feature map is calculated in advance based on the theoretical diameter of the cable and the calibration parameters of each industrial camera. The image of a cable sample is extracted by the convolutional neural network and then projected onto a unified cylindrical coordinate system. The image of the cable sample is the pixel average of multiple consecutively acquired frames. The offset correction regularization loss is used to constrain the spatial smoothness of the offset correction, so that the offset correction only compensates for the dynamic deviations caused by the radial runout and diameter fluctuations of the cable, without deviating from the physical geometric constraints established based on the theoretical diameter of the cable and the calibration parameters of each industrial camera.

[0030] Specifically, the convolutional neural network, the differentiable cylindrical projection layer, the axial sequence modeling network, and the anomaly detection network are jointly optimized and trained end-to-end. In this embodiment, a multi-task joint loss function is used. ,in These are the weighting coefficients. This represents the alignment loss due to the projection of the cylindrical surface. Regularization loss for offset correction. Cylindrical projection alignment loss. The calculation method is as follows: A cable sample with a length of 1 meter is pre-collected. To ensure the geometric ideality of the template, one of the following methods can be used to obtain the theoretical cylindrical feature map: (a) 10 frames of images of the sample are continuously collected, the average gray value of each pixel position in each frame is calculated, and then median filtering (3×3 window) is used to further suppress random noise; (b) a standard cable segment without surface defects is selected as the sample using a higher precision detection device (such as a laser profilometer); (c) the ideal cylindrical image is directly generated by rendering the CAD model of the cable (with known diameter and surface equation) through a ray tracing algorithm. In this embodiment, method (a) is used, which corresponds to the average pixel value of the continuously collected multi-frame images. The template image is input into the convolutional neural network, and the feature map (size 8×8×2048) output by its fourth residual block group is extracted. Then, the feature map is projected onto the unified cylindrical coordinate system through the aforementioned initial geometric mapping to obtain the theoretical cylindrical feature map. (i=1,2,3,4), the spatial resolution of each feature map is W×L=100×256, and the number of channels is C=2048. During training, for each viewpoint in each training batch... Calculate the aligned cylindrical feature map of the actual output of the network. Compared with the theoretical cylindrical feature diagram The per-pixel mean square error between: This loss forces the cylindrical feature map output by the differentiable cylindrical projection layer to maintain consistency with the geometry of the defect-free template. (Offset correction regularization loss) The calculation method is as follows: for each viewpoint The circumferential offset output by the offset regression branch. and axial offset Two offset fields are constructed, each with the same spatial resolution as the input image (in this embodiment, ...). Calculate the sum of squared spatial gradients of the offset field: ;in Given the total number of pixels in each image, the gradient operator uses central difference, i.e.: , At image boundaries, one-sided difference calculation is used, for example, for the left boundary ( Using forward difference For the right boundary (u=W), use backward difference. For the upper boundary Using forward difference For the lower boundary Using backward difference This regularization term encourages the offset correction to vary smoothly in space, compensating only for low-frequency dynamic deviations caused by cable radial runout and diameter fluctuations, without generating high-frequency noise. During training, the entire network consisting of the convolutional neural network, the differentiable cylindrical projection layer, the axial sequence modeling network, and the anomaly detection network is treated as a computational graph. The Adam optimizer is used, with an initial learning rate set to... Momentum parameters Weight decay is set to Each batch inputs 4 sets of synchronized multi-view images (each set corresponding to one axis sampling position), with a batch size of 8. Training is performed for a total of 200 epochs, with the learning rate multiplied by 0.9 every 10 epochs. During training, the total loss... All trainable parameters of the convolutional neural network, offset regression branch, axial sequence modeling network, and anomaly detection network are updated synchronously through backpropagation. After training, the model parameters are saved for actual cable production anomaly detection. Those skilled in the art should understand that the weight coefficients in the above loss function... The optimizer parameters (learning rate, momentum, weight decay), batch size, and number of training epochs are merely examples and can be adjusted based on training convergence in practical applications. As long as the training process employs both cylindrical projection alignment loss and offset correction regularization loss for joint optimization of the network, it does not depart from the technical concept of this invention.

[0031] It should be noted that the order of the embodiments described above is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. Furthermore, specific embodiments have been described above. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps described in the claims can be performed in a different order than that shown in the embodiments and still achieve the desired result. Additionally, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0032] The above description is only a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

[0033] This specification and accompanying drawings are merely illustrative examples of this application and are intended to cover any and all modifications, variations, combinations, or equivalents within the scope of this application. Clearly, those skilled in the art can make various alterations and modifications to this application without departing from its scope. Therefore, if such modifications and modifications fall within the scope of this application and its equivalents, this application intends to include such modifications and modifications.

Claims

1. A method for detecting cable production anomalies by integrating multi-view visual features, characterized in that, include: Multiple industrial cameras are arranged along the circumference of the cable and synchronously triggered by an encoder. During the cable's travel, a set of synchronous multi-view images are acquired at fixed spatial intervals. The calibration parameters of each industrial camera and the theoretical diameter of the cable are obtained. The calibration parameters include the intrinsic parameter matrix, extrinsic parameter rotation matrix, and translation vector of each camera. The multi-view images are preprocessed, and the effective area of ​​the cable surface in each view image is extracted based on the theoretical diameter and calibration parameters of the cable. The preprocessed multi-view images are input into a convolutional neural network to extract multi-scale feature maps of each view. A differentiable cylindrical projection layer is constructed. The differentiable cylindrical projection layer is used to establish an initial geometric mapping from the pixel coordinates of each view image to a unified cylindrical coordinate system based on the calibration parameters and the theoretical diameter of the cable. The output coordinates of the initial geometric mapping are corrected by adding the offset correction amount output by the convolutional neural network. Then, the multi-scale feature maps of each view are transformed to a unified cylindrical coordinate system through a differentiable sampling operation to generate aligned cylindrical feature maps. The aligned cylindrical feature maps are concatenated along the channel dimension to form a fused feature tensor, wherein the aligned cylindrical feature maps have circumferential properties. Each sampling point has, along the axial direction One location, The number of circumferential sampling points. This refers to the number of axial positions; The fused feature tensor is input into the axial sequence modeling network to extract axial sequence dependencies and output axial enhancement features. The axial enhancement features are input into the anomaly discrimination network, and the anomaly detection results are output. The anomaly detection results include the anomaly category, the anomaly confidence score, and the anomaly's position information in the unified cylindrical coordinate system.

2. The cable production anomaly detection method integrating multi-view visual features according to claim 1, characterized in that, The construction of the differentiable cylindrical projection layer includes: A virtual cylindrical template is created, the perimeter of which is the theoretical perimeter of the cable. ,in The theoretical diameter of the cable is given, and the height H of the virtual cylindrical template is given as the field of view coverage of the industrial camera in the axial direction. The virtual cylindrical template is discretized in the circumferential direction as follows: The sampling points are discretized into L positions along the axial direction, and the spatial resolution of the aligned cylindrical feature map is [value missing]. × ; Based on the calibration parameters of the industrial camera and the theoretical diameter of the cable, the pixel coordinates of the image from the i-th viewpoint are established. To cylindrical coordinates The initial geometric mapping relationship, where the cylindrical surface coordinates include circumferential angles. and axial position Let the output coordinates of the initial geometric mapping be ( , ).

3. The cable production anomaly detection method integrating multi-view visual features according to claim 2, characterized in that, The differentiable cylindrical projection layer further includes: The convolutional neural network includes a feature extraction branch and a shift regression branch. The feature extraction branch is used to extract multi-scale feature maps from various viewpoints. The shift regression branch consists of fully connected layers or convolutional layers and is used to regress the shift correction amount based on the multi-scale feature maps. The trainable parameters in the shift regression branch are denoted as follows: ; The offset correction amount output by the offset regression branch is used to additively correct the output coordinates of the initial geometric mapping to obtain the corrected geometric mapping relationship: ； ； in, and The offset regression branch pairs pixel coordinates respectively. The predicted circumferential and axial offsets are obtained, and the offset correction output by the offset regression branch has the same spatial resolution as the multi-view image; the offset correction is used to compensate for coordinate deviations caused by radial runout and diameter fluctuations of the cable.

4. The cable production anomaly detection method integrating multi-view visual features according to claim 3, characterized in that, The corrected geometric mapping relationship achieves feature transformation through differentiable sampling operations, including: The coordinates of the cylindrical surface are obtained based on the corrected geometric mapping relationship. , ), the coordinates of the cylindrical surface ( , The pixel coordinates mapped to the multi-scale feature map are calculated using the following formula: ； in, For the first The intrinsic parameter matrix of each camera. For the first The extrinsic rotation matrix and translation vector of each camera. Based on the intrinsic parameter matrix and external references ( The determined inverse projection function is used to map the cylindrical surface coordinates back to the first... A pixel coordinate system for an image from a single perspective; With the pixel coordinates As the sampling location, feature values ​​are sampled from multi-scale feature maps at various viewpoints using bilinear interpolation, and then filled into the virtual cylindrical template at coordinates (...). , At the corresponding positions, generate aligned cylindrical feature maps.

5. The cable production anomaly detection method integrating multi-view visual features according to claim 1, characterized in that, The axial sequence modeling network includes: The input layer is used to receive the fused feature tensor, wherein the dimension of the fused feature tensor is... ,in The number of channels in the fused feature tensor. The number of spatial positions along the axial direction of the cable. This represents the number of sampling points along the circumference of the cable. An axial sequence construction unit is used to expand the fused feature tensor along the axial dimension of the cable, and concatenate the features of the circumferential and channel dimensions corresponding to each axial position to form a feature vector for that axial position, generating an axial sequence feature matrix. The dimension of the axial sequence feature matrix is... ; The sequence modeling unit includes a dimensionality reduction layer for reducing the dimension of the feature vector at each axial position in the axial sequence feature matrix. A linear mapping to a preset embedding dimension yields a dimensionality-reduced sequence feature matrix. A sequence modeling subunit is then used to model sequence dependencies along the axial dimension of this dimensionality-reduced sequence feature matrix, outputting an axially enhanced feature matrix. The axially enhanced feature matrix has a dimension of [missing information]. ,in The dimension of the axial enhancement feature is defined as follows: the sequence modeling subunit is a bidirectional long short-term memory network or a Transformer network. The output layer is used to output the axial enhancement feature matrix as the axial enhancement feature to the anomaly discrimination network.

6. The cable production anomaly detection method integrating multi-view visual features according to claim 1, characterized in that, The anomaly detection network includes: A feature receiving layer is used to receive the axial enhancement feature matrix, wherein the dimension of the axial enhancement feature matrix is... ,in The number of spatial positions along the axial direction of the cable. The dimension of the axial enhancement feature; A segmentation head, connected to the feature receiving layer, is composed of a transposed convolutional layer and an upsampling layer, used to upsample the axially enhanced feature matrix to the same spatial resolution as the aligned cylindrical feature map. Each pixel is classified and the anomaly category and anomaly confidence score of each pixel are output, thereby generating a pixel-level position mask of the anomaly in the unified cylindrical coordinate system. The unified cylindrical coordinate system is discretized in the circumferential direction as follows: Each sampling point is discretized along the axial direction. One location, The number of circumferential sampling points and the number of axial positions are the same; The output integration unit is used to integrate the pixel-level anomaly categories and confidence scores output by the segmentation head to form an anomaly detection result containing the category label and location information of each anomaly pixel.

7. The cable production anomaly detection method integrating multi-view visual features according to claim 1, characterized in that, The convolutional neural network, the axial sequence modeling network, and the anomaly detection network are obtained through end-to-end joint optimization training. The joint optimization adopts a multi-task joint loss function that includes cylindrical surface projection alignment loss and offset correction regularization loss. Among them, the cylindrical projection alignment loss is used to constrain the geometric structure consistency between the aligned cylindrical feature map and the theoretical cylindrical feature map. The theoretical cylindrical feature map is calculated in advance based on the theoretical diameter of the cable and the calibration parameters of each industrial camera. The image of a cable sample is extracted by the convolutional neural network and then projected onto a unified cylindrical coordinate system. The image of the cable sample is the pixel average of multiple consecutively acquired frames. The offset correction regularization loss is used to constrain the spatial smoothness of the offset correction, so that the offset correction only compensates for the dynamic deviations caused by the radial runout and diameter fluctuations of the cable, without deviating from the physical geometric constraints established based on the theoretical diameter of the cable and the calibration parameters of each industrial camera.

Images