A wire straightening method based on deep learning

Abstract

The present invention belongs to the technical field of deep learning, and particularly relates to a wire straightening method based on deep learning. First, the method collects wire images from the side surface and the top through a high-frame-rate industrial camera. After image augmentation and enhancement processing, the enhanced side surface image is input into an improved YOLOV8 architecture to obtain bending feature data, and at the same time, the area deviation of the top surface image is calculated. A MIMO neural network straightening control model is constructed using the bending feature parameters, and pressure, stroke, and direction control instructions for the straightening device are output. During the straightening process, the straightness is judged whether it meets the standard through rough judgment, precise judgment, and combined with the area deviation. Compared with the traditional straightening method, the present invention can comprehensively obtain the bending features of the wire, accurately control the straightening process, and effectively improve the straightening accuracy and efficiency.

Description

Technical Field

[0001] The present invention belongs to the technical field of deep learning, and particularly relates to a steel wire straightening method based on deep learning. Background Art

[0002] In the field of steel wire production and processing, the straightness of steel wire plays a crucial role in its subsequent applications. Traditional steel wire straightening methods have many drawbacks. The mechanical straightening method mainly relies on mechanical rolling, which not only has low efficiency but also has poor adaptability to steel wires of different materials and specifications, making it difficult to ensure the straightening accuracy. Manual straightening highly depends on the experience of workers, which not only has a large labor intensity but also has unstable accuracy and is easily affected by human factors. With the development of technology, straightening methods based on machine vision have emerged, but the existing methods still have defects. On the one hand, image acquisition is mostly single-view, unable to comprehensively obtain the bending characteristics of the steel wire, resulting in inaccurate straightening parameters. On the other hand, the image processing algorithm is not advanced enough, with limited ability to identify the bending characteristics of the steel wire in a complex environment and insufficient robustness. At the same time, the straightening control model lacks intelligence and is difficult to make precise adjustments according to the real-time state of the steel wire. Summary of the Invention

[0003] The present invention aims at the technical problems existing in the background art and proposes a steel wire straightening method based on deep learning.

[0004] To achieve the above object, the technical solution adopted by the present invention is as follows, including the following steps:

[0005] S1. Multi-view image acquisition, collecting the surface image of the steel wire through a high-frame-rate industrial camera; the acquisition of the surface image is to collect the side surface image around the side surface of the steel wire and the top surface image at the top of the steel wire to form a multi-view image dataset;

[0006] S2. Perform image enhancement operations on the multi-view images;

[0007] S3. Input the enhanced side surface pictures into the improved YOLOV8 architecture, and after analysis, output the bending feature data, including the bending angle, curvature radius, and position coordinates;

[0008] The improvement of the YOLOV8 architecture is to add a lightweight curvature estimation branch before the convolutional layer, specifically:

[0009] Input feature map: Output from the backbone network;

[0010] Local gradient calculation: Input feature map After Depthwise separable convolution extracts the local horizontal gradient And the local vertical gradient , , , where , ;

[0011] Calculate the curvature tensor: Based on the local gradient and calculate the second-order derivative and synthesize the curvature tensor K: ;

[0012] Dynamic weight generation: Input the curvature tensor K into a multi-layer perceptron, and generate dynamic convolution kernel weights through the Sigmoid activation function and per-channel multiplication operation. The calculation method is: , where represents the initial convolution kernel weight, is the dynamic convolution kernel weight;

[0013] Feature fusion output: Perform weighted fusion on the results of static convolution and dynamic convolution to obtain the output feature map ;

[0014] Polar coordinate transformation: Use as the input for polar coordinate feature encoding. Taking the center of the detection box as the pole, convert the feature map in the Cartesian coordinate system to polar coordinate representation to obtain the polar coordinate feature map ;

[0015] Polar coordinate convolution: Perform radial and angular convolution operations on the polar coordinate feature map to obtain the radial convolution result and the angular convolution result, and then concatenate the two to obtain the processed polar coordinate feature map ;

[0016] Inverse transformation: Restore the polar coordinate feature map to the Cartesian coordinate system through bilinear interpolation to obtain , and then add it element-wise to the original input feature map to obtain the final output feature map ;

[0017] S4. For the top surface image after image enhancement, calculate the top surface contour area and compare it with the top area of the straightened wire to obtain the area deviation;

[0018] S5. After obtaining the bending feature parameters, construct a straightening control model and output the pressure, stroke, and direction control of the straightening device;

[0019] S6. During the straightening process, update the bending feature parameters in real time and combine the real-time area deviation of the top surface to determine whether the straightness meets the standard.

[0020] Preferably, between the operation of collecting the steel wire surface images in step S1 and the image enhancement in step S2, there is also an image augmentation operation, specifically: for the side surface images, taking the central axis of the steel wire as the rotation axis, rotating at intervals of 10 degrees, and then performing translation and scaling to generate new side surface images; for the top surface images, performing translation and scaling; combining the two operations to augment the multi-view image dataset.

[0021] Preferably, the image enhancement operation in step S2 includes: multi-scale feature extraction, feature fusion, and reconstruction of the enhanced image; multi-scale feature extraction and fusion: performing multi-scale feature extraction on the multi-view images, normalizing each layer of feature maps, and then adding them according to weights to obtain the fused feature map; reconstructing the enhanced image: using transposed convolution operations to upsample and reconstruct the fused feature map into the enhanced image.

[0022] Preferably, the specific multi-scale feature extraction and fusion in step S2 include: constructing a multi-branch feature extraction module: using three groups of convolution kernels set in parallel to extract features of different scales respectively; a dynamic weight fusion mechanism: generating the fusion weight coefficients of each scale feature through a channel attention network, and the calculation formula is: , where is the weight coefficient, is the global average pooling value of each branch feature map; cross-scale feature calibration: performing deformable convolution operations on the fused feature map, and using the offset prediction network to compensate for the geometric distortion under different perspectives.

[0023] Preferably, the process of reconstructing the enhanced image in step S2 includes: a hybrid upsampling module: combining transposed convolution with nearest neighbor interpolation to construct a dual-path reconstruction structure: , where Y is the enhanced image, is the transposed convolution, is the upsampling of 2 times nearest neighbor interpolation, is the element-wise multiplication, and Mask represents the binary mask generated by spatial attention; residual detail restoration: introducing skip connections during the reconstruction process to fuse the high-frequency components of the original input image with the reconstruction result, and the calculation method is: , where is the finally enhanced image, HPF represents the high-pass filtering operation, is the convolution operation, and W is the learnable convolution kernel; iterative optimization: when the local contrast variance of the reconstructed image is less than the set variance threshold, perform the feature re-extraction process, with a maximum of two iterations.

[0024] Preferably, the construction of the straightening control model in step S5 includes: adopting an MIMO neural network architecture and inputting the data of bending feature parameters; the output layer generates a three-dimensional control vector of pressure, stroke, and direction through the Softmax activation function; establishing a saturation constraint for control instructions: setting the upper and lower limits of pressure and stroke to prevent equipment overload.

[0025] Preferably, in step S6, determining whether the straightness meets the standard specifically includes rough judgment, precise judgment, and judgment in combination with area deviation; rough judgment: judging the bending angle of the steel wire after adjusting the straightening control model, and if it is less than the steel wire bending angle threshold, the rough judgment is passed; precise judgment: for the part that passes the rough judgment, further judge whether its curvature radius is less than the curvature radius threshold and whether the coordinate fluctuation range is less than the coordinate fluctuation range threshold; finally, considering the area deviation, if the area deviation is less than the area deviation threshold and the precise judgment is passed, the straightening meets the standard.

[0026] Compared with the prior art, the advantages and positive effects of the present invention are as follows: in terms of image acquisition and processing, multi-view acquisition combined with image augmentation is used to obtain rich data, and image enhancement operations utilize multi-scale feature extraction, fusion, and reconstruction technologies to improve image quality and feature extraction accuracy. The improved YOLOV8 architecture can accurately output bending feature data by adding a lightweight curvature estimation branch. The straightening control model adopts an MIMO neural network architecture, combines the Softmax activation function and saturation constraints for control instructions, and can intelligently output the control parameters of the straightening equipment. The straightness judgment combines rough judgment, precise judgment, and area deviation judgment to accurately determine the straightening effect. The overall technology effectively overcomes the deficiencies of traditional methods, improves the straightening accuracy and efficiency, enhances the adaptability to different steel wires, and ensures the straightening quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the following drawings are some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0028] Figure 1 It is the overall structure flow chart of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0029] In order to more clearly understand the above-mentioned objects, features, and advantages of the present invention, the following will further describe the present invention with reference to the drawings and embodiments. It should be noted that, without conflict, the embodiments of the present application and the features in the embodiments can be combined with each other.

[0030] In the following description, many specific details are set forth in order to provide a thorough understanding of the present invention. However, the present invention may be practiced in other ways than those described herein, and thus, the present invention is not limited by the limitations of the specific embodiments disclosed in the following specification.

[0031] Embodiment. In the steel wire production and processing industry, the straightness of the steel wire directly affects its subsequent application quality. Traditional mechanical straightening has low efficiency and poor adaptability. Manual straightening relies on experience and has unstable accuracy. Existing machine vision-based straightening methods also have problems such as a single image acquisition perspective, backward image processing algorithms, and a lack of intelligence in the straightening control model. In order to improve the straightening accuracy of the steel wire and enhance the straightening efficiency, the present invention adopts a steel wire straightening method based on deep learning, and the overall implementation process is as Figure 1 shown.

[0032] First, multi-perspective image acquisition. The surface image of the steel wire is acquired by a high-frame-rate industrial camera; the acquisition of the surface image is to acquire the side surface image around the side surface of the steel wire and the top surface image at the top of the steel wire to form a multi-perspective image dataset.

[0033] After image acquisition, an operation for expanding the multi-perspective image dataset is performed. For the side representation image, taking the central axis of the steel wire as the rotation axis, it is rotated at intervals of 10 degrees, and then translation and scaling are performed to generate a new side surface image; for the top surface image, translation and scaling are performed; the two operations are combined to expand the multi-perspective image dataset. Specifically, for the side surface image, using image processing software, taking the central axis of the steel wire as the rotation axis, starting from the initial angle, it is rotated 10 degrees clockwise each time to generate a new image until it is rotated 360 degrees to obtain a series of side surface images at different angles and perform translation and scaling operations. For the top surface image, translation is performed separately in the horizontal and vertical directions, and the translation distance is set to 5%-10% of the image side length; at the same time, scaling operation is performed, and the scaling ratio is between 0.8 and 1.2. The generated new side surface images and the processed top surface images are integrated into the original dataset to achieve the expansion of the multi-perspective image dataset, increase data diversity, and improve the model training effect.

[0034] Then, perform image enhancement operations, including multi-scale feature extraction, feature fusion, and reconstruction of the enhanced image; Multi-scale feature extraction and fusion: Perform multi-scale feature extraction on multi-view images. After normalizing each layer of feature maps, add them according to weights to obtain the fused feature map; Reconstruct the enhanced image: Use transposed convolution operations to upsample and reconstruct the fused feature map into the enhanced image. Specifically, the multi-scale feature extraction and fusion specifically include: Construct a multi-branch feature extraction module: Use three groups of convolution kernels set in parallel to extract features of different scales respectively; Dynamic weight fusion mechanism: Generate the fusion weight coefficients of each scale feature through a channel attention network. The calculation formula is: , where is the weight coefficient, is the global average pooling value of each branch feature map; Cross-scale feature calibration: Perform deformable convolution operations on the fused feature map, and use the offset prediction network to compensate for geometric distortions under different perspectives. Specifically, first construct an offset prediction network, which usually consists of multiple convolutional layers. Input the fused feature map into this network, and the network will learn the geometric distortion rules of the feature map under different perspectives and output offsets. These offsets represent the offset degree of each position in the feature map relative to the normal position. Then, perform deformable convolution operations. Based on the traditional convolution, adjust the sampling positions of the convolution kernel according to the offsets output by the offset prediction network. The original regular sampling points will move to new positions according to the offsets, so as to better adapt to the geometric distribution of the features under different perspectives. The process of reconstructing the enhanced image includes: Hybrid upsampling module: Combine transposed convolution with nearest neighbor interpolation to construct a dual-path reconstruction structure: , where Y is the enhanced image, is the transposed convolution, is the upsampling of 2 times nearest neighbor interpolation, is the element-wise multiplication, and Mask represents the binary mask generated by spatial attention; Residual detail restoration: Introduce skip connections during the reconstruction process to fuse the high-frequency components of the original input image with the reconstruction result. The calculation method is: , where is the finally enhanced image, HPF represents the high-pass filtering operation, is the convolution operation, and W is the learnable convolution kernel; Iterative optimization: When the local contrast variance of the reconstructed image is less than the set variance threshold, perform the feature re-extraction process, with a maximum of two iterations.

[0035] Then, analyze the side surface pictures after image enhancement and improve the YOLOV8 architecture. The improvement of the YOLOV8 architecture is to add a lightweight curvature estimation branch before the convolutional layer. Specifically, Input feature map: Output from the backbone network; Local gradient calculation: Input feature map After Depthwise separable convolution is used to extract local horizontal gradients and local vertical gradients , , , where , ; Calculate the curvature tensor: Based on the local gradients and calculate the second-order derivatives and synthesize the curvature tensor K: ; Dynamic weight generation: Input the curvature tensor K into a multi-layer perceptron, and generate dynamic convolution kernel weights through the Sigmoid activation function and per-channel multiplication operation. The calculation method is: , where represents the initial convolution kernel weight, is the dynamic convolution kernel weight; Feature fusion output: Perform weighted fusion on the results of static convolution and dynamic convolution to obtain the output feature map ; Polar coordinate transformation: Use as the input for polar coordinate feature encoding. With the center of the detection box as the pole, convert the feature map in the Cartesian coordinate system to polar coordinate representation to obtain the polar coordinate feature map ; Polar coordinate convolution: Perform radial and angular convolution operations on the polar coordinate feature map to obtain the radial convolution result and the angular convolution result, and then concatenate the two to obtain the processed polar coordinate feature map ; Inverse transformation: Restore the polar coordinate feature map to the Cartesian coordinate system through bilinear interpolation to obtain , and then add it element-wise to the original input feature map to obtain the final output feature map ; Input the obtained final output feature map into a specific regression network to output the bending feature data. This regression network has been pre-trained with a large amount of labeled wire image data and has learned the mapping relationship between the feature map and the bending features. Inside the network, through the calculations of multiple convolutional and fully connected layers, in-depth analysis of the feature map is performed. The output layer uses specific activation functions and calculation methods to output the bending angle, curvature radius, and position coordinates respectively.

[0036] For the top surface image after image enhancement, calculate the top surface contour area and compare it with the top area of the straightened wire to obtain the area deviation. Specifically, use the edge detection algorithm to extract the top surface contour, determine the pixel points on the contour, and then use the pixel counting method to calculate the top surface contour area. Store the top area data of the straightened wire in the database, directly call this data to subtract the currently calculated area, and obtain the area deviation.

[0037] After obtaining the above-mentioned bending characteristic parameters, a straightening control model is constructed to output the pressure, stroke, and direction control of the straightening device. Specifically, an MIMO neural network architecture is adopted, and the bending characteristic parameter data is input; the output layer generates a three-dimensional control vector of pressure, stroke, and direction through the Softmax activation function; a control instruction saturation constraint is established: the upper and lower limits of pressure and stroke are set to prevent equipment overload.

[0038] Then, the straightness compliance judgment part is carried out, which specifically includes rough judgment, fine judgment, and combined with area deviation judgment; rough judgment: judge the bending angle of the steel wire after adjusting the straightening control model, and if it is less than the steel wire bending angle threshold, the rough judgment is passed; fine judgment: for the part that passes the rough judgment, judge whether its curvature radius is less than the curvature radius threshold and whether the coordinate fluctuation range is less than the coordinate fluctuation range threshold; finally, considering the area deviation, if the area deviation is less than the area deviation threshold and the fine judgment is passed, the straightening is considered compliant. Specifically, during the straightening process, the straightness compliance judgment is divided into rough judgment, fine judgment, and combined with area deviation judgment. During rough judgment, the bending angle data of the steel wire after adjusting the straightening control model is obtained in real time. A bending angle threshold is preset, and the actual bending angle is compared with this threshold. If the bending angle is less than the threshold, it indicates that the bending degree of the steel wire has been significantly improved after preliminary straightening, and it is determined that the rough judgment is passed; if it is greater than the threshold, straightening needs to continue. After passing the rough judgment, it enters the fine judgment link. For the steel wire that passes the rough judgment, obtain its curvature radius data and coordinate fluctuation range data, and also set the curvature radius threshold and coordinate fluctuation range threshold. When the curvature radius of the steel wire is less than the curvature radius threshold and the coordinate fluctuation range is less than the coordinate fluctuation range threshold, it means that the bending condition of the steel wire is close to the ideal state, and the fine judgment is passed; if either condition is not met, the straightening parameters need to be adjusted again for straightening. Finally, the area deviation is combined for judgment. Calculate the surface contour area of the top of the steel wire after straightening, and compare it with the top area of the straightened steel wire to obtain the area deviation. Set the area deviation threshold. If the area deviation is less than this threshold and the fine judgment also passes, it is comprehensively determined that the straightening of the steel wire is compliant, and the straightening operation can be stopped; otherwise, continue to optimize the straightening process until the straightening standard is reached.

[0039] The above is only a preferred embodiment of the present invention, and it is not a limitation of the present invention in other forms. Any person skilled in the art may use the disclosed technical content to make changes or modifications into equivalent embodiments with equivalent changes and apply them to other fields. However, as long as it does not depart from the technical solution content of the present invention, any simple modification, equivalent change, and modification made to the above embodiments based on the technical essence of the present invention still fall within the protection scope of the technical solution of the present invention.

Claims

1. A steel wire straightening method based on deep learning, characterized in that: The following steps are involved: S1, multi-view image acquisition, collecting the surface image of the steel wire through a high frame rate industrial camera; the surface image acquisition is to collect the side surface image around the side surface of the steel wire and collect the top surface image at the top of the steel wire to form a multi-view image data set; S2, performing image enhancement operation on the multi-view images; S3, the enhanced side surface image is input into the improved YOLOV8 architecture, and the bending feature data is output after analysis, including the bending angle, curvature radius and position coordinates; The improvement of the YOLOV8 architecture is to add a lightweight curvature estimation branch before the convolution layer, specifically: Input feature map: Output from the backbone network; Local gradient calculation: input feature map go through Depthwise Separable Convolution to Extract Local Horizontal Gradients and the local vertical gradient , , ,in , ; Calculating the curvature tensor: based on local gradients and Calculate the second-order derivative and synthesize the curvature tensor K: ; Dynamic weight generation: The curvature tensor K is input into the multi-layer perceptron, and the dynamic convolution kernel weight is generated through the Sigmoid activation function and channel-by-channel multiplication operation. The calculation method is: ,in represents the initial convolution kernel weight, is the dynamic convolution kernel weight; Feature fusion output: The results of static convolution and dynamic convolution are weighted and fused to obtain the output feature map ; Polar coordinate transformation: As the input of polar coordinate feature encoding, the center of the detection box is taken as the pole, and the feature map in the Cartesian coordinate system is Convert to polar coordinates to get polar coordinate feature map ; Polar coordinate convolution: polar coordinate feature map Perform radial and angular convolution operations to obtain radial convolution results and angular convolution results, and then splice the two to obtain the processed polar coordinate feature map ; Inverse transformation: Transform the polar coordinate feature map through bilinear interpolation Restore to the Cartesian coordinate system, and we get , and then compare it with the original input feature map Add element by element to get the final output feature map ; S4. For the top surface image after image enhancement, calculate the top surface contour area, and compare it with the top area of ​​the straightened wire to obtain the area deviation; S5. After obtaining the bending characteristic parameters, a straightening control model is constructed to output the pressure, stroke and direction control of the straightening equipment; S6. During the straightening process, the bending characteristic parameters are updated in real time, and combined with the real-time area deviation of the top surface, it is determined whether the straightness meets the standard.

2. A steel wire straightening method based on deep learning according to claim 1, characterized in that: Between step S1 of collecting the steel wire surface image and step S2 of image enhancement, there is also an image expansion operation, specifically: For the side representation image, the central axis of the wire is used as the rotation axis, and the image is rotated at intervals of 10 degrees, and then translated and scaled to generate a new side surface image; For the top surface image, pan and zoom are performed; The two operations are combined to expand the multi-view image dataset.

3. A steel wire straightening method based on deep learning according to claim 1, characterized in that: The image enhancement operation in step S2 includes: multi-scale feature extraction, feature fusion and reconstruction of enhanced image; Multi-scale feature extraction and fusion: Multi-scale feature extraction is performed on multi-view images. After normalizing the feature maps of each layer, they are added according to the weights to obtain the fused feature maps. Reconstruct the enhanced image: Use the transposed convolution operation to upsample the fused feature map and reconstruct it into an enhanced image.

4. A steel wire straightening method based on deep learning according to claim 3, characterized in that: The multi-scale feature extraction and fusion in step S2 specifically include: Construct a multi-branch feature extraction module: use three sets of convolution kernels set in parallel to extract features of different scales respectively; Dynamic weight fusion mechanism: The fusion weight coefficients of each scale feature are generated through the channel attention network. The calculation formula is: ,in is the weight coefficient, is the global average pooling value of each branch feature map; Cross-scale feature calibration: Perform deformable convolution operations on the fused feature maps and use the offset prediction network to compensate for geometric distortions under different viewpoints.

5. A steel wire straightening method based on deep learning according to claim 3, characterized in that: The process of reconstructing the enhanced image in step S2 includes: Hybrid upsampling module: Combine transposed convolution with nearest neighbor interpolation to build a dual-path reconstruction structure: , where Y is the enhanced image, is the transposed convolution, is 2 times the upsampling of the nearest neighbor interpolation, It is an element-by-element multiplication, where Mask represents the binary mask generated by spatial attention; Residual detail restoration: Introduce jump connections in the reconstruction process to fuse the high-frequency components of the original input image with the reconstruction result. The calculation method is: ,in For the final enhanced image, HPF stands for high-pass filtering operation. is the convolution operation, W is the learnable convolution kernel; Iterative optimization: When the local contrast variance of the reconstructed image is less than the set variance threshold, the feature re-extraction process is performed, with a maximum of two iterations.

6. A steel wire straightening method based on deep learning according to claim 1, characterized in that: The step S5 of constructing the straightening control model includes: Using MIMO neural network architecture, input bending feature parameter data; The output layer generates a three-dimensional control vector of pressure, stroke, and direction through the Softmax activation function; Establish control command saturation constraints: set upper and lower limits for pressure and stroke to prevent equipment overload.

7. A steel wire straightening method based on deep learning according to claim 1, characterized in that: The step S6 determines whether the straightness meets the standard, specifically including rough judgment, precise judgment and combined with area deviation judgment; Rough judgment: judge the bending angle of the steel wire after the adjustment of the straightening control model. If it is less than the threshold of the bending angle of the steel wire, it is roughly judged to have passed; Precise judgment: for the part that passes the rough judgment, judge whether its curvature radius is less than the curvature radius threshold and whether the coordinate fluctuation range is less than the coordinate fluctuation range threshold; Finally, the area deviation is considered. If the area deviation is less than the area deviation threshold and the precision judgment passes, the alignment is up to standard.

Images