Steel wire straightening method based on deep learning

Abstract

The invention belongs to the technical field of deep learning, and particularly relates to a steel wire straightening method based on deep learning. According to the method, steel wire images are collected from the side surface and the top through a high-frame-rate industrial camera, after image expansion and enhancement processing, the enhanced side surface images are input into an improved YOLOV8 architecture to obtain bending feature data, and meanwhile the area deviation of the top surface images is calculated. And constructing an MIMO neural network straightening control model by utilizing the bending characteristic parameters, outputting pressure, stroke and direction control instructions of straightening equipment, and judging whether the straightness reaches the standard or not by combining rough judgment and fine judgment with the area deviation in the straightening process. Compared with a traditional straightening method, the steel wire bending characteristic can be comprehensively obtained, the straightening process is precisely controlled, and the straightening precision and efficiency are effectively improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of deep learning, and in particular relates to a steel wire straightening method based on deep learning. Background Art

[0002] In the field of steel wire production and processing, the straightness of the steel wire plays a key role in its subsequent application. Traditional steel wire straightening methods have many disadvantages. The mechanical straightening method mainly relies on mechanical rolling, which is not only inefficient, but also has poor adaptability to steel wires of different materials and specifications, and it is difficult to ensure the straightening accuracy. Manual straightening is highly dependent on the experience of workers. It is not only labor-intensive, but also has unstable accuracy and is easily affected by human factors. With the development of technology, straightening methods based on machine vision have begun to appear, but existing methods still have defects. On the one hand, image acquisition is mostly single-view, and it is impossible to fully obtain the bending characteristics of the steel wire, resulting in inaccurate straightening parameters. On the other hand, the image processing algorithm is not advanced enough, and its ability to recognize the bending characteristics of the steel wire in complex environments is limited, and its robustness is insufficient. At the same time, the straightening control model lacks intelligence, and it is difficult to make precise adjustments according to the real-time status of the steel wire. Summary of the invention

[0003] In view of the technical problems existing in the background technology, the present invention proposes a steel wire straightening method based on deep learning.

[0004] In order to achieve the above object, the technical solution adopted by the present invention comprises the following steps:

[0005] S1, multi-view image acquisition, collecting the surface image of the steel wire through a high frame rate industrial camera; the surface image acquisition is to collect the side surface image around the side surface of the steel wire and collect the top surface image at the top of the steel wire to form a multi-view image data set;

[0006] S2, performing image enhancement operation on the multi-view images;

[0007] S3, the enhanced side surface image is input into the improved YOLOV8 architecture, and the bending feature data is output after analysis, including the bending angle, curvature radius and position coordinates;

[0008] The improvement of the YOLOV8 architecture is to add a lightweight curvature estimation branch before the convolution layer, specifically:

[0009] Input feature map: Output from the backbone network;

[0010] Local gradient calculation: input feature map go through Depthwise Separable Convolution to Extract Local Horizontal Gradients and the local vertical gradient , , ,in , ;

[0011] Calculating the curvature tensor: based on local gradients and Calculate the second-order derivative and synthesize the curvature tensor K: ;

[0012] Dynamic weight generation: The curvature tensor K is input into the multi-layer perceptron, and the dynamic convolution kernel weight is generated through the Sigmoid activation function and channel-by-channel multiplication operation. The calculation method is: ,in represents the initial convolution kernel weight, is the dynamic convolution kernel weight;

[0013] Feature fusion output: The results of static convolution and dynamic convolution are weighted and fused to obtain the output feature map ;

[0014] Polar coordinate transformation: As the input of polar coordinate feature encoding, the center of the detection box is taken as the pole, and the feature map in the Cartesian coordinate system is Convert to polar coordinates to get polar coordinate feature map ;

[0015] Polar coordinate convolution: polar coordinate feature map Perform radial and angular convolution operations to obtain radial convolution results and angular convolution results, and then splice the two to obtain the processed polar coordinate feature map ;

[0016] Inverse transformation: Transform the polar coordinate feature map through bilinear interpolation Restore to the Cartesian coordinate system, and we get , and then compare it with the original input feature map Add element by element to get the final output feature map ;

[0017] S4. For the top surface image after image enhancement, calculate the top surface contour area, and compare it with the top area of ​​the straightened wire to obtain the area deviation;

[0018] S5. After obtaining the bending characteristic parameters, a straightening control model is constructed to output the pressure, stroke and direction control of the straightening equipment;

[0019] S6. During the straightening process, the bending characteristic parameters are updated in real time, and combined with the real-time area deviation of the top surface, it is determined whether the straightness meets the standard.

[0020] Preferably, between step S1 of collecting the wire surface image and step S2 of image enhancement, there is also an image expansion operation, specifically: for the side representation image, the central axis of the wire is used as the rotation axis, and it is rotated at intervals of 10 degrees, and then translated and scaled to generate a new side surface image; for the top surface image, translation and scaling are performed; and the two operations are combined to expand the multi-view image data set.

[0021] Preferably, the image enhancement operation in step S2 includes: multi-scale feature extraction, feature fusion and reconstruction of enhanced image; multi-scale feature extraction and fusion: multi-scale feature extraction is performed on the multi-view image, and the feature maps of each layer are normalized and then added according to the weights to obtain the fused feature map; and enhanced image reconstruction: using the transposed convolution operation, the fused feature map is upsampled and reconstructed into an enhanced image.

[0022] Preferably, the multi-scale feature extraction and fusion in step S2 specifically includes: constructing a multi-branch feature extraction module: using three sets of convolution kernels set in parallel to extract features of different scales respectively; dynamic weight fusion mechanism: generating fusion weight coefficients of features of each scale through a channel attention network, and the calculation formula is: ,in is the weight coefficient, is the global average pooling value of the feature maps of each branch; Cross-scale feature calibration: perform deformable convolution operations on the fused feature maps, and use the offset prediction network to compensate for geometric distortions under different perspectives.

[0023] Preferably, the process of reconstructing the enhanced image in step S2 comprises: a hybrid upsampling module: combining transposed convolution with nearest neighbor interpolation to construct a dual-path reconstruction structure: , where Y is the enhanced image, is the transposed convolution, is 2 times the upsampling of the nearest neighbor interpolation, is element-by-element multiplication, Mask represents the binary mask generated by spatial attention; Residual detail recovery: In the reconstruction process, jump connections are introduced to fuse the high-frequency components of the original input image with the reconstruction result. The calculation method is: ,in For the final enhanced image, HPF stands for high-pass filtering operation. is a convolution operation, W is a learnable convolution kernel; iterative optimization: when the local contrast variance of the reconstructed image is less than the set variance threshold, the feature re-extraction process is performed, with a maximum of two iterations.

[0024] Preferably, the straightening control model construction in step S5 includes: adopting a MIMO neural network architecture and inputting bending feature parameter data; the output layer generates a three-dimensional control vector of pressure, stroke, and direction through a Softmax activation function; and establishing a control instruction saturation constraint: setting upper and lower limits of pressure and stroke to prevent equipment overload.

[0025] Preferably, the step S6 determines whether the straightness meets the standard, specifically including rough judgment, precise judgment and combined with area deviation judgment; rough judgment: judging the bending angle of the steel wire after the straightening control model is adjusted, and if it is less than the steel wire bending angle threshold, the rough judgment passes; precise judgment: for the part that passes the rough judgment, judging whether its curvature radius is less than the curvature radius threshold and whether the coordinate fluctuation range is less than the coordinate fluctuation range threshold; finally, considering the area deviation, if the area deviation is less than the area deviation threshold and the precise judgment passes, the straightening meets the standard.

[0026] Compared with the prior art, the advantages and positive effects of the present invention are that, in terms of image acquisition and processing, multi-view acquisition is combined with image expansion to obtain rich data, and the image enhancement operation uses multi-scale feature extraction, fusion and reconstruction technology to improve image quality and feature extraction accuracy. The improved YOLOV8 architecture can accurately output bending feature data by adding a lightweight curvature estimation branch. The straightening control model adopts a MIMO neural network architecture, combined with the Softmax activation function and the control instruction saturation constraint, and can intelligently output the control parameters of the straightening equipment. The straightness judgment integrates rough judgment, fine judgment and area deviation judgment to accurately determine the straightening effect. The overall technology effectively overcomes the shortcomings of traditional methods, improves straightening accuracy and efficiency, enhances adaptability to different steel wires, and ensures straightening quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0028] Figure 1 It is the overall structure flow chart of the present invention. DETAILED DESCRIPTION

[0029] In order to more clearly understand the above-mentioned purpose, features and advantages of the present invention, the present invention is further described below in conjunction with the accompanying drawings and embodiments. It should be noted that the embodiments of the present application and the features in the embodiments can be combined with each other without conflict.

[0030] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways than those described herein. Therefore, the present invention is not limited to the specific embodiments of the following disclosure.

[0031] Embodiment: In the steel wire production and processing industry, the straightness of the steel wire is directly related to its subsequent application quality. Traditional mechanical straightening has low efficiency and poor adaptability, manual straightening relies on experience and has unstable accuracy, and existing machine vision-based straightening methods also have problems such as single image acquisition perspective, backward image processing algorithm, and lack of intelligence in the straightening control model. In order to improve the accuracy of steel wire straightening and enhance the straightening efficiency. The present invention adopts a steel wire straightening method based on deep learning. The overall implementation process is as follows: Figure 1 shown.

[0032] First, multi-view image acquisition is performed, and the surface image of the steel wire is acquired by a high frame rate industrial camera; the surface image is acquired by acquiring side surface images around the side surface of the steel wire and top surface images at the top of the steel wire, forming a multi-view image data set.

[0033] After image acquisition, the multi-view image dataset is expanded. For the side representation image, the central axis of the wire is used as the rotation axis, and it is rotated at intervals of 10 degrees, and then translated and scaled to generate a new side surface image; for the top surface image, translation and scaling are performed; the two operations are combined to expand the multi-view image dataset. Specifically, for the side surface image, the image processing software is used to rotate 10 degrees clockwise each time with the central axis of the wire as the rotation axis, starting from the initial angle, until it rotates 360 degrees, and a series of side surface images at different angles are obtained and translated and scaled. For the top surface image, translation is performed in the horizontal and vertical directions respectively, and the translation distance is set to 5%-10% of the image side length; scaling is performed at the same time, and the scaling ratio is between 0.8-1.2. The generated new side surface image and the processed top surface image are integrated into the original data set to achieve the expansion of the multi-view image data set, increase data diversity, and improve the model training effect.

[0034] Then, image enhancement operations are performed, including multi-scale feature extraction, feature fusion, and reconstruction of enhanced images; multi-scale feature extraction and fusion: multi-scale feature extraction is performed on multi-view images, and the feature maps of each layer are normalized and then added according to the weights to obtain the fused feature maps; the enhanced image is reconstructed: the fused feature maps are upsampled and reconstructed into enhanced images using the transposed convolution operation. Specifically, the multi-scale feature extraction and fusion specifically include: constructing a multi-branch feature extraction module: using three sets of convolution kernels set in parallel to extract features of different scales respectively; dynamic weight fusion mechanism: generating the fusion weight coefficients of the features of each scale through the channel attention network, and the calculation formula is: ,in is the weight coefficient, is the global average pooling value of the feature maps of each branch; Cross-scale feature calibration: Perform deformable convolution operations on the fused feature maps, and use the offset prediction network to compensate for geometric distortions under different perspectives. Specifically, first construct an offset prediction network, which is usually composed of multiple convolutional layers. The fused feature map is input into the network, and the network will learn the geometric distortion rules of the feature maps under different perspectives and output the offsets. These offsets represent the degree of offset of each position in the feature map relative to the normal position. Next, perform a deformable convolution operation. On the basis of traditional convolution, the convolution kernel sampling position is adjusted according to the offset output by the offset prediction network. The originally regular sampling points will be moved to the new position according to the offset, so that they can better adapt to the geometric distribution of features under different perspectives. The process of reconstructing enhanced images includes: Hybrid upsampling module: Combine transposed convolution with nearest neighbor interpolation to construct a dual-path reconstruction structure: , where Y is the enhanced image, is the transposed convolution, is 2 times the upsampling of the nearest neighbor interpolation, is element-by-element multiplication, Mask represents the binary mask generated by spatial attention; Residual detail recovery: In the reconstruction process, jump connections are introduced to fuse the high-frequency components of the original input image with the reconstruction result. The calculation method is: ,in For the final enhanced image, HPF stands for high-pass filtering operation. is a convolution operation, W is a learnable convolution kernel; iterative optimization: when the local contrast variance of the reconstructed image is less than the set variance threshold, the feature re-extraction process is performed, with a maximum of two iterations.

[0035] Then, the enhanced side surface image is analyzed to improve the YOLOV8 architecture by adding a lightweight curvature estimation branch before the convolution layer. Specifically, the input feature map is: Output from backbone network; local gradient calculation: input feature map go through Depthwise Separable Convolution to Extract Local Horizontal Gradients and the local vertical gradient , , ,in , ; Calculate the curvature tensor: based on local gradient and Calculate the second-order derivative and synthesize the curvature tensor K: ; Dynamic weight generation: Input the curvature tensor K into the multi-layer perceptron, and generate dynamic convolution kernel weights through the Sigmoid activation function and channel-by-channel multiplication operation. The calculation method is: ,in represents the initial convolution kernel weight, is the dynamic convolution kernel weight; feature fusion output: the results of static convolution and dynamic convolution are weighted and fused to obtain the output feature map ; Polar coordinate transformation: As the input of polar coordinate feature encoding, the center of the detection box is taken as the pole, and the feature map in the Cartesian coordinate system is Convert to polar coordinates to get polar coordinate feature map ; Polar coordinate convolution: polar coordinate feature map Perform radial and angular convolution operations to obtain radial convolution results and angular convolution results, and then splice the two to obtain the processed polar coordinate feature map ; Inverse transformation: Use bilinear interpolation to transform the polar coordinate feature map Restore to the Cartesian coordinate system, and we get , and then compare it with the original input feature map Add element by element to get the final output feature map ; The final output feature map The input is sent to a specific regression network to output the bending feature data. The regression network has been pre-trained with a large amount of annotated wire image data to learn the mapping relationship between the feature map and the bending feature. Inside the network, the feature map is deeply analyzed through multi-layer convolution and full connection layer calculations. The output layer uses a specific activation function and calculation method to output the bending angle, curvature radius, and position coordinates.

[0036] For the top surface image after image enhancement, the top surface contour area is calculated and compared with the top area of ​​the straightened wire to obtain the area deviation. Specifically, the top surface contour is extracted using the edge detection algorithm, the pixel points on the contour are determined, and then the top surface contour area is calculated using the pixel counting method. The top area data of the straightened wire is stored in the database, and the data is directly called to subtract from the currently calculated area to obtain the area deviation.

[0037] After obtaining the bending characteristic parameters as above, the straightening control model is constructed to output the pressure, stroke and direction control of the straightening equipment. Specifically, the MIMO neural network architecture is adopted to input the bending characteristic parameter data; the output layer generates a three-dimensional control vector of pressure, stroke and direction through the Softmax activation function; and the control instruction saturation constraint is established: the upper and lower limits of pressure and stroke are set to prevent equipment overload.

[0038] Then the straightness standard judgment part is carried out, which specifically includes rough judgment, fine judgment and combined area deviation judgment; rough judgment: judge the bending angle of the steel wire after the adjustment of the straightening control model, and it is rough judged if it is less than the threshold of the steel wire bending angle; fine judgment: for the part that passes the rough judgment, judge whether its curvature radius is less than the curvature radius threshold and whether the coordinate fluctuation range is less than the coordinate fluctuation range threshold; finally consider the area deviation, if the area deviation is less than the area deviation threshold and the fine judgment passes, the straightening is up to standard. Specifically, in the straightening process, the straightness standard judgment is divided into rough judgment, fine judgment and combined area deviation judgment. During the rough judgment, the bending angle data of the steel wire after the adjustment of the straightening control model is obtained in real time. The bending angle threshold is set in advance, and the actual bending angle is compared with the threshold. If the bending angle is less than the threshold, it indicates that the bending degree of the steel wire has been significantly improved after the initial straightening, and the rough judgment is judged to be passed; if it is greater than the threshold, it is necessary to continue straightening. After the rough judgment is passed, enter the fine judgment link. For the steel wire that passes the rough judgment, obtain its curvature radius data and coordinate fluctuation range data, and also set the curvature radius threshold and coordinate fluctuation range threshold. When the curvature radius of the steel wire is less than the curvature radius threshold, and the coordinate fluctuation range is less than the coordinate fluctuation range threshold, it means that the bending condition of the steel wire is close to the ideal state and the precision judgment is passed; if any of the conditions is not met, it is necessary to adjust the straightening parameters again for straightening. Finally, the area deviation is combined for judgment. Calculate the surface contour area of ​​the top of the steel wire after straightening, and compare it with the top area of ​​the straightened steel wire to obtain the area deviation. Set the area deviation threshold. If the area deviation is less than the threshold and the precision judgment is passed, it is comprehensively judged that the steel wire straightening meets the standard, and the straightening operation can be stopped; otherwise, continue to optimize the straightening process until the straightening standard is met.

[0039] The above description is only a preferred embodiment of the present invention and does not limit the present invention in other forms. Any technician familiar with the profession may use the technical content disclosed above to change or modify it into an equivalent embodiment with equivalent changes and apply it to other fields. However, any simple modification, equivalent change and modification made to the above embodiment based on the technical essence of the present invention without departing from the content of the technical solution of the present invention still falls within the protection scope of the technical solution of the present invention.

Claims

1. A steel wire straightening method based on deep learning, characterized in that: The following steps are involved: S1, multi-view image acquisition, collecting the surface image of the steel wire through a high frame rate industrial camera; the surface image acquisition is to collect the side surface image around the side surface of the steel wire and collect the top surface image at the top of the steel wire to form a multi-view image data set; S2, performing image enhancement operation on the multi-view images; S3, the enhanced side surface image is input into the improved YOLOV8 architecture, and the bending feature data is output after analysis, including the bending angle, curvature radius and position coordinates; The improvement of the YOLOV8 architecture is to add a lightweight curvature estimation branch before the convolution layer, specifically: Input feature map: Output from the backbone network; Local gradient calculation: input feature map go through Depthwise Separable Convolution to Extract Local Horizontal Gradients and the local vertical gradient , , ,in , ; Calculating the curvature tensor: based on local gradients and Calculate the second-order derivative and synthesize the curvature tensor K: ; Dynamic weight generation: The curvature tensor K is input into the multi-layer perceptron, and the dynamic convolution kernel weight is generated through the Sigmoid activation function and channel-by-channel multiplication operation. The calculation method is: ,in represents the initial convolution kernel weight, is the dynamic convolution kernel weight; Feature fusion output: The results of static convolution and dynamic convolution are weighted and fused to obtain the output feature map ; Polar coordinate transformation: As the input of polar coordinate feature encoding, the center of the detection box is taken as the pole, and the feature map in the Cartesian coordinate system is Convert to polar coordinates to get polar coordinate feature map ; Polar coordinate convolution: polar coordinate feature map Perform radial and angular convolution operations to obtain radial convolution results and angular convolution results, and then splice the two to obtain the processed polar coordinate feature map ; Inverse transformation: Transform the polar coordinate feature map through bilinear interpolation Restore to the Cartesian coordinate system, and we get , and then compare it with the original input feature map Add element by element to get the final output feature map ; S4. For the top surface image after image enhancement, calculate the top surface contour area, and compare it with the top area of ​​the straightened wire to obtain the area deviation; S5. After obtaining the bending characteristic parameters, a straightening control model is constructed to output the pressure, stroke and direction control of the straightening equipment; S6. During the straightening process, the bending characteristic parameters are updated in real time, and combined with the real-time area deviation of the top surface, it is determined whether the straightness meets the standard.

2. A steel wire straightening method based on deep learning according to claim 1, characterized in that: Between step S1 of collecting the steel wire surface image and step S2 of image enhancement, there is also an image expansion operation, specifically: For the side representation image, the central axis of the wire is used as the rotation axis, and the image is rotated at intervals of 10 degrees, and then translated and scaled to generate a new side surface image; For the top surface image, pan and zoom are performed; The two operations are combined to expand the multi-view image dataset.

3. A steel wire straightening method based on deep learning according to claim 1, characterized in that: The image enhancement operation in step S2 includes: multi-scale feature extraction, feature fusion and reconstruction of enhanced image; Multi-scale feature extraction and fusion: Multi-scale feature extraction is performed on multi-view images. After normalizing the feature maps of each layer, they are added according to the weights to obtain the fused feature maps. Reconstruct the enhanced image: Use the transposed convolution operation to upsample the fused feature map and reconstruct it into an enhanced image.

4. A steel wire straightening method based on deep learning according to claim 3, characterized in that: The multi-scale feature extraction and fusion in step S2 specifically include: Construct a multi-branch feature extraction module: use three sets of convolution kernels set in parallel to extract features of different scales respectively; Dynamic weight fusion mechanism: The fusion weight coefficients of each scale feature are generated through the channel attention network. The calculation formula is: ,in is the weight coefficient, is the global average pooling value of each branch feature map; Cross-scale feature calibration: Perform deformable convolution operations on the fused feature maps and use the offset prediction network to compensate for geometric distortions under different viewpoints.

5. A steel wire straightening method based on deep learning according to claim 3, characterized in that: The process of reconstructing the enhanced image in step S2 includes: Hybrid upsampling module: Combine transposed convolution with nearest neighbor interpolation to build a dual-path reconstruction structure: , where Y is the enhanced image, is the transposed convolution, is 2 times the upsampling of the nearest neighbor interpolation, It is an element-by-element multiplication, where Mask represents the binary mask generated by spatial attention; Residual detail restoration: Introduce jump connections in the reconstruction process to fuse the high-frequency components of the original input image with the reconstruction result. The calculation method is: ,in For the final enhanced image, HPF stands for high-pass filtering operation. is the convolution operation, W is the learnable convolution kernel; Iterative optimization: When the local contrast variance of the reconstructed image is less than the set variance threshold, the feature re-extraction process is performed, with a maximum of two iterations.

6. A steel wire straightening method based on deep learning according to claim 1, characterized in that: The step S5 of constructing the straightening control model includes: Using MIMO neural network architecture, input bending feature parameter data; The output layer generates a three-dimensional control vector of pressure, stroke, and direction through the Softmax activation function; Establish control command saturation constraints: set upper and lower limits for pressure and stroke to prevent equipment overload.

7. A steel wire straightening method based on deep learning according to claim 1, characterized in that: The step S6 determines whether the straightness meets the standard, specifically including rough judgment, precise judgment and combined with area deviation judgment; Rough judgment: judge the bending angle of the steel wire after the adjustment of the straightening control model. If it is less than the threshold of the bending angle of the steel wire, it is roughly judged to have passed; Precise judgment: for the part that passes the rough judgment, judge whether its curvature radius is less than the curvature radius threshold and whether the coordinate fluctuation range is less than the coordinate fluctuation range threshold; Finally, the area deviation is considered. If the area deviation is less than the area deviation threshold and the precision judgment passes, the alignment is up to standard.

Images