A method for detecting surface defects of cylindrical rollers based on machine vision

Abstract

The present application belongs to the field of target detection of computer vision detection, and particularly relates to a cylindrical roller surface defect detection method based on machine vision, which comprises the following steps: collecting cylindrical roller surface defect pictures and marking; making a data set for the marked file and picture, and expanding the data set; improving the residual component in the backbone network of the YOLOv5s model and replacing the activation function; changing the loss function CIoU to SIoU; packaging the improved YOLOv5s model after training the improved model to obtain the best weight file; packaging the Halcon operator; making an interface for the packaged operator on the QT framework, and detecting the cylindrical roller surface defect by using the improved YOLOv5s inference model. The present application detects the cylindrical roller surface defect by using the packaged detection operator and the improved YOLOv5s model, greatly increases the detection accuracy, and improves the production efficiency.

Description

Technical Field

[0001] This invention relates to the field of target detection using computer vision, specifically a method for detecting surface defects on cylindrical rollers based on machine vision. Background Technology

[0002] Cylindrical roller bearings are mainly used in industrial equipment such as compressors, pumps, wind power equipment, gearboxes, and unbalanced vibrators. They are designed for heavy loads and high speeds, and can withstand axial displacement, making them an indispensable part of the industrial field. As one of the key components of cylindrical roller bearings, the surface quality of the cylindrical roller directly affects the working performance and service life of the bearing.

[0003] Currently, the screening of roller surface defects mainly relies on manual inspection. However, workers are prone to eye fatigue from prolonged exposure to bright light reflecting off metal surfaces to inspect for defects, leading to a higher rate of missed defects. This results in defective rollers being used in the production of roller bearings, causing a decline in bearing performance, substandard product quality, and even major safety accidents involving industrial equipment. Furthermore, manual inspection of roller surface defects is inefficient and costly, making it unsuitable for batch inspection of rollers.

[0004] In recent years, with the rapid development of machine vision technology, the field of inspection has begun to apply this technology to replace manual labor. However, traditional machine vision algorithms have low accuracy in detecting minute defects and low classification accuracy. Summary of the Invention

[0005] Therefore, to address the shortcomings of existing roller inspection methods, this invention provides a roller surface defect detection method based on an improved model of Halcon and YOLOv5s. This method uses Halcon to detect easily detectable defects and YOLOv5s to detect difficult-to-detect defects; this dual detection approach improves detection reliability.

[0006] The technical solution of the present invention is as follows:

[0007] A machine vision-based method for detecting surface defects in cylindrical rollers includes the following steps:

[0008] S1. Collect and label images of surface defects on cylindrical rollers;

[0009] S2, create a dataset from the labeled files and images, and expand the dataset;

[0010] S3, Improve the residual components in the backbone network of the YOLOv5s model and change the activation function;

[0011] S4, the loss function CIoU is replaced with SIoU;

[0012] S5. After training the improved model and obtaining the optimal weight file, encapsulate the improved YOLOv5s model.

[0013] S6 encapsulates the Halcon operator;

[0014] S7 encapsulates the operators and interfaces them on the QT framework. It then uses the encapsulated C++ program and the improved YOLOv5s model to detect surface defects on cylindrical rollers.

[0015] Furthermore, S1 specifically includes: using a high-brightness light source in conjunction with a PCCD012 ring outer lens to capture images of the roller side, using a parallel light source in conjunction with a 50mm focal length lens to capture images of the roller end face, using an upper 30° light source and a lower 45° light source to form an auxiliary light source, using a high-resolution, small-focal-length lens with a camera target size to capture images of the roller side, and using LabelImg annotation software to mark defects in PASCAL VOC format to obtain an XML file, and converting the XML tag file into a TXT tag file.

[0016] Furthermore, S2 specifically includes: dividing the dataset into a training set, a validation set, and a test set in a ratio of 8:1:1; expanding the dataset by one or more of the following methods: flipping, cropping, adding noise, rotating, translating, and mirroring, to prevent underfitting and overfitting during training; and performing Mosaic operations on the images to enrich the defect background, increase defect diversity, and prevent overfitting.

[0017] Further, S3 includes: replacing several residual components in the last C3 module of the YOLOv5s model backbone with several residual components with multi-head attention modules; changing the non-linear activation function Silu in the multi-head attention module of the residual components to the ReLU function; setting the multi-head attention module to 4 heads; first encoding the relative position of the height and width on the feature map of the P5 layer, then adding them element by element to generate relative position encoding information r; multiplying this relative position encoding information r with the query q matrix to obtain a satisfactory position matrix; multiplying the key k with the query q matrix to obtain satisfactory content information, then adding it element by element, and performing Softmax to obtain an output matrix with a range of [0,1]; finally multiplying it with the v matrix to form a single-head attention module.

[0018] Furthermore, the loss function SIoU in S4 is:

[0019]

[0020] Where IoU is the crossover ratio loss, Δ is the distance loss, and Ω is the shape loss.

[0021] Furthermore, S5 includes: generating a .wts sequence file from the trained .pt weight file, then using TensorRT to accelerate inference on the weight file to generate an .engine project file, encapsulating it into a class function, configuring the required environment, and starting to compile it into a dynamic link library and corresponding library files of the trained improved YOLOv5s model in the Release environment, adding the dynamic link library and corresponding library files of the trained improved YOLOv5s model to its own QT detection framework, and connecting the interface of the encapsulated function.

[0022] Furthermore, S6 and S7 include: after writing the Halcon operator, encapsulating it into a detection function, using C++, connecting the function interface and adding it to the QT framework, first using the Halcon operator for detection during deployment, and then using the improved YOLOv5s inference model to detect defects on the cylindrical roller surface.

[0023] Compared with existing technologies, this invention can ensure 100% detection of large defects while improving the detection accuracy of small defects. Moreover, after changing the loss function, the bounding box regression speed is faster and the error is smaller. Attached Figure Description

[0024] Figure 1 This is a flowchart of the present invention;

[0025] Figure 2 and Figure 3 This is a diagram of the improved C3 structure of the present invention, wherein... Figure 2 This is a structural diagram of the bottle transformer of the present invention. Figure 3 This is a structural diagram of the MHSA of the present invention;

[0026] Figure 4 This is a schematic diagram of the angle cost of the present invention;

[0027] Figure 5 This is a comparison diagram of the bounding box convergence effect in this invention;

[0028] Figure 6 The images show the training and validation results of the improved YOLOv5s model in this invention, including (a) training bounding box regression map, (b) training confidence loss map, (c) training classification loss map, (d) precision map, (e) recall map, (f) validation bounding box regression map, (g) validation confidence loss map, (h) validation classification loss map, (i) accuracy map with a confidence threshold of 0.5, and (j) accuracy map with a confidence threshold from 0.5 to 0.95.

[0029] Figure 7 This is a diagram showing the detection and prediction results in this invention. Detailed Implementation

[0030] The technical solution of the present invention will be further described below with reference to the accompanying drawings and specific embodiments. Obviously, the described embodiments are only some embodiments of the present invention, but are not intended to limit the present invention.

[0031] Please see Figures 1-7 A machine vision-based method for detecting surface defects in cylindrical rollers includes the following steps:

[0032] S1, capturing images of the roller surface, including the side, end, and chamfer. The roller side image is captured using a high-brightness MPBG-100100-HW light source manufactured by Shanghai Moran Optoelectronics Co., Ltd., paired with an OPTO PCCD012 lens (outer ring lens). This lens can capture images of cylindrical rollers with a diameter-to-height ratio of approximately 3:2. The roller end face image is captured using a parallel light source paired with a 50mm focal length lens. The chamfer image capture scheme uses an upper 30° light source and a lower 45° light source to form an auxiliary light source, illuminating the blind spots of the chamfer area, and is equipped with a high-resolution, short focal length lens adapted to the camera's target size.

[0033] S2, Creating and Expanding Datasets.

[0034] A dataset was created using LabelImg annotation software, with defects labeled in PASCAL VOC format. Since different acquisition schemes were used to photograph different parts of the roller surface with industrial cameras, the background and morphological characteristics of the same defect varied slightly in different locations. Therefore, three datasets were created based on the roller surface: sides, ends, and chamfers, to improve detection accuracy. Side defects were further categorized into scratches, corrosion, and dents; end defects into scratches, corrosion, and dents; and chamfer defects into missing corners. The acquired images underwent initial screening, selecting clear images with obvious defects for labeling. The labeled XML tag files were then converted to TXT tag files, and the dataset was divided into training, validation, and test sets in an 8:1:1 ratio.

[0035] To expand the dataset, the labeled files are statistically analyzed. If the number of defects of a certain type falls below a certain threshold, the dataset is expanded a certain number of times using methods such as flipping, cropping, adding noise, rotating, translating, and mirroring to make the number of defects for each type approximately similar. The total number of defects for each type is expanded to 500 images to prevent underfitting during training. Mosaic enhancement is then applied to the dataset to prevent overfitting.

[0036] S3 replaces several residual components in the last C3 module of the YOLOv5s model backbone with several residual components with multi-head attention. The original residual components consisted of two 1x1 kernel convolutions with a stride of 1 and a 3x3 kernel convolution with a stride of 2. The improved residual components replace the 3x3 kernel convolutions with strides of 2 with multi-head attention modules. This splits the data shape into 4 heads, allowing the learning of 4 different features and enhancing the model's expressive learning ability. The relative positional encoding of the height and width on the feature map of the layer is first performed to obtain R. h and R w Then, element-wise addition is performed to generate relative position encoding information r. This relative position encoding information r is multiplied by the query matrix q to obtain the satisfactory position matrix qr. T On the other hand, multiplying the key k by the query yields the desired content information qk. T Later with qr T Add each element together and perform Softmax to obtain an output matrix in the range [0,1]. Finally, multiply the output matrix by the v matrix.

[0037] The linear normalization after convolution in the C3 module was changed to batch normalization. The non-linear activation function Silu in the original multi-attention module was changed to ReLU to make training faster and prevent gradient vanishing.

[0038] S4, change the loss function CIoU to SIoU. The loss function SIoU consists of angle loss, distance loss, shape loss, and crossover ratio (CRO).

[0039]

[0040] Where IoU is the crossover ratio loss, Δ is the distance loss, and Ω is the shape loss.

[0041] The purpose of adding the angle loss component is to decompose the distance σ between the center points of the predicted box and the ground truth box into x-axis and y-axis values, gradually moving closer along either the X or Y axis. The angle between the line connecting the centers of the predicted and ground truth boxes and the X-axis is α, and the angle with the Y-axis is β. α + β = 90°. During convergence, α is minimized first, otherwise β is minimized. To achieve this, a component Λ is introduced:

[0042]

[0043] in,

[0044]

[0045]

[0046]

[0047] Distance loss is defined as Δ

[0048]

[0049] in,

[0050]

[0051] In the above formula (b) cx ,b cy ), (b cx gt b cx gt ) represent the coordinates of the center point of the predicted bounding box and the center point of the ground truth bounding box, respectively. w To determine the horizontal distance between the center point of the predicted bounding box and the center point of the ground truth bounding box, c h σ is the vertical distance between the center point of the predicted bounding box and the center point of the ground truth bounding box;

[0052] Shape loss is defined as Ω:

[0053]

[0054] in,

[0055] w, h, w gt ,h gt These represent the width of the predicted bounding box, the height of the predicted bounding box, the width of the ground truth bounding box, and the height of the ground truth bounding box, respectively.

[0056] The value of θ is crucial in determining the magnitude of the shape loss, and its value is unique. Through experiments, this value was set to 4.

[0057] Crossover Union (IoU) loss is defined as:

[0058]

[0059] Where B is the size of the prediction box, B GT The actual box size;

[0060] The final regression loss function is:

[0061]

[0062] In step S5, the training set is fed into the improved YOLOv5s model, and training begins with parameter settings. Training parameters are set with batchsize of 32, work of 6, and images compressed to 640x640. Because a multi-head attention mechanism is used, the optimizer is first Adam for rapid gradient descent, followed by SGD tuning. Hyperparameters are set with momentum of 0.937 and cosine annealing decay weight_decay of 0.0005. An empty C++ project is created using Visual Studio 2017, and the header and library files for TensorRT, OpenCV, QT, and CUDA are added to the project environment. The optimal weight file (.pt) obtained from training is serialized into a .wts file, and then TensorRT is used to accelerate and improve model inference, generating the engine file. The improved model is encapsulated into a class containing an initialization function and an inference function. After encapsulating the functions, the class is compiled to generate a dynamic link library and the corresponding library files.

[0063] S6 encapsulates the detection operators from the Halcon software into a function and then exports it as a C++ program.

[0064] S7 interfaces the pre-packaged detection functions written in Halcon software and the improved YOLOv5s encapsulation model with the QT framework. The detection operators exported from the pre-packaged C++ program and the improved YOLOv5s model are then used to detect defects in images that were not detected by traditional detection methods.

[0065] The foregoing general description of the invention and its specific embodiments should not be construed as a limitation on the technical solution of the invention. Those skilled in the art, based on the disclosure of this application, can add, reduce, or combine the disclosed technical features in the foregoing general description or specific embodiments (including examples) without departing from the constituent elements of the invention, to form other technical solutions within the scope of protection of this application.

Claims

1. A method for detecting surface defects in cylindrical rollers based on machine vision, characterized in that, Includes the following steps: S1. Collect and label images of surface defects on cylindrical rollers; S2, create a dataset from the labeled files and images, and expand the dataset; S3, improve the residual components in the backbone network of the YOLOv5s model and change the activation function, including: replacing several residual components in the last C3 module of the YOLOv5s model backbone with several residual components with multi-head attention modules, changing the non-linear activation function Silu in the multi-head attention module of the residual components to the ReLU function, and setting the multi-head attention module to 4 heads. On the feature map of the P5 layer, the height and width are first encoded relative to each other, and then the results are added element by element to generate relative position encoding information r. This relative position encoding information r is multiplied with the query q matrix to obtain a satisfactory position matrix; the key k is multiplied with the query q matrix to obtain satisfactory content information, and then added element by element and Softmax is performed to obtain an output matrix with a range of [0,1]. Finally, it is multiplied with the v matrix to form a single-head attention module. S4. Change the loss function CIoU to SIoU. The loss function SIoU consists of angle loss, distance loss, shape loss, and cross-union ratio. The loss function SIoU is: ， Where IoU is the crossover ratio loss, Δ is the distance loss, and Ω is the shape loss; The purpose of adding the angle loss component is to decompose the distance σ between the center points of the predicted box and the ground truth box into x-axis and y-axis values, gradually moving closer along the X or Y axis; where the angle between the line connecting the centers of the predicted box and the ground truth box and the X-axis is α, and the angle with the Y-axis is β; α + β = 90°, if During convergence, α is minimized first, otherwise β is minimized; a component Λ is introduced: ， in, ， Distance loss is defined as Δ ， in, ， In the above formula (b) cx ,b cy ) ， (b) cx gt ， b cx gt ) represent the coordinates of the center point of the predicted bounding box and the center point of the ground truth bounding box, respectively. w To determine the horizontal distance between the center point of the predicted bounding box and the center point of the ground truth bounding box, c h σ is the vertical distance between the center point of the predicted bounding box and the center point of the ground truth bounding box; Shape loss is defined as Ω: ， in, w, h, w gt ,h gt These represent the width of the predicted bounding box, the height of the predicted bounding box, the width of the ground truth bounding box, and the height of the ground truth bounding box, respectively. The value of θ is crucial in determining the magnitude of the shape loss, and its value is unique. Through experiments, this value was set to 4. S5. After training the improved model and obtaining the optimal weight file, encapsulate the improved YOLOv5s model. S6, the detection operator written in Halcon software is exported as a C++ program and encapsulated; S7: The encapsulated C++ program and the improved YOLOv5s encapsulation model are interfaced on the QT framework, and the encapsulated C++ program and the improved YOLOv5s model are used to detect surface defects of cylindrical rollers.

2. The method for detecting surface defects of cylindrical rollers based on machine vision according to claim 1, characterized in that, S1 specifically includes: using a high-brightness light source in conjunction with a PCCD012 ring outer lens to take pictures of the side of the roller; using a parallel light source in conjunction with a 50mm focal length lens to take pictures of the end face of the roller; using an upper 30° light source and a lower 45° light source to form an auxiliary light source; using a high-resolution, small focal length lens with a camera target size to take pictures of the side of the roller; and using LabelImg annotation software to mark defects in PASCAL VOC format to obtain an XML file.

3. The method for detecting surface defects of cylindrical rollers based on machine vision according to claim 1, characterized in that, S2 specifically includes: dividing the dataset into a training set, a validation set, and a test set, and expanding the dataset by one or more of the following methods: flipping, cropping, adding noise, rotating, translating, and mirroring.

4. The method for detecting surface defects of cylindrical rollers based on machine vision according to claim 3, characterized in that, S5 includes: feeding the training set into the improved YOLOv5s model for training, obtaining the optimal weight file .pt through training, generating a .wts sequence file from the optimal weight file .pt, using TensorRT to accelerate inference on the weight file to generate an .engine project file, encapsulating it into a class function, configuring the required environment, and starting compilation in the Release environment into a dynamic link library of the trained improved YOLOv5s model and the corresponding library files.

5. The method for detecting surface defects of cylindrical rollers based on machine vision according to claim 1, characterized in that, S6 includes: after writing the Halcon operator, encapsulating it into a detection function, exporting the detection function as a C++ program, connecting the function interface, and adding it to the QT framework.

Images