Compression spring detection method and system based on machine vision

Abstract

The invention discloses a compression spring detection method and system based on machine vision, and the method comprises the steps: firstly, carrying out the calculation and detection of the inner and outer diameters and the free height through median filtering, HSV image segmentation, Canny edge extraction, and least square fitting of a circle and a minimum enclosing rectangle based on spring image collection; and then detecting the position of an end thickness key point by adopting improved yov8-pose, and detecting the thickness of the end. According to the method, the traditional image processing and deep learning technology are combined, the size of the compression spring is detected, the defects of manual detection are effectively avoided, and the detection precision of the spring is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of machine vision detection, and in particular to a compression spring detection method and system based on machine vision. Background Art

[0002] Compression springs have many different sizes, difficult size detection, inaccurate test results, and a complex detection process. But at the same time, the various dimensional parameters of compression springs have a significant impact on the performance of the springs, and are important indicators for quality inspection before submission to users. It is necessary to screen the compression springs according to their dimensional tolerances when processing and leaving the factory, and a large amount of manpower and material resources will be consumed in the spring inspection stage. At present, factory inspection of springs generally uses tooling manpower for inspection, and contact inspection is performed using tools such as vernier calipers and feeler gauges. Compared with automated inspection, the accuracy of manual inspection is easily affected by the current state of the inspection worker, and manual inspection is inefficient and has large errors. For some smaller springs, measurements can only be performed after destruction. Summary of the invention

[0003] In response to the problems existing in the prior art, the present invention provides a compression spring detection method and system based on machine vision, which adopts a method combining traditional image processing and deep learning technology to perform online intelligent detection of the inner diameter, outer diameter, free height, and end thickness of the compression spring, effectively avoiding the shortcomings of manual detection.

[0004] To achieve the above object, the technical solution adopted by the present invention is: A compression spring detection method based on machine vision comprises the following steps: Step 1: Place the spring to be tested into the testing area and turn the spring clamp to clamp the spring; Step 2: Use camera No. 1 to collect pictures of the spring in the top view direction, and perform image processing on the collected top view; use the Canny operator to perform edge detection on the processed top view, and use the least squares method to fit the circle to the detected edge to measure the inner and outer diameters of the compression spring; Step 3: Obtain the position of the spring end in the top view through the detection algorithm, calculate the rotation angle of the turntable, and control the rotation of the turntable so that the thickness section of the end faces the camera No. 1; Step 4: Use the second camera to collect pictures of the main view direction of the spring, detect the key point positions of the compression spring end through the trained yolov8-pose neural network, and calculate the end thickness; perform image processing on the collected main view, use the Canny operator to perform edge detection on the pre-processed main view to obtain the spring contour, and use the minimum circumscribed rectangle algorithm for the contour to obtain the free height; Step 5: Display the processed images and results in the user interface and store relevant data.

[0005] Furthermore, the step 2 comprises the following steps: Step 2.1: Use median filtering on the acquired top view to remove noise in the image; Step 2.2, convert the RGB image into HSV color space, and set the thresholds of the H, S, and V channels according to the color characteristics of the spring and the background, and obtain the spring contour target image through threshold segmentation; Step 2.3, extract the edge of the spring contour target image through the Canny operator to obtain the edge pixels of the compressed spring; Step 2.4: Obtain the inner diameter and outer diameter of the spring by fitting a circle to the obtained spring edge pixels using the least squares method.

[0006] Furthermore, the step 3 comprises the following steps: Step 3.1, determine the center of the circle according to the inner and outer diameters of the compression spring measured in step 2, draw rays outward from the center of the circle at intervals of 1°, calculate the point with the smallest distance between each ray and the inner and outer diameter contours, and take it as the intersection point of the ray and the inner and outer contours, and the distance between the two intersection points as the intercept; Step 3.2, calculate the point with the largest difference among the adjacent intercepts, record it as the end position, calculate the angle between the ray where the end position is located and the vertical straight line, which is the rotation angle of the turntable; Step 3.3: Control the stepper motor turntable to rotate to the specified position through PLC.

[0007] Furthermore, step 4 includes the following steps: Step 4.1: After the turntable completes its rotation, control the second camera to shoot; Step 4.2, use the yolov8-pose neural network to detect key points of the main view obtained, detect and obtain the tip thickness prediction points, and calculate the tip thickness through the prediction points; Step 4.3: Perform image processing on the main view obtained by the acquisition using the method in step 2, and after obtaining the spring contour, use the minimum circumscribed rectangle algorithm to obtain the free length.

[0008] Furthermore, the step 4.2 specifically includes: the method of calculating the end thickness through the predicted points is: detecting the three key points A, B, and C of the spring end through the yolov8-pose neural network, where AB is the upper endpoint and point C is the lower endpoint; after the deep learning neural network completes the detection, the coordinates of the three key points will be given, and the distance from point C to AB is the end thickness; the algorithm calculates the straight line passing through AB based on the coordinates of points AB, and then calculates the distance from the point to the straight line to obtain the size of the end thickness.

[0009] Furthermore, the yolov8-pose neural network includes a backbone network, a neck network and a head network; wherein the feature extraction layer in the backbone network adopts a FasterBLOCK module, and the feature fusion part in the neck network adopts an SPD-C2f module to replace the original convolution core.

[0010] A compression spring detection system based on machine vision comprises an image acquisition device and an image processing terminal; the image acquisition device comprises a base, and a main bracket and an auxiliary bracket mounted on the base, a turntable is provided at the bottom of the main bracket, a spring clamp is installed on the turntable, a compression spring is fixed on the spring clamp, a camera No. 1 is provided on the main bracket and above the turntable, the camera No. 1 is used to acquire a top view of the compression spring; a camera No. 2 is provided on the auxiliary bracket, the camera No. 2 is used to acquire a front view of the compression spring; the image processing terminal is connected to the camera No. 1, the camera No. 2 and the turntable by signal.

[0011] Furthermore, the compression spring detection system also includes a lighting system, which includes a ring light source and a back light source. The ring light source is installed on the main bracket and is located between the camera No. 1 and the turntable, and the middle of the ring light source has a through hole; the back light source is installed on the base and is arranged opposite to the camera No. 2.

[0012] Furthermore, the first camera is a Hikvision MV-CU120-10-GC, and the lens is a dual telecentric lens.

[0013] Furthermore, the second camera is Hikvision MV-CU120-10-GC.

[0014] Compared with the prior art, the present invention has the following advantages or beneficial effects: The present invention provides a compression spring detection method based on machine vision. The method is based on spring image acquisition. First, median filtering, HSV image segmentation, Canny edge extraction and least square method fitting circle and minimum circumscribed rectangle calculation are used to detect the inner and outer diameters and free heights. Then, the improved yolov8-pose is used to detect the key point position of the end thickness and the end thickness. The compression spring is dimensionally detected by combining traditional image processing with deep learning technology, which effectively avoids the shortcomings of manual detection and improves the detection accuracy of the spring. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying creative work.

[0016] Figure 1 It is a schematic diagram of the image acquisition device in the present invention; Figure 2 It is a detection flow chart of the present invention; Figure 3 It is a flow chart of the detection algorithm in the present invention; Figure 4 It is a diagram illustrating the tip position detection algorithm in the present invention; Figure 5 Improved yolov8-pose neural network diagram in the present invention; Figure 6 This is a diagram illustrating the key point detection algorithm in the present invention; Figure 7 This is an interface diagram of the image processing terminal in the present invention; In the figure, 1. base, 2. main bracket, 3. auxiliary bracket, 4. turntable, 5. spring clamp, 6. camera No. 1, 7. camera No. 2, 8. ring light source, 9. back light source. DETAILED DESCRIPTION

[0017] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments.

[0018] This embodiment provides a compression spring detection system based on machine vision, such as Figure 1 As shown, the system includes an image acquisition device and an image processing terminal. The image acquisition device includes a base 1, and a main bracket 2 and an auxiliary bracket 3 installed on the base 1. A turntable 4 is provided at the bottom of the main bracket 2, and a spring clamp 5 is installed on the turntable 4. The turntable 4 drives the spring clamp 5 to rotate, and the compression spring is fixed on the spring clamp 5. A camera No. 1 is provided on the main bracket 2 and above the turntable 4. The camera No. 1 is used to capture a top view of the compression spring. The auxiliary bracket 3 is provided with a camera No. 2. The camera No. 2 is used to capture a front view of the compression spring. The image processing terminal is connected to the camera No. 1 6, the camera No. 2, and the turntable 4 by signal, and is used to remotely control the two cameras to shoot, and the turntable rotates to the corresponding position.

[0019] Further explanation, the above-mentioned image acquisition device is also equipped with a lighting system. Specifically, the lighting system includes a ring light source 8 and a back light source 9. The ring light source 8 is installed on the main bracket 2 and is located between the first camera 6 and the turntable 4. The middle of the ring light source 8 has a through hole. The back light source 9 is installed on the base 1 and is arranged opposite to the second camera 7. The lighting system is used to improve the clarity of the picture taken.

[0020] This embodiment uses the above system to detect the compression spring. Figure 2 , 3 As shown, the detection method comprises the following steps: 1. Collect images: Image acquisition is performed through two industrial cameras. Camera No. 1 6 is responsible for acquiring the top view of the spring and detecting the inner and outer diameter lengths from the top view. Camera No. 2 7 is responsible for acquiring the main view of the spring and acquiring the free height and end thickness from the main view. Preferably, the model of Camera No. 1 6 is Hikvision MV-CU120-10-GC, and the lens is a dual telecentric lens; the model of Camera No. 2 7 is Hikvision MV-CU120-10-GC, and the lens is an ordinary lens. The lighting system uses a combined lighting system of 30° ring light source high-angle illumination and back lighting to improve image clarity.

[0021] 2. Processing of top view: S2.1. After comparing several commonly used filtering methods, median filtering is selected. Median filtering is a commonly used nonlinear filtering technology. Its basic principle is to sort the pixels in the image and then take the median value to replace the current pixel value. Since median filtering does not directly average the pixels, but takes the median of the local area, it can better maintain the edge information of the image while removing noise, and will not cause image blur like mean filtering, but it is easy to cause image discontinuity. Median filtering is expressed by the following formula: , In the formula is the original image, is the filtered image, is the pixel point that coincides with the template center, Covers the area for the template.

[0022] S2.2. Convert the original image color space into the HSV color space. The purpose is to intuitively express the brightness, hue and vividness of the image color, and to facilitate the determination of the value range of the three channels of a certain color in the HSV space. The color space conversion calculation formula used is: Among them, r, g, and b are the values ​​of the RGB channels, h, s, and v are the values ​​of the HSV channels, and max and min are the maximum and minimum values ​​of r, g, and b respectively.

[0023] S2.3. Use the Canny operator to perform edge extraction on the spring binary image in order to extract the edge contour in the top view of the spring.

[0024] S2.4. Use the least squares method to fit the edge contour. According to the distance between the center of the fitted circle and the center of the image, eliminate the incorrectly fitted circles with too large distance to obtain the inner diameter and outer diameter.

[0025] 3. Acquisition and control of turntable rotation angle: S3.1. Draw rays outward every 1° from the center coordinates obtained in the previous step, such as Figure 4 As shown, A is the intersection of the ray and the inner diameter contour, B is the intersection of the ray and the outer diameter contour, and AB is the intercept of the ray on the contour. It is worth noting that due to the imaging principle of the acquired image, the extracted contour information is not a continuous function, so points A and B are the points in the contour closest to the ray.

[0026] S3.2. Each ray has a corresponding AB intercept. The difference between adjacent intercepts is calculated. The position with the largest absolute value of the difference indicates that the intercept has changed dramatically at that position. Due to the imaging principle of the end position, there is a large difference in the roughness of the upper and lower surfaces, and a large difference in the degree of reflection under illumination, so the position where the intercept changes dramatically is the end position. Calculate the angle between the line connecting this position and the center of the circle and the vertical line, which is the rotation angle.

[0027] S3.3, the processing terminal communicates with the PLC via the Snap7 protocol and controls the rotation of the turntable by reading and writing the memory that controls the rotation position of the turntable.

[0028] 4. Processing of the main view: S4.1. After the turntable completes its rotation, the processing terminal controls the second camera to take a picture of the main view of the spring.

[0029] S4.2, use the improved yolov8-pose neural network to detect key points. The improved neural network is as follows Figure 5As shown. The FasterBLOCK module is used to improve the feature extraction layer and reduce the computational complexity. The SPD-C2f module is used to replace the original convolution core in the feature fusion part to improve the feature response to small targets. The ATFL loss function is used to make the model focus more on small targets rather than the background. FasterNet is an efficient neural network architecture designed to improve computing speed without sacrificing accuracy, especially in visual tasks. It reduces redundant calculations and memory accesses through a new technology called partial convolution (PConv). This approach allows FasterNet to run much faster than other networks on a variety of devices while maintaining high accuracy in various visual tasks. FasterBLOCK is part of the FasterNet network, which consists of 1 Pconv and two subsequent 1×1 pointwise convolutions (PointwiseConvolution). The three form an inverted residual architecture with more channels in the middle layer and a shortcut connection placed to reuse input features. Replace the second Conv module in Bottleneck with SPD-Conv (spatial to depth convolution) to reduce information loss. SPD-Conv is a technique that converts image spatial information into depth information, so that convolutional neural networks (CNNs) can learn image features more effectively. This method optimizes the model's ability to handle small objects and low-resolution images by reducing information loss and improving the accuracy of feature extraction. The ATFL loss function first uses threshold setting to decouple easily recognizable backgrounds from difficult-to-recognize targets; secondly, by strengthening the loss associated with the target and reducing the loss associated with the background, it forces the model to allocate more attention to the target features, thereby alleviating the imbalance between the target and the background. The loss function expression is: The end thickness detection method is: improve the yolov8-pose neural network to detect the three key points A, B, and C of the spring end position, where AB is the upper end point and C is the lower end point. Figure 6 As shown. After the deep learning neural network completes the detection, it will give the coordinates of the three key points. The distance from point C to AB is the end thickness. The algorithm calculates the straight line through AB based on the coordinates of the two points AB, and then calculates the distance from the point to the straight line to obtain the end thickness.

[0030] S4.3. After obtaining the image edge contour using median filtering, HSV image segmentation, and Canny filtering on the main view image, the free height is detected using the minimum enclosing rectangle.

[0031] 5. Image processing terminal interface:

[0032] The interface of the image processing terminal is as follows Figure 7As shown, the interface is divided into two parts, the upper part is the inner and outer diameter detection area, and the lower part is the length end thickness detection area. Each part has a picture display area, a result display area, an operation area, and a parameter modification area. In the actual detection process, the operator needs to click the buttons in the operation area from top to bottom to complete the detection process. Each time an operation is completed, the image display area on the left will display the result of the image processing. If the background separation effect is poor in the image segmentation operation, you can click the HSV parameter setting in the parameter modification area to enter the modification interface, and improve the image segmentation effect by adjusting the upper and lower limits of the three parameters H, S, and V; if it is found that the end position cannot be accurately detected during the end position detection process, you can enter the control interface by clicking the turntable manual control button. After clicking the inner and outer diameter detection, free length detection, and end detection, the software will display the processed detection results in the corresponding display box below the picture. After completing all the detection items, you can enter the current detection spring number and write the detection results into the log to complete the detection.

[0033] The present invention can detect the inner diameter, outer diameter, free height and end thickness of the compression spring, wherein the inner and outer diameters and free height have an accuracy of 0.05 mm and the end thickness has an accuracy of 0.02 mm after experimental verification.

[0034] The above specific implementations are only descriptions of the preferred implementations of the present invention, and do not limit the protection scope of the present invention. Without departing from the design concept and spirit of the present invention, various modifications, substitutions and improvements made by ordinary technicians in this field to the technical solution of the present invention based on the text description and drawings provided by the present invention should all fall within the protection scope of the present invention.

Claims

1. A compression spring detection method based on machine vision, characterized in that: The following steps are involved: Step 1: Place the spring to be tested into the testing area and turn the spring clamp to clamp the spring; Step 2: Use camera No. 1 to collect pictures of the spring in the top view direction, and perform image processing on the collected top view; use the Canny operator to perform edge detection on the processed top view, and use the least squares method to fit the circle to the detected edge to measure the inner and outer diameters of the compression spring; Step 3: Obtain the position of the spring end in the top view through the detection algorithm, calculate the rotation angle of the turntable, and control the rotation of the turntable so that the thickness section of the end faces the camera No. 1; Step 4: Use the second camera to collect pictures of the main view direction of the spring, detect the key point positions of the compression spring end through the trained yolov8-pose neural network, and calculate the end thickness; perform image processing on the collected main view, use the Canny operator to perform edge detection on the pre-processed main view to obtain the spring contour, and use the minimum circumscribed rectangle algorithm for the contour to obtain the free height; Step 5: Display the processed images and results in the user interface and store relevant data.

2. A compression spring detection method based on machine vision according to claim 1, characterized in that: The step 2 specifically includes: Step 2.1: Use median filtering on the acquired top view to remove noise in the image; Step 2.2, convert the RGB image into HSV color space, and set the thresholds of the H, S, and V channels according to the color characteristics of the spring and the background, and obtain the spring contour target image through threshold segmentation; Step 2.3, extract the edge of the spring contour target image through the Canny operator to obtain the edge pixels of the compressed spring; Step 2.4: Obtain the inner diameter and outer diameter of the spring by fitting a circle to the obtained spring edge pixels using the least squares method.

3. The compression spring detection method based on machine vision according to claim 1 is characterized in that: The step 3 specifically includes: Step 3.1, determine the center of the circle according to the inner and outer diameters of the compression spring measured in step 2, draw rays outward from the center of the circle at intervals of 1°, calculate the point with the smallest distance between each ray and the inner and outer diameter contours, and take it as the intersection point of the ray and the inner and outer contours, and the distance between the two intersection points as the intercept; Step 3.2, calculate the point with the largest difference among the adjacent intercepts, record it as the end position, calculate the angle between the ray where the end position is located and the vertical straight line, which is the rotation angle of the turntable; Step 3.3: Control the stepper motor turntable to rotate to the specified position through PLC.

4. The compression spring detection method based on machine vision according to claim 2 is characterized in that: The step 4 specifically includes: Step 4.1: After the turntable completes its rotation, control the second camera to shoot; Step 4.2, use the yolov8-pose neural network to detect key points of the main view obtained, detect and obtain the tip thickness prediction points, and calculate the tip thickness through the prediction points; Step 4.3: Perform image processing on the main view acquired by the acquisition using the method in step 2, and after obtaining the spring contour, use the minimum circumscribed rectangle algorithm to obtain the free length.

5. The method for detecting a compression spring based on machine vision according to claim 4, characterized in that: The step 4.2 specifically includes: the method of calculating the end thickness through the predicted points is: using the yolov8-pose neural network to detect the three key points A, B, and C of the spring end, where AB is the upper endpoint and C is the lower endpoint; after the deep learning neural network completes the detection, the coordinates of the three key points will be given, and the distance from point C to AB is the end thickness; the algorithm calculates the straight line passing through AB based on the coordinates of the two points AB, and then calculates the distance from the point to the straight line to obtain the size of the end thickness.

6. The compression spring detection method based on machine vision according to claim 4 is characterized in that: The yolov8-pose neural network includes a backbone network, a neck network and a head network; wherein the feature extraction layer in the backbone network adopts the FasterBLOCK module, and the feature fusion part in the neck network adopts the SPD-C2f module to replace the original convolution core.

7. A compression spring detection system based on machine vision, the detection system is used to perform the detection method according to any one of claims 1 to 6, characterized in that: The detection system includes an image acquisition device and an image processing terminal; the image acquisition device includes a base, and a main bracket and an auxiliary bracket installed on the base, a turntable is provided at the bottom of the main bracket, a spring clamp is installed on the turntable, a compression spring is fixed on the spring clamp, a camera No. 1 is provided on the main bracket and above the turntable, and the camera No. 1 is used to capture a top view of the compression spring; a camera No. 2 is provided on the auxiliary bracket, and the camera No. 2 is used to capture a front view of the compression spring; the image processing terminal is connected to the camera No. 1, the camera No. 2 and the turntable signal.

8. The compression spring detection system based on machine vision according to claim 7, characterized in that: It also includes a lighting system, which includes a ring light source and a back light source. The ring light source is installed on the main bracket and is located between the first camera and the turntable. The middle of the ring light source has a through hole. The back light source is installed on the base and is arranged opposite to the second camera.

9. The compression spring detection system based on machine vision according to claim 7, characterized in that: The camera No. 1 is Hikvision MV-CU120-10-GC, and the lens is a double telecentric lens.

10. The compression spring detection system based on machine vision according to claim 7, characterized in that: The second camera is Hikvision MV-CU120-10-GC.

Images