A Compression Spring Detection Method and System Based on Machine Vision

Abstract

The present invention discloses a method and system for detecting compression springs based on machine vision. The detection method is based on spring image acquisition. First, median filtering, HSV image segmentation, Canny edge extraction, least squares method for fitting circles and calculating the minimum circumscribed rectangle are used to detect the inner and outer diameters and free height. Then, an improved yolov8-pose is used to detect the key point positions of the end thickness to detect the end thickness. By combining traditional image processing and deep learning techniques, the size of the compression spring is detected, effectively avoiding the deficiencies of manual detection and improving the detection accuracy of the spring.

Description

Technical Field

[0001] The present invention relates to the technical field of machine vision detection, and particularly relates to a method and system for detecting compression springs based on machine vision. Background Art

[0002] Compression springs have the characteristics of a large variety of sizes, difficult size detection, inaccurate detection results, and a complex detection process. However, at the same time, each dimensional parameter of the compression spring has a significant impact on the performance of the spring and is an important indicator for quality inspection before submitting to users. It is necessary to screen the compression springs according to their dimensional tolerances during the processing and factory production. A large amount of manpower and material resources will be consumed during the spring detection stage. Currently, spring factory inspections generally use tooling and manual labor for detection, and contact detection is carried out using tools such as vernier calipers and feeler gauges. Compared with automated detection, the accuracy of manual detection is easily affected by the current state of the detection workers. Manual detection has low efficiency and large errors. For some springs with smaller sizes, only destructive measurement can be carried out. Summary of the Invention

[0003] Aiming at the problems existing in the prior art, the present invention provides a method and system for detecting compression springs based on machine vision, which adopts a method combining traditional image processing and deep learning technology to perform online intelligent detection on the inner diameter, outer diameter, free height, and end thickness of the compression spring, effectively avoiding the deficiencies of manual detection.

[0004] To achieve the above object, the technical solution adopted by the present invention is as follows:

[0005] A method for detecting compression springs based on machine vision includes the following steps:

[0006] Step 1: Place the spring to be inspected in the detection area, and rotate the spring clamp to clamp the spring;

[0007] Step 2: Use the first camera to collect pictures of the spring in the top view direction, and perform image processing on the collected top view; Use the Canny operator to perform edge detection on the processed top view, and use the least squares method to fit a circle to the detected edge to measure the inner and outer diameters of the compression spring;

[0008] Step 3: Obtain the position of the spring end in the top view through the detection algorithm, calculate the rotation angle of the turntable, and control the turntable to rotate so that the end thickness section is facing the first camera;

[0009] Step 4: Use the second camera to collect pictures of the spring in the front view direction. Detect the key point positions of the end of the compression spring through the trained yolov8-pose neural network, and calculate the thickness of the end; perform image processing on the collected front view, use the Canny operator to perform edge detection on the preprocessed front view to obtain the spring contour, and use the minimum bounding rectangle algorithm on the contour to obtain the free height;

[0010] Step 5: Display the processed image and results in the user operation interface, and store the relevant data.

[0011] Further, the said Step 2 includes the following steps:

[0012] Step 2.1: Use median filtering on the collected top view to remove the noise in the picture;

[0013] Step 2.2: Convert the RGB image to the HSV color space, and set the thresholds for the H, S, and V channels according to the color characteristics of the spring and the background, and obtain the spring contour target image through threshold segmentation;

[0014] Step 2.3: Perform edge extraction on the spring contour target image through the Canny operator to obtain the edge pixels of the compression spring;

[0015] Step 2.4: Fit a circle to the obtained spring edge pixels using the least squares method to obtain the inner diameter and outer diameter of the spring.

[0016] Further, the said Step 3 includes the following steps:

[0017] Step 3.1: Determine the center of the circle according to the inner and outer diameters of the compression spring measured in Step 2. Make rays outward every 1° from the center of the circle, calculate the point on each ray with the smallest distance from the inner and outer diameter contours, and use it as the intersection point of the ray and the inner and outer contours. The distance between the two intersection points is used as the intercept;

[0018] Step 3.2: Calculate the point with the largest difference among adjacent intercepts, record it as the end position, and calculate the angle between the ray where the end position is located and the vertically upward line, which is the rotation angle of the turntable;

[0019] Step 3.3: Control the stepping motor turntable to rotate to the specified position through the PLC.

[0020] Further, the said Step 4 includes the following steps:

[0021] Step 4.1: After the turntable completes rotation, control the second camera to take pictures;

[0022] Step 4.2: Perform key point detection on the collected front view using the yolov8-pose neural network, detect the predicted points of the end thickness, and calculate the end thickness through the predicted points;

[0023] Step 4.3: Use the method in Step 2 to perform image processing on the acquired front view. After obtaining the spring contour, use the minimum bounding rectangle algorithm to obtain the free length.

[0024] Furthermore, the specific steps of Step 4.2 are as follows: The method for calculating the end thickness through predicted points is as follows: Detect three key points A, B, and C of the spring end through the yolov8-pose neural network, where AB is the upper end point and point C is the lower end point; after the deep learning neural network completes the detection, it will give the coordinates of the 3 key points, and the distance from point C to AB is the end thickness; the algorithm calculates the straight line passing through AB based on the coordinates of points A and B, and then calculates the distance from a point to the straight line to obtain the size of the end thickness.

[0025] Furthermore, the yolov8-pose neural network includes a backbone network, a neck network, and a head network; among them, the feature extraction layer in the backbone network uses the FasterBLOCK module, and the feature fusion part in the neck network uses the SPD-C2f module to replace the original convolution core.

[0026] A compression spring detection system based on machine vision includes an image acquisition device and an image processing terminal; the image acquisition device includes a base, a main bracket and an auxiliary bracket installed on the base, a turntable is provided at the bottom of the main bracket, a spring clamp is installed on the turntable, a compression spring is fixed on the spring clamp, a first camera is provided above the turntable on the main bracket, and the first camera is used to collect the top view of the compression spring; a second camera is provided on the auxiliary bracket, and the second camera is used to collect the front view of the compression spring; the image processing terminal is signal-connected to the first camera, the second camera, and the turntable.

[0027] Furthermore, the compression spring detection system further includes an illumination system, the illumination system includes an annular light source and a back light source, the annular light source is installed on the main bracket and between the first camera and the turntable, and the middle of the annular light source has a through hole; the back light source is installed on the base and is disposed opposite to the second camera.

[0028] Furthermore, the first camera is selected as Hikvision MV-CU120-10-GC, and the lens is a double telecentric lens.

[0029] Furthermore, the second camera is selected as Hikvision MV-CU120-10-GC.

[0030] Compared with the prior art, the present invention has the following advantages or beneficial effects:

[0031] The present invention provides a method for detecting compression springs based on machine vision. This method is based on spring image acquisition. First, median filtering, HSV image segmentation, Canny edge extraction, fitting circles by the least squares method, and calculating the minimum circumscribed rectangle are used to detect the inner and outer diameters and the free height. Then, an improved yolov8-pose is used to detect the key point positions of the end thickness and measure the end thickness. By combining traditional image processing and deep learning techniques, the dimensional detection of compression springs is carried out, effectively avoiding the deficiencies of manual detection and improving the detection accuracy of springs. Description of the Drawings

[0032] To more clearly illustrate the technical solutions in the embodiments of the present invention or in the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on the structures shown in these drawings.

[0033] Figure 1 Schematic diagram of the image acquisition device in the present invention;

[0034] Figure 2 Flowchart of the detection in the present invention;

[0035] Figure 3 Flowchart of the detection algorithm in the present invention;

[0036] Figure 4 Explanation diagram of the end position detection algorithm in the present invention;

[0037] Figure 5 Improved yolov8-pose neural network diagram in the present invention;

[0038] Figure 6 Explanation diagram of the key point detection algorithm in the present invention;

[0039] Figure 7 Interface diagram of the image processing terminal in the present invention;

[0040] In the figure, 1 is the base, 2 is the main bracket, 3 is the auxiliary bracket, 4 is the turntable, 5 is the spring clamp, 6 is the first camera, 7 is the second camera, 8 is the annular light source, and 9 is the back light source. Detailed Embodiments

[0041] To make the objectives, technical solutions, and advantages of the present invention clearer and more understandable, the following further elaborates on the present invention in conjunction with the drawings and embodiments.

[0042] This embodiment provides a compression spring detection system based on machine vision, as Figure 1As shown in the figure, the system includes an image acquisition device and an image processing terminal. The image acquisition device includes a base 1, a main bracket 2 and an auxiliary bracket 3 mounted on the base 1. A turntable 4 is provided at the bottom of the main bracket 2, and a spring clamp 5 is mounted on the turntable 4. The turntable 4 drives the spring clamp 5 to rotate. A compression spring is fixed on the spring clamp 5. Above the turntable 4 on the main bracket 2, a first camera 6 is provided, and the first camera 6 is used to collect the top view of the compression spring. The auxiliary bracket 3 is provided with a second camera 7, and the second camera 7 is used to collect the front view of the compression spring. The image processing terminal is signal-connected to the first camera 6, the second camera 7 and the turntable 4, and is used to remotely control the two cameras to take pictures and rotate the turntable to the corresponding position.

[0043] Furthermore, the above-mentioned image acquisition device is also equipped with a lighting system. Specifically, the lighting system includes an annular light source 8 and a back light source 9. The annular light source 8 is mounted on the main bracket 2 and located between the first camera 6 and the turntable 4. The middle part of the annular light source 8 has a through hole. The back light source 9 is mounted on the base 1 and is arranged opposite to the second camera 7. The lighting system is used to improve the clarity of picture taking.

[0044] In this embodiment, the above system is used to detect the compression spring, as Figure 2 、 3 shown, the detection method includes the following steps:

[0045] 1. Image acquisition:

[0046] Image acquisition is performed by two equipped industrial cameras. The first camera 6 is responsible for collecting the top view of the spring, and the inner and outer diameter lengths are detected from the top view. The second camera 7 is responsible for collecting the front view of the spring, and the free height and end thickness are collected from the front view. Preferably, the model of the first camera 6 is Hikvision MV-CU120-10-GC, and the lens is a double telecentric lens; the model of the second camera 7 is Hikvision MV-CU120-10-GC, and the lens is an ordinary lens. The lighting system adopts a combined lighting system of 30° annular light source high-angle illumination and backlighting to improve the image clarity.

[0047] 2. Processing of the top view:

[0048] S2.1. After comparing several common filtering methods, median filtering is selected. Median filtering is a common non-linear filtering technique. Its basic principle is to sort the pixels in the image and then take the median value to replace the value of the current pixel. Since median filtering does not directly perform an average operation on the pixels, but takes the median value of the local area, it can better preserve the edge information of the image while removing noise, and will not cause image blurring like mean filtering. However, it is easy to cause image discontinuity. Median filtering is expressed by the following formula:

[0049] ,

[0050] In the formula is the original image, is the filtered image, is the pixel point where the template center coincides, is the area covered by the template.

[0051] S2.2. Convert the color space of the original image to the HSV color space, aiming to intuitively express the brightness, hue, and vividness of the image color, and facilitating the determination of the value range of the three channels of a certain color in the HSV space. The calculation formula for color space conversion used is:

[0052]

[0053] Among them, r, g, and b are the values of the RGB three channels, h, s, and v are the values of the HSV three channels, and max and min are the maximum and minimum values among the three values of r, g, and b respectively.

[0054] S2.3. Use the Canny operator to extract the edges of the binary image of the spring, aiming to extract the edge contour in the top view of the spring.

[0055] S2.4. Use the least squares method to fit the edge contour. According to the distance between the center of the circle fitted and the center of the image, eliminate the wrongly fitted circles with too large distances to obtain the inner diameter and outer diameter.

[0056] 3. Acquisition and control of the turntable rotation angle:

[0057] S3.1. Draw rays outward every 1° from the center coordinates obtained in the previous step. As Figure 4 shown, where A is the intersection point of the ray and the inner diameter contour, B is the intersection point of the ray and the outer diameter contour, and AB is the intercept of the ray on the contour. It should be noted that due to the imaging principle of the collected image, the extracted contour information is not a continuous function, so points A and B are respectively the points on the contour closest to the ray.

[0058] S3.2. Each ray has a corresponding AB intercept. Calculate the difference between adjacent intercepts. The position where the absolute value of the difference is the largest indicates that the intercept has changed drastically at this position. Due to the imaging principle of the end position, there is a large difference in the surface roughness of the upper and lower surfaces, and there is a large difference in the degree of reflection under light, so the position where the intercept changes drastically is the end position. Calculate the angle between the line connecting this position and the center of the circle and the vertical line, which is the rotation angle.

[0059] S3.3. The processing terminal communicates with the PLC through the Snap7 protocol, and controls the rotation of the turntable by reading and writing the memory that controls the rotation position of the turntable.

[0060] 4. Processing of the front view:

[0061] S4.1 After the turntable completes rotation, control the second camera through the processing terminal to capture the front view picture of the spring.

[0062] S4.2 Use the improved yolov8-pose neural network for key point detection. The improved neural network is as Figure 5 shown. Among them, the FasterBLOCK module is used to improve the feature extraction layer and reduce the computational complexity. In the feature fusion part, the SPD-C2f module is used to replace the original convolution core, which improves the feature response to small targets. The ATFL loss function makes the model pay more attention to small targets rather than the background. FasterNet is an efficient neural network architecture designed to improve computational speed without sacrificing accuracy, especially in visual tasks. It reduces redundant calculations and memory access through a new technology called partial convolution (PConv). This method makes FasterNet run much faster than other networks on various devices while maintaining high accuracy in various visual tasks. FasterBLOCK is a part of the FasterNet network. It consists of 1 Pconv and two subsequent 1×1 pointwise convolutions. These three form an inverted residual architecture with more channels in the middle layer and a Shortcut connection is placed to reuse the input features. Replace the second Conv module in the Bottleneck with SPD-Conv (spatial to depth convolution) to reduce information loss. SPD-Conv is a technology that converts image spatial information into depth information, enabling the convolutional neural network (CNN) to learn image features more effectively. This method optimizes the model's ability to process small objects and low-resolution images by reducing information loss and improving the accuracy of feature extraction. The ATFL loss function first decouples the easily recognizable background from the difficult-to-recognize target using threshold settings; secondly, by strengthening the loss related to the target and reducing the loss related to the background, it forces the model to allocate more attention to the target features, thus alleviating the imbalance between the target and the background. The expression of the loss function is:

[0063]

[0064] The method for detecting the end thickness is as follows: The improved yolov8-pose neural network will detect three key points A, B, and C at the end of the spring. Among them, AB is the upper end point and point C is the lower end point as Figure 6 shown. After the deep learning neural network completes the detection, it will give the coordinates of the 3 key points. The distance from point C to AB is the end thickness. The algorithm calculates the straight line passing through AB based on the coordinates of points A and B, and then calculates the distance from the point to the straight line to obtain the size of the end thickness.

[0065] S4.3. After obtaining the image edge contour of the front view image by using median filtering, HSV image segmentation, and Canny filtering, the free height is detected using the minimum bounding rectangle.

[0066] 5. Image processing terminal interface:

[0067] The interface of the image processing terminal is as Figure 7 shown. The interface is divided into two parts, the upper part is the inner and outer diameter detection area, and the lower part is the length, end thickness detection area. Each part has a picture display area, a result display area, an operation area, and a parameter modification area. In the actual detection process, the operator needs to click the buttons in the operation area from top to bottom in sequence to complete the detection process. After each operation is completed, the result of the image processing for that time will be displayed in the picture display area on the left. If it is found that the background separation effect is poor during the image segmentation operation, the HSV parameter settings in the parameter modification area can be clicked to enter the modification interface, and the upper and lower limits of the three parameters H, S, and V can be adjusted to improve the image segmentation effect; if it is found that the end position cannot be accurately detected during the end position detection and processing, the control interface can be entered by clicking the turntable manual control button. After clicking the inner and outer diameter detection, free length detection, and end detection, the software will display the processed detection results in the corresponding display box below the picture. After all detection items are completed, the current detected spring number can be input and the detection results can be written into the log to complete the detection.

[0068] The present invention can detect the inner diameter, outer diameter, free height, and end thickness of a compression spring. Among them, the accuracy of the inner and outer diameters and the free height has been verified by experiments to reach 0.05 mm, and the accuracy of the end thickness reaches 0.02 mm.

[0069] The above specific embodiments only describe the preferred embodiments of the present invention, rather than limiting the protection scope of the present invention. Without departing from the design concept and spirit scope of the present invention, various deformations, substitutions, and improvements made by those of ordinary skill in the art to the technical solutions of the present invention based on the written description and drawings provided by the present invention shall fall within the protection scope of the present invention.

Claims

1. A method for detecting compression springs based on machine vision, characterized in that, It includes the following steps: Step 1: Place the spring to be inspected in the detection area and rotate the spring clamp to clamp the spring; Step 2: Use the first camera to collect pictures of the spring from the top view direction, and perform image processing on the collected top view. Use the Canny operator to perform edge detection on the processed top view, and use the least squares method to fit a circle to the detected edges to measure the inner and outer diameters of the compression spring; Step 3: Obtain the position of the spring end in the top view through the detection algorithm, calculate the rotation angle of the turntable, and control the turntable to rotate so that the end thickness section is facing the second camera; Step 4: Use the second camera to collect pictures of the spring from the front view direction, detect the key point positions of the compression spring end through the trained yolov8-pose neural network, and calculate the end thickness. Perform image processing on the collected front view, use the Canny operator to perform edge detection on the preprocessed front view to obtain the spring contour, and use the minimum bounding rectangle algorithm on the contour to obtain the free height; Step 5: Display the processed images and results in the user operation interface and store the relevant data; The specific content of Step 3 includes: Step 3.1: Determine the center of the circle according to the inner and outer diameters of the compression spring measured in Step 2. Make rays outward from the center of the circle every 1°, calculate the point on each ray with the smallest distance from the inner and outer diameter contours as the intersection point of the ray and the inner and outer contours, and the distance between the two intersection points is used as the intercept; Step 3.2: Calculate the point with the largest difference among adjacent intercepts, record it as the end position, and calculate the angle between the ray where the end position is located and the vertically upward line, which is the rotation angle of the turntable; Step 3.3: Control the stepping motor turntable to rotate to the specified position through the PLC.

2. The method for detecting a compression spring based on machine vision according to claim 1, wherein The specific content of Step 2 includes: Step 2.1: Use median filtering on the collected top view to remove the noise in the picture; Step 2.2: Convert the RGB image to the HSV color space, set the thresholds of the H, S, and V channels according to the color characteristics of the spring and the background, and obtain the spring contour target image through threshold segmentation; Step 2.3: Perform edge extraction on the spring contour target image through the Canny operator to obtain the edge pixels of the compression spring; Step 2.4: Use the least squares method to fit a circle to the obtained spring edge pixels to obtain the inner and outer diameters of the spring.

3. The method for detecting a compression spring based on machine vision according to claim 2, wherein, The specific content of Step 4 includes: Step 4.1: After the turntable completes rotation, control the second camera to take pictures; Step 4.2: Perform key point detection on the collected front view using the yolov8-pose neural network, detect the predicted points of the end thickness, and calculate the end thickness through the predicted points; Step 4.3: Perform image processing on the collected front view using the method of Step 2, and use the minimum bounding rectangle algorithm to obtain the free length after obtaining the spring contour.

4. The method for detecting a compression spring based on machine vision according to claim 3, wherein The specific steps of step 4.2 are as follows: The method for calculating the end thickness through the prediction points is as follows: Detect three key points A, B, and C of the spring end through the yolov8-pose neural network, where AB is the upper end point and point C is the lower end point; After the deep learning neural network completes the detection, it will give the coordinates of the 3 key points, and the distance from point C to AB is the end thickness; The algorithm calculates the straight line passing through AB according to the coordinates of points A and B, and then calculates the distance from the point to the straight line to obtain the size of the end thickness.

5. A method for detecting compression springs based on machine vision according to claim 3, characterized in that The yolov8-pose neural network includes a backbone network, a neck network, and a head network; Among them, the feature extraction layer in the backbone network uses the FasterBLOCK module, and the feature fusion part in the neck network replaces the original convolution core with the SPD-C2f module.

6. A compression spring detection system based on machine vision, which is used to execute the detection method described in any one of claims 1-5, characterized in that, The detection system includes an image acquisition device and an image processing terminal; The image acquisition device includes a base, a main bracket and an auxiliary bracket installed on the base. A turntable is provided at the bottom of the main bracket, and a spring clamp is installed on the turntable. The compression spring is fixed on the spring clamp. A first camera is provided on the main bracket and above the turntable. The first camera is used to collect the top view of the compression spring; A second camera is provided on the auxiliary bracket, and the second camera is used to collect the front view of the compression spring; The image processing terminal is signal-connected to the first camera, the second camera, and the turntable.

7. The compression spring detection system based on machine vision according to claim 6, characterized in that, It also includes a lighting system, which includes an annular light source and a back light source. The annular light source is installed on the main bracket and between the first camera and the turntable. The middle of the annular light source has a through hole; The back light source is installed on the base and is arranged opposite to the second camera.

8. The compression spring detection system based on machine vision according to claim 6, characterized in that, The first camera is selected as Hikvision MV-CU120-10-GC, and the lens is a double telecentric lens.

9. The compression spring detection system based on machine vision according to claim 6, characterized in that, The second camera is selected as Hikvision MV-CU120-10-GC.

Images