A fastener loosening detection method

Abstract

The application relates to the field of visual detection and discloses a fastener loosening detection method. The method comprises the following steps: S1. First and second mark lines are arranged on the end face of a fastener; S2. When initialized, a first image of the end face of the fastener is acquired by using a camera, and the first image is transmitted to an image recognition module; S3. The first and second mark lines in the first image are recognized by using the image recognition module, and the reference slope K1 of the first mark line and the reference slope K2 of the second mark line are obtained; S4. When detected, a second image of the end face of the fastener is acquired by using the camera; S5. The actual slope K5 of the first mark line and the actual slope K6 of the second mark line are obtained by using the image recognition module; and S6. The slope change conditions of the first and second mark lines are judged by using an analysis module, and a loosening result of the fastener is given. The scheme is used for solving the problems of low accuracy and high false alarm rate when the fastener loosening is detected by using vision.

Description

Technical Field

[0001] This invention relates to the field of visual inspection, and more specifically, to a method for detecting loose fasteners. Background Technology

[0002] Trains, as a common mode of transportation in modern society, provide convenience for people's travel and play an important role in social development. As such a frequently used mode of transportation, the safe operation of trains is paramount, necessitating routine safety inspections on every trip or daily. These routine safety inspections are complex and frequent, making automated safety inspections a key research focus in related fields. Trains generate significant and prolonged vibrations during operation, making fasteners prone to loosening. Failure to detect this in time can easily lead to serious accidents. Therefore, fastener loosening checks are the most numerous and frequently performed part of routine safety inspections.

[0003] Traditional fastener loosening detection uses a method of drawing anti-loosening lines. First, after each fastener is tightened, a line is drawn through the surface of the fastener and the surface of the object being fastened, serving as an anti-loosening line. Then, during routine safety checks, each fastener's anti-loosening line is observed for breakage or misalignment. If the anti-loosening line is broken or misaligned, it indicates that the fastener has become loose. Traditional fastener loosening detection requires manual inspection of each fastener, which is extremely labor-intensive and prone to oversights.

[0004] Visual inspection refers to the process of converting the captured target into an image signal using an image acquisition device (such as a camera or video camera), transmitting it to a dedicated image processing system, and converting it into a digital signal based on pixel distribution, brightness, color, and other information. The image processing system then performs various operations on these signals to extract the target's features and outputs a judgment result. Visual inspection has immeasurable value in detecting defects and preventing defective products from leaving the market. Applying visual inspection to the aforementioned fastener loosening detection, by using the anti-loosening line as a feature of the captured target, can achieve automated safety detection and effectively improve detection efficiency. However, in practical applications, because the anti-loosening line in the captured fastener image is often difficult to identify correctly, the image processing system often issues false alarm signals, and the accuracy and false alarm rate of visual inspection are insufficient to meet application requirements. Summary of the Invention

[0005] The present invention aims to overcome at least one of the shortcomings of the prior art and provide a fastener loosening detection method to solve the problems of low accuracy and high false alarm rate when applying visual detection to detect fastener loosening.

[0006] The technical solution adopted in this invention is a fastener loosening detection method, including a camera and an image recognition module, the specific steps of which are as follows:

[0007] S1. Set a first marker line and a second marker line on the end face of the fastener; S2. During initialization, use a camera to acquire a first image of the end face of the fastener and transmit the first image to the image recognition module; S3. Use the image recognition module to identify the first marker line and the second marker line in the first image, and obtain the reference slope K1 of the first marker line and the reference slope K2 of the second marker line; S4. During detection, use a camera to acquire a second image of the end face of the fastener and transmit the second image to the image recognition module; S5. Use the image recognition module to identify the first marker line and the second marker line in the second image, and obtain the actual slope K5 of the first marker line and the actual slope K6 of the second marker line; S6. Use the analysis module to determine the slope changes of the first marker line and the second marker line, and give the result of fastener loosening.

[0008] This solution uses the detection of loose fasteners on train bogies as an example. A camera is fixed to one side of the track, forming a detection area within its field of view. Wheel sensors are installed on the track, triggering a shooting signal to the camera. When the train passes through the detection area at a certain speed, the wheel sensors detect the train's wheels and send a shooting signal to the camera. The camera captures an image of the bogie containing several fasteners. The image recognition module and analysis module process the image to obtain the looseness status of each fastener.

[0009] In this scheme, after the fasteners are tightened, maintenance personnel set a first and a second marker line on the end face of the fasteners. When the train first enters the inspection area (during initialization), the camera is triggered at a fixed point to capture a first image of the bogie, which covers the end face of the fasteners. The image recognition module obtains the reference slope K1 of the first marker line and the reference slope K2 of the second marker line. When the train enters the inspection area a second time (during inspection), the camera is triggered at a fixed point to capture a second image of the bogie, which covers the end face of the fasteners. The image recognition module obtains the actual slope K5 of the first marker line and the actual slope K6 of the second marker line. Then, the analysis module compares the difference between the reference slope and the actual slope with a preset threshold. If the difference is greater than the threshold, it indicates that the fastener has become loose; if the difference is less than the threshold, it indicates that the fastener remains tight. There are two ways to calculate the difference between the reference slope and the actual slope. One way is to first calculate the first difference between the reference slope K1 and the actual slope K5 of the first marker line, and the second difference between the reference slope K2 and the actual slope K6 of the second marker line, and then calculate the average of the first difference and the second difference. The other way is to first calculate the slope of the angle bisector of the first and second marker lines in the first image and use it as the reference slope, then calculate the slope of the angle bisector of the first and second marker lines in the second image and use it as the actual slope, and finally calculate the difference between the reference slope and the actual slope of the angle bisector.

[0010] In this solution, on the one hand, since both the first and second images acquired by the camera include the first and second marker lines, the analysis module eliminates errors caused by image recognition by averaging when evaluating their slope changes, thus solving the problem of low accuracy when using visual detection to detect fastener loosening. On the other hand, because a double-insurance method using two marker lines is adopted, even if the image recognition of one marker line is incorrect, the analysis module can still determine the loosening result of the fastener by observing the slope change of the other marker line, thus solving the problem of high false alarm rate when using visual detection to detect fastener loosening.

[0011] It is important to note that the first and second entries of the train into the detection area only indicate their sequential order, not the actual number of times the train entered. That is, when the train enters the detection area again, the result of the previous successful detection can be used as initial data. The previous image is used as the first image, and the actual slopes of its first and second marker lines are used as reference slopes K1 and K2, respectively, to analyze the slope changes in the next instance. Secondly, the first and second marker lines are not limited to single line segments; they can also be the edges of a shape. For example, the first and second marker lines can be the two sides of a triangle. Furthermore, depending on the camera's field of view, the first and second images can cover multiple fasteners. The image analysis module assigns numbers to multiple fasteners according to rules, and the image recognition and analysis modules can simultaneously process the first and second marker lines of multiple fasteners in a single image.

[0012] Preferably, step S3 further includes: if the image recognition module fails to obtain the reference slope K1 of the first marker line and the reference slope K2 of the second marker line, then an alarm message is sent to the user, and the process returns to step S2. This added judgment step ensures that the reference slope K1 of the first marker line and the reference slope K2 of the second marker line are obtained during initialization, so that the looseness of the fastener can be determined by averaging during detection.

[0013] Preferably, the method further includes step S7: if the slope change of the first marker line and the second marker line cannot be identified or exceeds the threshold, an alarm message is sent to the user, and a second image is pushed at the same time for manual confirmation.

[0014] Preferably, the first and second marker lines intersect at a single point on the end face of the fastener. After the first and second marker lines intersect, they become interconnected due to the intersection point. By adding the intersection point as a recognition feature when recognizing the first and second marker lines, the image recognition module can reduce recognition errors and improve the accuracy of the obtained reference slopes K1 and K2.

[0015] Furthermore, a third marker line is included, which intersects both the first and second marker lines. This third marker line is used to verify the slope changes of the first and second marker lines. The addition of the third marker line provides another intersection point for both the first and second marker lines. By adding these two intersection points as recognition features when recognizing the first and second marker lines, the image recognition module further reduces its recognition error and improves the accuracy of the obtained reference slopes K1 and K2.

[0016] Furthermore, the first and second marker lines are crosshairs located at the center of the fastener's end face. When they form a crosshair, the angle between the first and second marker lines is the largest, and near their intersection, pixel interference between them is minimal. In addition, the first and second marker lines cover the largest area on the fastener's end face, so even if the first and second marker lines are partially damaged or partially blocked, it will not affect the image recognition module. Therefore, the crosshair shape of the first and second marker lines further improves the recognition accuracy of the image recognition module.

[0017] In this solution, the setting of the first and second marking lines on the fastener end face includes two methods: one is based on the traditional marking method, and the other is based on the pasting method of marking stickers.

[0018] When using the marking method, step S1 includes: drawing the first marking line and the second marking line on the end face of the fastener using a marker. Since maintenance personnel will inevitably set anti-loosening lines after the fasteners are tightened, requiring them to draw the first and second marking lines does not incur additional application costs, thus reducing the impact on existing safety inspection processes and making this method easy to promote and use. Furthermore, the image recognition module uses a regression algorithm to calculate the slope of the first and second marking lines; the first and second marking lines do not need to be perfectly straight line segments, thereby reducing the drawing requirements for maintenance personnel.

[0019] Furthermore, the first and / or second marking lines extend from the end face of the fastener to the surface of the fastened object. The first or second marking line on the end face of the fastener is used for recognition by the image recognition module and judgment by the analysis module; the first or second marking line on the extended portion is equivalent to a traditional anti-loosening line, used for manual inspection. In this case, it is equivalent to maintenance personnel directly drawing the first and second marking lines based on the original anti-loosening line. The extended first or second marking lines are suitable for visual inspection, ensuring the reliability of image recognition and enabling automated loosening detection; they also retain the function of the original anti-loosening line, allowing for intuitive comparison and verification, achieving human-machine integration, and flexible switching between automatic and manual inspection.

[0020] When using the adhesive method, step S1 includes: attaching a sticker with the first and second marking lines drawn on it to the end face of the fastener. The first and second marking lines are batch-printed on the sticker, ensuring uniformity in thickness, straightness, and color depth, thereby reducing the recognition difficulty for the image recognition module and improving recognition accuracy. In use, maintenance personnel simply attach the sticker, roughly centered, to the end face of the fastener after tightening it.

[0021] Furthermore, the first and second marker lines possess waterproof and reflective properties, and / or waterproof and fluorescent properties. The waterproof properties prevent damage to the first and second marker lines, while the reflective and fluorescent properties improve their exposure in the first or second image, enhancing their contrast and making recognition by the image recognition module easier and more accurate.

[0022] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0023] In this solution, on the one hand, since both the first and second images acquired by the camera include the first and second marker lines, the analysis module eliminates errors caused by image recognition by averaging when evaluating their slope changes, thus solving the problem of low accuracy when using visual detection to detect fastener loosening. On the other hand, because a double-insurance method using two marker lines is adopted, even if the image recognition of one marker line is incorrect, the analysis module can still determine the loosening result of the fastener by observing the slope change of the other marker line, thus solving the problem of high false alarm rate when using visual detection to detect fastener loosening. Attached Figure Description

[0024] Figure 1 This is a schematic diagram of the present invention applied to a train bogie.

[0025] Figure 2 This is a partial view of the first image during initialization in Embodiment 1 of the present invention.

[0026] Figure 3 This is a partial view of the second image after the fastener has loosened in Embodiment 1 of the present invention.

[0027] Figure 4 This is a partial view of the first image during initialization in Embodiment 2 of the present invention.

[0028] Figure 5 This is a partial view of the second image after the fastener has loosened in Embodiment 2 of the present invention.

[0029] Figure 6 This is a partial view of the first image during initialization in Embodiment 3 of the present invention.

[0030] Figure 7 This is a partial view of the second image after the fastener has loosened in Embodiment 3 of the present invention.

[0031] Labeling explanation: Camera 11, Bogie 12, Fastener 13, First Marker Line 21, Second Marker Line 22, Third Marker Line 23, Anti-loosening Line 24. Detailed Implementation

[0032] The accompanying drawings are for illustrative purposes only and should not be construed as limiting the invention. To better illustrate the following embodiments, some parts in the drawings may be omitted, enlarged, or reduced, and do not represent the actual product dimensions; it is understandable to those skilled in the art that some well-known structures and their descriptions may be omitted in the drawings.

[0033] Example 1

[0034] like Figure 1 , 2 As shown in Figure 3, this embodiment is a method for detecting fastener loosening, including a camera and an image recognition module. The specific steps are as follows:

[0035] S1. Set a first marker line and a second marker line on the end face of the fastener; S2. During initialization, use a camera to acquire a first image of the end face of the fastener and transmit the first image to the image recognition module; S3. Use the image recognition module to identify the first marker line and the second marker line in the first image, and obtain the reference slope K1 of the first marker line and the reference slope K2 of the second marker line; S4. During detection, use a camera to acquire a second image of the end face of the fastener and transmit the second image to the image recognition module; S5. Use the image recognition module to identify the first marker line and the second marker line in the second image, and obtain the actual slope K5 of the first marker line and the actual slope K6 of the second marker line; S6. Use the analysis module to determine the slope changes of the first marker line and the second marker line, and give the result of fastener loosening.

[0036] This solution uses the detection of loose fasteners on train bogies as an example. A camera is fixed to one side of the track, forming a detection area within its field of view. Wheel sensors are installed on the track, triggering a shooting signal to the camera. When the train passes through the detection area at a certain speed, the wheel sensors detect the train's wheels and send a shooting signal to the camera. The camera captures an image of the bogie containing several fasteners. The image recognition module and analysis module process the image to obtain the looseness status of each fastener.

[0037] In this scheme, after the fasteners are tightened, maintenance personnel set a first and a second marker line on the end face of the fasteners. When the train first enters the inspection area (during initialization), the camera is triggered at a fixed point to capture a first image of the bogie, which covers the end face of the fasteners. The image recognition module obtains the reference slope K1 of the first marker line and the reference slope K2 of the second marker line. When the train enters the inspection area a second time (during inspection), the camera is triggered at a fixed point to capture a second image of the bogie, which covers the end face of the fasteners. The image recognition module obtains the actual slope K5 of the first marker line and the actual slope K6 of the second marker line. Then, the analysis module compares the difference between the reference slope and the actual slope with a preset threshold. If the difference is greater than the threshold, it indicates that the fastener has become loose; if the difference is less than the threshold, it indicates that the fastener remains tight. There are two ways to calculate the difference between the reference slope and the actual slope. One way is to first calculate the first difference between the reference slope K1 and the actual slope K5 of the first marker line, and the second difference between the reference slope K2 and the actual slope K6 of the second marker line, and then calculate the average of the first difference and the second difference. The other way is to first calculate the slope of the angle bisector of the first and second marker lines in the first image and use it as the reference slope, then calculate the slope of the angle bisector of the first and second marker lines in the second image and use it as the actual slope, and finally calculate the difference between the reference slope and the actual slope of the angle bisector.

[0038] In this solution, on the one hand, since both the first and second images acquired by the camera include the first and second marker lines, the analysis module eliminates errors caused by image recognition by averaging when evaluating their slope changes, thus solving the problem of low accuracy when using visual detection to detect fastener loosening. On the other hand, because a double-insurance method using two marker lines is adopted, even if the image recognition of one marker line is incorrect, the analysis module can still determine the loosening result of the fastener by observing the slope change of the other marker line, thus solving the problem of high false alarm rate when using visual detection to detect fastener loosening.

[0039] It is important to note that the first and second entries of the train into the detection area only indicate their sequential order, not the actual number of times the train entered. That is, when the train enters the detection area again, the result of the previous successful detection can be used as initial data. The previous image is used as the first image, and the actual slopes of its first and second marker lines are used as reference slopes K1 and K2, respectively, to analyze the slope changes in the next instance. Secondly, the first and second marker lines are not limited to single line segments; they can also be the edges of a shape. For example, the first and second marker lines can be the two sides of a triangle. Furthermore, depending on the camera's field of view, the first and second images can cover multiple fasteners. The image analysis module assigns numbers to multiple fasteners according to rules, and the image recognition and analysis modules can simultaneously process the first and second marker lines of multiple fasteners in a single image.

[0040] Preferably, step S3 further includes: if the image recognition module fails to obtain the reference slope K1 of the first marker line and the reference slope K2 of the second marker line, then an alarm message is sent to the user, and the process returns to step S2. This added judgment step ensures that the reference slope K1 of the first marker line and the reference slope K2 of the second marker line are obtained during initialization, so that the looseness of the fastener can be determined by averaging during detection.

[0041] Preferably, the method further includes step S7: if the slope change of the first marker line and the second marker line cannot be identified or exceeds the threshold, an alarm message is sent to the user, and a second image is pushed at the same time for manual confirmation.

[0042] This solution uses a line drawing method to set the first and second marking lines. Step S1 specifically includes: drawing the first and second marking lines on the end face of the fastener using a marker. Since maintenance personnel will inevitably set anti-loosening lines after the fasteners are tightened, requiring them to draw the first and second marking lines will not incur additional application costs, thus reducing the impact on the existing safety inspection process and making this method easy to promote and use. In addition, the image recognition module uses a regression algorithm to calculate the slope of the first and second marking lines. The first and second marking lines do not need to be straight line segments, thereby reducing the drawing requirements for maintenance personnel.

[0043] Furthermore, the first and / or second marking lines extend from the end face of the fastener to the surface of the fastened object. The first or second marking line on the end face of the fastener is used for recognition by the image recognition module and judgment by the analysis module; the first or second marking line on the extended portion is equivalent to a traditional anti-loosening line, used for manual inspection. In this case, it is equivalent to maintenance personnel directly drawing the first and second marking lines based on the original anti-loosening line. The extended first or second marking lines are suitable for visual inspection, ensuring the reliability of image recognition and enabling automated loosening detection; they also retain the function of the original anti-loosening line, allowing for intuitive comparison and verification, achieving human-machine integration, and flexible switching between automatic and manual inspection.

[0044] Preferably, the first and second marker lines intersect at a single point on the end face of the fastener. After the first and second marker lines intersect, they become interconnected due to the intersection point. By adding the intersection point as a recognition feature when recognizing the first and second marker lines, the image recognition module can reduce recognition errors and improve the accuracy of the obtained reference slopes K1 and K2.

[0045] In this solution, the marker ink should have waterproof and reflective properties, and / or waterproof and fluorescent properties. The waterproof property is to prevent the drawn first and second marking lines from being easily damaged. The reflective and fluorescent properties are to improve the exposure of the first and second marking lines in the first or second image, enhance their contrast, and make recognition by the image recognition module easier and more accurate.

[0046] In this embodiment, the maintenance personnel start from a point on the edge of the fastener and make two consecutive straight lines, extending to the surface of the fastened object, forming the first marking line and the second marking line. The left one is designated as the first marking line, and the right one as the second marking line, or vice versa. The first and second marking lines can serve as traditional anti-loosening lines, eliminating the need for additional anti-loosening lines. The angle between the first and second marking lines is not less than 30°, and the pixel length of both marking lines in the first or second image is not less than 20 pixels to ensure that the image recognition module can accurately identify and obtain their slope.

[0047] Example 2

[0048] like Figure 1 , 4 As shown in Figure 5, this embodiment is a method for detecting fastener loosening, which is another implementation based on Embodiment 1. The following only describes the differences, and the same parts will not be repeated.

[0049] This solution uses an adhesive method to set the first and second marking lines. Step S1 specifically includes: using a sticker with the first and second marking lines drawn on it to the end face of the fastener. The first and second marking lines are batch-printed on the sticker, ensuring uniformity in thickness, straightness, and color depth, thereby reducing the recognition difficulty for the image recognition module and improving recognition accuracy. In use, maintenance personnel simply paste the sticker approximately in the center of the end face of the fastener after tightening it.

[0050] Furthermore, the first and second marker lines are crosshairs located at the center of the fastener's end face. When they form a crosshair, the angle between the first and second marker lines is the largest, and near their intersection, pixel interference between them is minimal. In addition, the first and second marker lines cover the largest area on the fastener's end face, so even if the first and second marker lines are partially damaged or partially blocked, it will not affect the image recognition module. Therefore, the crosshair shape of the first and second marker lines further improves the recognition accuracy of the image recognition module.

[0051] In this embodiment, the marking sticker is a circular crosshair symbol comprising four quadrants. The first and third quadrants are filled with a dark color, while the second and fourth quadrants are unfilled. The image recognition module identifies the color differences between the quadrants to determine the first and second marking lines. Vertical lines are designated as the first marking line, and horizontal lines as the second marking line, or vice versa. Furthermore, anti-loosening lines must still be retained on the fasteners to allow for manual inspection.

[0052] Furthermore, the dark-filled inks in the first and third quadrants possess waterproof, reflective, and / or fluorescent properties. The waterproof properties prevent damage to the first and second marker lines, while the reflective and fluorescent properties improve the exposure of the first and second marker lines in the first or second image, enhancing their contrast and making recognition by the image recognition module easier and more accurate.

[0053] Example 3

[0054] like Figure 6 , 7 As shown, this embodiment is another implementation of the label sticker of Embodiment 2. The following description only focuses on the differences, and the same parts will not be repeated.

[0055] This scheme includes a first marker line, a second marker line, and a third marker line. The third marker line intersects with both the first and second marker lines. The third marker line is used to verify the slope changes of the first and second marker lines. The addition of the third marker line provides another intersection point for both the first and second marker lines. When recognizing the first and second marker lines, the image recognition module uses these two intersection points as recognition features, further reducing recognition errors and improving the accuracy of the obtained reference slopes K1 and K2.

[0056] In this embodiment, the label sticker is a dark-colored triangular sticker. One apex of the triangle points to the anti-loosening line, and the first, third, and second marking lines can be sequentially arranged counterclockwise from this apex. Furthermore, the dark-colored ink used to fill the triangle should also possess waterproof and reflective properties, and / or waterproof and fluorescent properties.

[0057] Obviously, the above embodiments of the present invention are merely examples to clearly illustrate the technical solutions of the present invention, and are not intended to limit the specific implementation of the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the claims of the present invention should be included within the protection scope of the claims of the present invention.

Claims

1. A method for detecting loose fasteners, comprising a camera and an image recognition module, characterized in that, Includes the following steps: S1. Set a first marking line and a second marking line on the end face of the fastener; S2. During initialization, the camera is used to acquire the first image of the end face of the fastener, and the first image is transmitted to the image recognition module; S3. Use the image recognition module to identify the first and second marker lines in the first image, and obtain the reference slope K1 of the first marker line and the reference slope K2 of the second marker line; S4. During inspection, a second image of the end face of the fastener is acquired using a camera, and the second image is transmitted to the image recognition module; S5. Use the image recognition module to identify the first and second marker lines in the second image, and obtain the actual slope K5 of the first marker line and the actual slope K6 of the second marker line; S6. Use the analysis module to determine the slope changes of the first and second marker lines and provide the results of the fastener loosening. When judging the changes in the slopes of the first and second marker lines, the analysis module calculates the average value. The first marking line and the second marking line intersect at a point on the end face of the fastener; The first marking line and / or the second marking line extend from the end face of the fastener to the surface of the object being fastened; It also includes a third marker line, which intersects with the first marker line and the second marker line respectively; the third marker line is used to check the slope changes of the first marker line and the second marker line.

2. The fastener loosening detection method according to claim 1, characterized in that, The first and second marking lines are cross lines located at the center of the fastener end face.

3. The fastener loosening detection method according to claim 1, characterized in that, Step S1 specifically includes: using a marker to draw the first marking line and the second marking line on the end face of the fastener.

4. The fastener loosening detection method according to claim 1, characterized in that, Step S1 specifically includes: using a sticker with the first and second marking lines drawn on it to stick to the end face of the fastener.

5. The fastener loosening detection method according to claim 1, characterized in that, The first and second marking lines have waterproof and reflective properties, or waterproof and fluorescent properties, or waterproof, reflective, and fluorescent properties.

6. A method for detecting fastener loosening according to any one of claims 1 to 5, characterized in that, Step S3 further includes: if the image recognition module fails to obtain the reference slope K1 of the first marker line and the reference slope K2 of the second marker line, it sends an alarm message to the user and returns to step S2.

7. A method for detecting fastener loosening according to any one of claims 1 to 5, characterized in that, It also includes step S7: if the slope change of the first marker line and the second marker line cannot be identified or exceeds the threshold, an alarm message is sent to the user, and a second image is pushed at the same time for manual confirmation.

Images