A train bolt fastener loosening detection method based on anti-loosening marks

Abstract

The application discloses a train bolt fastener loosening detection method based on anti-loosening marks, first, a two-dimensional image of a train bolt fastener with anti-loosening marks is acquired through an image acquisition device, and a region where the bolt fastener is located is recognized, then, arc detection is performed on the region to obtain a circle matched with each element of the bolt fastener; the bolt fastener image is cut in turn with the matched circle as a boundary to obtain circular cutting images of different radii, then, a blank image is created, the circular cutting image is translated into the blank image to obtain a corrected bolt fastener image; whether the bolt fastener is loosened is judged by detecting the anti-loosening marks in the corrected bolt fastener image. The application can correct display errors of the anti-loosening marks caused by a shooting angle, so that whether the bolt fastener is loosened can be judged relatively accurately.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of rail transit detection technology, and in particular to a train bolt fastener loosening detection method based on a loosening prevention mark. BACKGROUND

[0002] There are a large number of bolt fasteners for connection and fixation on the sides and bottoms of trains. These fasteners may loosen due to vibration, impact, thermal expansion and contraction, etc. Severe loosening may cause train accidents. In order to timely detect the loosening of bolt fasteners, a loosening prevention mark is usually painted on the bolt, nut, gasket and other components. The loosening prevention mark is a three-dimensional marker line that extends from the bolt head through the nut, gasket and other components to the fixed base surface. If the nut loosens, a segment of the marker line will be misaligned, which can be easily detected by maintenance workers and checked out. However, there are some disadvantages in visual observation. First, the number of fasteners is large, and manual inspection is time-consuming. Second, it is easy to cause misjudgment and missed detection due to visual fatigue or negligence. Third, it is difficult to achieve traceable management.

[0003] Machine vision detection is a method that captures bolt fastener images through a camera and then uses a trained model to determine whether the loosening prevention mark is misaligned. This method is time-saving and does not have the problems of visual fatigue and missed detection. However, the detection quality of machine vision is closely related to the quality of the original bolt fastener image. For example, the invention patent CN113469966B discloses a train bolt loosening detection method based on loosening line identification. It directly extracts the red pixel area of the collected image to obtain the loosening line image, then performs a closing operation on the image to improve the loosening line extraction quality, and finally uses the smallest circumscribed rectangle of each loosening line to represent the shape of the loosening line and calculates the rectangle angle to determine whether the bolt is loose. The disadvantage of this method or similar visual detection methods is that it is difficult to directly capture the bolt fastener in practice due to factors such as train stopping location and camera installation angle. The loosening prevention mark in the two-dimensional image may appear "display misalignment", i.e., even if the three-dimensional loosening prevention mark is not misaligned, it may still appear misaligned to varying degrees in the two-dimensional image (the display error of the original image). Figure 2 SUMMARY

[0004] The purpose of the present application is to provide a train bolt fastener loosening detection method based on a loosening prevention mark, which can correct the display error of the loosening prevention mark caused by not directly capturing the bolt fastener, and provide a more reliable image sample for subsequent detection of bolt fastener loosening based on the loosening prevention mark.

[0005] To achieve the above purpose, the present application adopts the following technical solutions:

[0006] ​A train bolt fastener loosening detection method based on anti-loosening mark, comprising the following steps:

[0007] Step 1: obtaining a two-dimensional image of a train bolt fastener with an anti-loosening mark through an image acquisition device, and identifying and locating the region of the two-dimensional image where the bolt fastener is located;

[0008] Step 2: performing arc detection on the region, then fitting a complete circle using the detected arcs, and screening the obtained circle to match the element radius of the bolt fastener, to obtain a matching circle;

[0009] Step 3: cutting the bolt fastener in the region in turn according to the matching circle radius from small to large, taking the matching circle as the boundary, to obtain circular cutting images of different radii;

[0010] Step 4: creating a blank image, translating the circular cutting images into the blank image in order of radius from large to small, and aligning the centers of the circular cutting images with the same point in the blank image, to obtain a modified bolt fastener image;

[0011] Step 5: judging whether the bolt fastener is loose by detecting the anti-loosening mark in the modified bolt fastener image.

[0012] Preferably, the step 5 specifically comprises the following steps:

[0013] Step 511: taking the boundary of the circular cutting image with the largest radius as the outer boundary, and taking a certain concentric circle in the circular cutting image with the smallest radius as the inner boundary, to divide the modified bolt fastener image into a ring image by difference;

[0014] Step 512: identifying and extracting the anti-loosening mark in the ring image to obtain a plurality of anti-loosening mark segments;

[0015] Step 513: converting the ring image and the anti-loosening mark segments from rectangular coordinates to polar coordinates, to obtain a rectangular image from the ring image and a plurality of anti-loosening mark deformation segments from the plurality of anti-loosening mark segments, and mapping the anti-loosening mark deformation segments onto the rectangular image;

[0016] Step 514: calculating the area of each anti-loosening mark deformation segment, and then selecting the maximum range of the horizontal coordinates and the vertical coordinates of the anti-loosening mark deformation segment with the largest area as the reference range;

[0017] Step 515: calculate whether the horizontal coordinate and the vertical coordinate of the loosening-proof mark deformation section except the loosening-proof mark deformation section with the largest area are within the reference range, if yes, it is determined that the bolt fastener is not loosened, if not, it is determined that the bolt fastener is loosened.

[0018] Or preferably, the step 5 specifically comprises the following steps:

[0019] Step 521: perform straight line fitting on the area where each loosening-proof mark in the modified bolt fastener image is located;

[0020] Step 522: calculate the included angle between each fitted straight line and the coordinate axis in the coordinate system, if the obtained included angle is within the preset threshold range, it is determined that the bolt fastener is not loosened, if the obtained included angle is not within the preset threshold range, it is determined that the bolt fastener is loosened.

[0021] Preferably, the image acquisition device in step 1 is a camera, and the blank image in step 4 is a square.

[0022] Preferably, the elements of the bolt fastener in step 2 include a bolt and a nut, or include a bolt, a nut and a gasket.

[0023] Preferably, in step 511, the radius of the concentric circle is 1 / 2, 1 / 3, 1 / 4 or 1 / 5 of the radius of the circular cutting image with the smallest radius.

[0024] Preferably, the step 521 comprises the following specific steps:

[0025] Step 5211: calculate the area of each of the mark regions;

[0026] Step 5212: if the area of the mark region is less than a first threshold value, it is considered that the mark region is a point, and the center point thereof is directly calculated, and the line connecting the center point and the origin of the coordinate axis is taken as the fitted straight line; if the area of the mark region is greater than a second threshold value, it is considered that the mark region is a line segment with length, the mark region is dilated and eroded to obtain a direction region, and then a skeleton extraction algorithm is used to extract the skeleton of the mark region; then the skeleton is straight line fitted to obtain the fitted straight line.

[0027] The present application has the following beneficial effects: a complete circle is fitted through a circular arc, then a matching circle with a radius matching that of a bolt, nut, gasket, etc. is screened out, then the bolt, nut and gasket, etc. are cut according to the matching circle as a boundary and in order of increasing radius, and the cut images are translated into a blank image in order of decreasing radius and each cut image is aligned with the same point (such as the coordinate center) in the blank image, so that the cut images form concentric circles, which are consistent with the actual arrangement of the bolt, nut and gasket in the bolt fastener, thereby effectively correcting the display distortion caused by the camera not being directly opposite the bolt fastener when taking a picture, and providing a more reliable image sample for subsequent detection of whether the anti-loose mark is misaligned. BRIEF DESCRIPTION OF DRAWINGS

[0028] Figure 1 Actual photo of train bolt fastener and environment;

[0029] Figure 2 Region where the identified and located bolt fastener is located; Figure 1 Region where the identified and located bolt fastener is located;

[0030] Figure 3 Bolt top surface image cut out by the matching circle with the smallest radius;

[0031] Figure 4 Nut top surface image cut out by the matching circle with the middle radius;

[0032] Figure 5 Gasket top surface image cut out by the matching circle with the largest radius;

[0033] Figure 6 Modified bolt fastener image obtained by translating the three images in and in order of decreasing radius into a blank image; Figure 3 Figure 4 Figure 5 Modified bolt fastener image obtained by translating the three images in and in order of decreasing radius into a blank image;

[0034] Figure 7 Annular image segmented by taking difference of the images;

[0035] Figure 8 Rectangular image obtained by polar coordinate conversion of the annular image;

[0036] Figure 9 Image obtained by mapping the anti-loose mark onto the rectangular image after polar coordinate conversion of the anti-loose mark. DETAILED DESCRIPTION

[0037] The present application will be described in detail below with reference to the accompanying drawings.

[0038] ​​The application is a method for detecting whether a bolt fastener is loose based on a loose-proof mark, which mainly improves the detection accuracy by correcting the display distortion of whether the loose-proof mark is misaligned caused by the camera not directly facing the bolt fastener. Specifically, the method comprises the following steps:

[0039] Step 1: obtaining a two-dimensional image containing a bolt fastener through a camera (the bolt fastener in the box is a target to be processed, and other bolt fasteners are also processed according to the method of the embodiment), and identifying and locating the area where the bolt fastener is located, and cutting out the area (the bolt fastener in the box is a target to be processed, and other bolt fasteners are also processed according to the method of the embodiment). Figure 1 Figure 2

[0040] Step 2: performing circular arc detection on the area, using the edges_sub_pix algorithm to extract all edges in the image, then using the segment_contours_xld algorithm to segment all edges into circular arcs, and using the union_cocircular_contours_xld algorithm to merge all contours belonging to the same circle, then using the fit_circle_contour_xld algorithm to fit the detected circular arcs into complete circles (at this time, multiple circles including those not matching the radii of bolts, nuts and washers may be fitted), and then using the pre-stored radius information of bolt, nut and washer elements combined with a threshold to screen the above complete circles to obtain circles matching the radii of bolts, nuts and washers, i.e. matching circles. For a bolt fastener composed of a bolt, a nut and a washer, the matching circles obtained are the outer boundary circle of the top surface of the bolt, the outer boundary circle of the top surface of the nut (the inner tangent circle of the regular hexagon of the outer edge of the common nut is obtained because there is a transition chamfer between the outer edge and the top surface, i.e. the outer boundary circle of the top surface of the nut) and the outer boundary circle of the top surface of the washer.

[0041] Step 3: cutting the area where the bolt fastener is located according to the matching circles as boundaries in the order of the radii of the matching circles from small to large (using the reduce_domain algorithm to cut the image), i.e. cutting the area with the smallest radius of the matching circle first, such as the top surface of the bolt in Figure 3 , then cutting the top surface of the nut, as shown in Figure 4 , and finally cutting the washer, as shown in Figure 5 , thereby obtaining circular cutting images with different radii.

[0042] Step 4: creating a blank square image, taking the center of the blank square image as the coordinate origin, and placing the circular cutting images, i.e. Figure 3 , Figure 4 and Figure 5 ​​Translate to the blank image in the order of radius from large to small (put small image in the back can avoid small image being covered by large image) and make the center of the circular cut image align with the coordinate origin in the blank image, get the modified bolt fastener image, as shown in Figure 6 . Figure 6 It can be seen that, because the circle of each circular cut image aligns with the coordinate origin, the dislocation of the bolt, nut, washer and lockout mark in the original image caused by the lateral deviation of the shooting angle is corrected, that is, the lockout mark is corrected in the modified bolt fastener image, and the display effect is closer to the camera shooting the fastener directly.

[0043] Step 5: judging whether the bolt fastener is loose by detecting the lockout mark in the modified bolt fastener image.

[0044] For step 5, one method is to detect the lockout mark and then judge whether the bolt fastener is loose by the following steps:

[0045] Step 511: circular segmentation of the modified bolt fastener image, taking the coordinate axis origin as the center point, taking the boundary of the circular cut image with the largest radius (i.e. the image of the washer in Figure 6 ) as the outer boundary (i.e. the outer diameter of the annulus), taking a certain concentric circle inside the circular cut image with the smallest radius (i.e. the image of the bolt in Figure 6 ) as the inner boundary (i.e. the distance between the coordinate origin and a certain point near it is the inner diameter of the annulus, which can be 1 / 2, 1 / 3, 1 / 4 or 1 / 5 of the radius of the bolt image, etc.), and taking the difference to segment a annular image from the modified bolt fastener image in Figure 6 . Figure 7 .

[0046] Step 512: Identify and extract the anti-loose mark in the annular image to obtain several anti-loose mark segments. Specifically, the following methods can be used: Method 1: First, train a large number of anti-loose marks through deep learning to obtain an anti-loose mark model, and then detect the anti-loose mark; Method 2: Convert the RGB color space of the image to the hsv space using the trans_from_rgb algorithm, and then extract the anti-loose mark according to its color; Method 3: Label a large number of anti-loose mark images using the sam2 image and video general segmentation model, then train the segmentation model to obtain an anti-loose mark segmentation model, and then use the model to detect and extract the anti-loose mark. In this step, because the center of each circularly cut image in the corrected bolt fastener image has been re-aligned, the anti-loose marks on different cut images will be disconnected from each other and form multiple segments, i.e. several anti-loose mark segments will be obtained; but for the case where the camera is directly facing the fastener, the anti-loose mark will not be disconnected, and only one anti-loose mark segment will be obtained (i.e. the anti-loose marks on the bolt, nut and washer are included).

[0047] Step 513: Convert the annular image in step 512 from rectangular coordinates to polar coordinates using the polar_trans_image_ext algorithm to obtain a rectangular image Figure 8 ), and convert the anti-loose mark from rectangular coordinates to polar coordinates using the same algorithm to obtain several anti-loose mark deformation segments from the anti-loose mark segments, and map the anti-loose mark deformation segments to the rectangular image Figure 9 using the polar_trans_region algorithm to obtain a dark region in Figure 8 , which covers the corresponding region in Figure 8 , thereby verifying the consistency and accuracy of the annular image transformation and the anti-loose mark change).

[0048] Step 514: Calculate the area of each anti-loose mark deformation segment using the area_center, sort the areas using the tuple_sort_index algorithm, select the largest anti-loose mark deformation segment using the select_obj algorithm, and then sort the coordinates of the region to obtain the maximum range of the horizontal and vertical coordinates of the anti-loose mark region as the reference range.

[0049] Step 515: Calculate whether the horizontal and vertical coordinates of the anti-loose mark deformation segments other than the largest anti-loose mark deformation segment are within the reference range, if yes, determine that the bolt fastener is not loose, if no, determine that the bolt fastener is loose.

[0050] For step 5, another method is to detect the anti-loose mark and determine whether the bolt fastener is loose as follows:

[0051] Step 521: straight line fitting is performed on the area of each anti-loosening mark in the modified bolt fastener image, i.e., the mark area, and the specific operation steps are as follows:

[0052] Step 5211: the area of each mark area is calculated.

[0053] Step 5212: if the area of the mark area is less than a first threshold value, it is considered that the mark area is a point, and the center point thereof is directly calculated, and the line connecting the center point and the origin of the coordinate axis is taken as the fitted straight line. If the area of the mark area is greater than a second threshold value, it is considered that the mark area is a line segment with a length, and then the mark area is dilated and eroded to obtain a more accurate direction area; then a skeleton extraction algorithm commonly used in machine vision is used to extract the mark area; then a fit_line_contour_xld algorithm is used to perform straight line fitting on the skeleton to obtain the fitted straight line.

[0054] Step 522: a line_orientation algorithm is used to calculate the included angle between each fitted straight line and the coordinate axis in the coordinate system, and if the obtained included angle is within a third threshold value range, it is determined that the bolt fastener has not been loosened, and if the obtained included angle is not within the third threshold value range, it is determined that the bolt fastener has been loosened.

[0055] The above embodiments are only descriptions of the concept and implementation of the present application and do not limit the same. Under the concept of the present application, technical solutions without substantial changes are still within the protection scope.

Claims

1. A method for detecting loose train bolt fasteners based on anti-loosening markings, characterized in that... Includes the following steps: Step 1: Acquire a two-dimensional image of the train bolt fastener with anti-loosening markings using an image acquisition device, and identify and locate the area where the bolt fastener is located on the two-dimensional image; Step 2: Perform arc detection on the area, then fit a complete circle using the detected arc, and filter the obtained circle to make the circle radius match the component radius of the bolt fastener, thus obtaining a matching circle; Step 3: According to the matching circle radius from small to large, cut the bolt fasteners in the area in sequence with the matching circle as the boundary to obtain circular cut images with different radii; Step 4: Create a blank image. Translate the circular cropped image into the blank image in descending order of radius, and align the center of each circular cropped image with the same point in the blank image to obtain the corrected bolt fastener image. Step 5: Determine whether the bolt fastener is loose by detecting the anti-loosening mark in the corrected bolt fastener image.

2. The method for detecting loose train bolt fasteners based on anti-loosening markings as described in claim 1, characterized in that: Step 5 specifically includes the following steps: Step 511: Using the boundary of the circular cropped image with the largest radius as the outer boundary and a concentric circle within the circular cropped image with the smallest radius as the inner boundary, the modified bolt fastener image is divided into an annular image by taking the difference. Step 512: Identify and extract the anti-loosening markings in the circular image to obtain several anti-loosening marking segments; Step 513: Convert the circular image and the anti-loosening mark segment from rectangular coordinates to polar coordinates respectively, so as to obtain a rectangular image from the circular image, obtain a number of anti-loosening mark deformation segments from the number of anti-loosening mark segments, and map the anti-loosening mark deformation segments onto the rectangular image; Step 514: Calculate the area of ​​each of the anti-loosening mark deformation segments, and then select the maximum range of the horizontal and vertical coordinates of the anti-loosening mark deformation segment with the largest area as the reference range; Step 515: Calculate whether the horizontal and vertical coordinates of the anti-loosening mark deformation segments other than the largest anti-loosening mark deformation segment are within the reference range. If yes, it is determined that the bolt fastener is not loose; if no, it is determined that the bolt fastener is loose.

3. The method for detecting loose train bolt fasteners based on anti-loosening markings as described in claim 1, characterized in that: Step 5 specifically includes the following steps: Step 521: Perform linear fitting on each segment of the anti-loosening mark area in the corrected bolt fastener image, i.e., the mark area; Step 522: Calculate the angle between each fitted straight line and the coordinate axis in the coordinate system. If the angle is within the preset threshold range, it is determined that the bolt fastener is not loose. If the angle is not within the preset threshold range, it is determined that the bolt fastener is loose.

4. The method for detecting loose train bolt fasteners based on anti-loosening markings as described in claim 1, characterized in that: The image acquisition device mentioned in step 1 is a camera, and the blank image mentioned in step 4 is a square.

5. The method for detecting loose train bolt fasteners based on anti-loosening markings as described in claim 1, characterized in that: The components of the bolt fastener described in step 2 include bolts and nuts, or bolts, nuts and washers.

6. The method for detecting loose train bolt fasteners based on anti-loosening markings as described in claim 2, characterized in that: In step 511, the radius of the concentric circles is 1 / 2, 1 / 3, 1 / 4 or 1 / 5 of the radius of the smallest circular cropped image.

7. The method for detecting loose train bolt fasteners based on anti-loosening markings as described in claim 3, characterized in that: Step 521 includes the following specific steps: Step 5211: Calculate the area of ​​each of the marked areas; Step 5212: If the area of ​​the marked region is less than the first threshold, the marked region is considered to be a point, and its center point is directly calculated. The line connecting this center point and the origin of the coordinate axis is used as the fitted straight line. If the area of ​​the marked region is greater than the second threshold, the marked region is considered to be a line segment with length. The marked region is expanded and eroded to obtain the directional region. Then, the skeleton of the marked region is extracted using a skeleton extraction algorithm. Then, the skeleton is fitted with a straight line to obtain the fitted straight line.

Images