A multi-dimensional vision-driven method for detecting loose bolts in subway bogies

Abstract

The present invention relates to the field of train monitoring technology and discloses a multi-dimensional vision-co-driven method for detecting bolt loosening on a subway bogie. The method comprises the following steps: obtaining two-dimensional image data and three-dimensional image data of bolts at key locations on the subway bogie; performing data enhancement on the two-dimensional image data of the bolts, inputting the data into a BOLT-YOLO bolt detection model to identify key locations on the subway bogie, outputting bolt images containing the key locations, and numbering each key location; obtaining independent bolt region images, inputting the data into a Unet semantic segmentation network model to identify two-segment bolt anti-loosening lines, and determining the loosening state by the angle change between the anti-loosening line and the bolt; if the two-dimensional image information is insufficient, generating a three-dimensional point cloud model of the bolts, and determining whether the bolts are outside an error range by precise registration and calculation of the height difference of a reference surface. The method can not only detect abnormal states of train bolts in real time, but also improve detection accuracy and reduce resource utilization.

Description

Technical Field

[0001] The present invention relates to the technical field of train monitoring, and in particular to a method for detecting loose bolts on a subway bogie driven by multi-dimensional vision. Background Art

[0002] With the rapid development of rail transit, rail trains are playing an increasingly important role in daily life. Trains not only carry out a vast array of transportation tasks, but ensuring their safe operation is also paramount. This is especially true during train inspection and maintenance, where ensuring train safety is crucial. Loose bolts can lead to equipment failures and even serious problems like wheel loss, seriously threatening safe train operations. Therefore, detecting loose bolts has become a critical step in train maintenance.

[0003] Traditional bolt inspection relies primarily on manual inspections, where workers visually identify the anti-loosening markings on bolts to determine if they are loose. However, this manual inspection method is not only inefficient but also highly subjective and susceptible to human error. Therefore, there is an urgent need for a technology that can automatically detect bolt loosening in real time to improve the accuracy and efficiency of bolt inspection, thereby enhancing the safety and reliability of train operations. Currently, several bolt loosening detection methods have been proposed, including those based on acoustic elasticity and torsion. The acoustic elasticity method measures the change in the bolt's stress state by measuring the change in the velocity of sound within the solid with stress, while the torsion method uses the approximate linear relationship between bolt preload and torque to detect changes in preload to determine bolt loosening. While these two methods can detect bolt loosening, they have high requirements for the external environment and a complex measurement process, making them difficult to scale to complex practical applications. In contrast, a deep learning-based bolt loosening detection method detects the anti-loosening markings on the bolts and uses semantic segmentation to determine bolt loosening. This method refines and extracts the anti-loosening marking line skeleton, then uses deep learning methods combined with image processing technology to quickly identify and determine whether the bolt is loose. Its advantages lie in its high detection speed and ability to efficiently process large amounts of data. However, this method also has certain limitations. For example, an inappropriate camera angle can result in poor image quality, which can affect the accuracy of anti-loosening marking line recognition and, consequently, the judgment result. Summary of the Invention

[0004] In response to the above-mentioned deficiencies in the prior art, the present invention provides a method for detecting loose subway bogie bolts driven by multi-dimensional vision, which solves the problem of low detection accuracy caused by the detection limitations of two-dimensional or three-dimensional detection in the prior art of subway bogie bolt anomaly detection.

[0005] In order to achieve the above-mentioned object of the invention, the technical solution adopted by the present invention is:

[0006] A multi-dimensional vision-driven method for detecting loose bolts on subway bogies includes the following steps:

[0007] S1. Acquire 2D image data and 3D image data of bolts at key locations of a subway bogie;

[0008] S2. Data enhancement is performed on the bolt 2D image data, and the enhanced bolt 2D image data is input into the BOLT-YOLO bolt detection model for recognition. It is determined whether the key parts of the subway bogie are recognized. If so, the bolt image containing the key parts is output and each key part is numbered. Otherwise, the bolt is lost.

[0009] S3, cropping each numbered key part in the bolt image containing the key parts to obtain an independent bolt area image;

[0010] S4. Input the independent bolt area image into the trained Unet semantic segmentation network model for recognition, and determine whether two bolt anti-loosening lines are recognized. If so, execute step S5; otherwise, execute step S6;

[0011] Among them, the two sections of bolt anti-loosening lines are the bolt surface anti-loosening line and the fixed reference surface anti-loosening line;

[0012] S5. After skeleton thinning, the two segments of the bolt anti-loosening line are fitted to obtain the best fitting straight line of the two thinned segments of the bolt anti-loosening line, calculate the angle between the best fitting straight lines of the two thinned segments of the bolt anti-loosening line, and determine whether the angle is greater than the bolt detection threshold. If so, the bolt is loose; otherwise, proceed to step S6;

[0013] S6. Mapping and preprocessing the pixel coordinates corresponding to the three-dimensional image data of the bolt and the two-dimensional image data of the bolt to generate dictionary data, and extracting the pixel coordinates of the independent bolt area image, cropping and performing point cloud registration with the dictionary data to obtain matching bolt point cloud data;

[0014] S7. Align the registered bolt point cloud data with the dictionary data, extract the upper plane and the reference plane of the aligned bolt point cloud data and perform fitting to generate an upper plane model and a reference plane model, calculate the distance between the upper plane model and the reference plane model, and determine whether the distance exceeds the error threshold. If so, the bolt is loose, otherwise, the bolt is normal.

[0015] Furthermore, step S1 specifically includes:

[0016] A monocular structured light camera module was installed on the robotic arm of a subway operation and maintenance robot, and tested in a trench in a vehicle depot parking lot. The phase shift method combined with Gray code technology was used to collect two-dimensional image data and three-dimensional image data of bolts at key parts of subway bogies.

[0017] Furthermore, step S2 specifically includes:

[0018] S21, performing rotation transformation and mirror transformation on the bolt two-dimensional image data to obtain transformed bolt two-dimensional image data;

[0019] S22, performing brightness conversion, grayscale generation, and salt and pepper noise addition on the transformed bolt two-dimensional image data to obtain enhanced bolt two-dimensional image data;

[0020] S23, dividing the enhanced bolt two-dimensional image data into a training set and a test set in proportion, and marking label data of the training set to obtain a labeled training set;

[0021] S24. Introduce a dual convolution module at the backbone network end and a content-aware feature reconstruction module at the neck network end to build a BOLT-YOLO bolt detection model.

[0022] Among them, the dual convolution module is an efficient convolution block that includes group convolution and heterogeneous convolution, which is used to improve feature extraction efficiency and reduce computational complexity;

[0023] Content-aware feature reorganization module for feature upsampling in dense prediction tasks;

[0024] S25. Input the labeled training set into the BOLT-YOLO bolt detection model for training to obtain a trained BOLT-YOLO bolt detection model;

[0025] S26. Input the test set into the trained BOLT-YOLO bolt detection model to identify the key parts of the bolts and determine whether the key parts of the subway bogie are identified. If so, output the bolt image containing the key parts and number each key part. Otherwise, the bolt is lost.

[0026] Furthermore, in step S24, the BOLT-YOLO bolt detection model includes a backbone network end, a neck network end, and a head network end;

[0027] The backbone network includes a first convolutional layer, a first double convolutional layer, a first residual connection module, a second double convolutional layer, a second residual connection module, a third double convolutional layer, a third residual connection module, a fourth double convolutional layer, a fourth residual connection module, and a multi-scale feature aggregation pyramid.

[0028] The neck network end includes a first content-aware feature reorganization module, a first connection module, a fifth residual connection module, a second content-aware feature reorganization module, a second connection module, a sixth residual connection module, a fifth double convolutional layer, a third connection module, a seventh residual connection module, a sixth double convolutional layer, a fourth connection module, and an eighth residual connection module;

[0029] The head network end includes a large detection head, a medium detection head and a small detection head.

[0030] Furthermore, step S25 specifically includes:

[0031] S251. Input the labeled training set into the backbone network of the BOLT-YOLO bolt detection model for feature extraction to obtain feature maps of different scales.

[0032] S252. Input the feature maps of different scales into the neck network end of the BOLT-YOLO bolt detection model to perform feature fusion to obtain fused features of different scales.

[0033] S253. Input the fused features of different scales into the head network of the BOLT-YOLO bolt detection model for bolt identification, and obtain the trained BOLT-YOLO bolt detection model and the key parts of the subway bogie.

[0034] Furthermore, step S5 specifically includes:

[0035] S51, performing binarization processing on the two sections of bolt anti-loosening lines to obtain two binarized sections of bolt anti-loosening lines;

[0036] S52, dividing the two binary segments of the bolt anti-loosening line to be processed pixel points, and simultaneously obtaining eight neighborhood pixel points centered on the pixel to be processed, and determining whether to retain the pixel to be processed based on a refinement rule, thereby obtaining a refined bolt surface anti-loosening line and a refined fixed reference surface anti-loosening line;

[0037] S53, performing best straight line fitting on the refined bolt surface anti-loosening line and the refined fixed reference surface anti-loosening line using the least squares method, to obtain a first best fitting straight line for the refined bolt surface anti-loosening line and a second best fitting straight line for the refined fixed reference surface anti-loosening line;

[0038] S54. Calculate the angle between the first best-fit straight line and the second best-fit straight line, that is:

[0039]

[0040] Wherein, θ represents the angle between the first best-fit straight line and the second best-fit straight line, m1 represents the slope of the first best-fit straight line, and m2 represents the slope of the second best-fit straight line;

[0041] S55 , determining whether the angle between the first best fitting straight line and the second best fitting straight line is greater than a bolt detection threshold; if so, the bolt is loose; otherwise, executing step S6 .

[0042] Furthermore, the detailed rules in step S52 are:

[0043] If the pixel to be processed is an internal point, it will be retained;

[0044] If the pixel to be processed is a key internal point, it will be retained;

[0045] If the pixel to be processed is a boundary point, determine whether removing the pixel to be processed will affect the connectivity of the anti-loosening line. If so, retain it; otherwise, remove it.

[0046] If the pixel to be processed is an irrelevant boundary point and its removal does not affect the positional relationship of the anti-loosening line, it is removed; otherwise, it is retained;

[0047] If the pixel to be processed is the endpoint of the anti-loosening line and its removal affects the integrity of the anti-loosening line, it is retained; otherwise, it is removed.

[0048] Furthermore, step S53 specifically includes:

[0049] S531, extracting all pixel coordinates from the refined bolt surface anti-loosening line and the refined fixed reference surface anti-loosening line respectively to obtain two sets of point sets;

[0050] Among them, each set of points corresponds to a straight line;

[0051] S532. Fit the straight lines corresponding to each set of points using the least squares method to obtain a first best-fitting straight line for the refined bolt surface anti-loosening line and a second best-fitting straight line for the refined fixed reference surface anti-loosening line.

[0052] Furthermore, step S6 specifically includes:

[0053] S61, mapping the three-dimensional image data of the bolt to the corresponding pixel coordinates in the two-dimensional image data of the bolt, and performing filtering, downsampling and denoising processing to obtain dictionary data;

[0054] The dictionary data includes point cloud data of several types of bolts;

[0055] S62, extracting pixel coordinates of the independent bolt area image, and cropping the pixel coordinates of the independent bolt area image along the x-axis and y-axis of the dictionary data to obtain cropped local point cloud data;

[0056] S63, converting the cropped local point cloud data into a world coordinate system to obtain local point cloud data in the world coordinate system;

[0057] S64, using a principal component analysis method based on neighborhood search to calculate the covariance matrix of the local point cloud data in the world coordinate system, and extracting the principal component direction as the normal vector;

[0058] S65. Based on the normal vector, perform point cloud registration on the local point cloud data in the world coordinate system and the dictionary data to obtain registered bolt point cloud data.

[0059] Furthermore, step S7 specifically includes:

[0060] S71, performing rigid body transformation on the registered bolt point cloud data, and aligning the coordinates with the dictionary data to obtain aligned bolt point cloud data;

[0061] The aligned bolt point cloud data includes the bolt model and bolt height information;

[0062] S72. Extracting the upper plane and the reference plane of the aligned bolt point cloud data according to the z-axis range information of the dictionary data;

[0063] The z-axis range information of the dictionary data includes the standard value of the bolt height;

[0064] S73, fitting the upper plane and the reference plane of the aligned bolt point cloud data using the least squares method to generate an upper plane model and a reference plane model;

[0065] S74. Calculate the distance between the upper plane model and the reference plane model, that is:

[0066]

[0067] Where d represents the distance between the upper plane model and the reference plane model, x0, y0, and z0 represent the coordinates of the 0th point cloud data of the upper plane model on the x, V, and z axes, respectively, and A, B, C, and D represent the equation coefficients of the reference plane model, respectively;

[0068] S75. Calculate the error between the distance between the upper plane model and the reference plane model and the standard value of the bolt height in the dictionary data, and determine whether the error is greater than an error threshold. If so, the bolt is loose; otherwise, the bolt is normal.

[0069] The present invention has the following beneficial effects:

[0070] 1. This invention proposes a multi-dimensional vision-driven subway bogie bolt loosening detection method. Using a robot-mounted monocular structured light camera module to continuously capture 2D and 3D images of bolts at key locations on the subway bogie, combined with a bolt detection method, it can detect abnormal bolt conditions in real time.

[0071] 2. In 2D bolt inspection, deep learning methods are used to not only inspect bolts on subway bogies in multiple scenarios and models, but also to perform comprehensive discrimination between 2D and 3D inspection information. If a 2D image detects an abnormal bolt condition, 3D inspection is not wasted, thus improving inspection accuracy and reducing resource utilization.

[0072] 3. By processing and extracting features from the two-dimensional image of the bolt, the key features of the bolt can be extracted, enabling accurate analysis and identification of the bolt status;

[0073] 4. Applying this method to subway train inspection operations, that is, deploying it in operation and maintenance robots, can achieve comprehensive detection of bogie bolt loosening, thereby improving the safety and reliability of train operation. BRIEF DESCRIPTION OF THE DRAWINGS

[0074] Figure 1 This is a flow chart of a multi-dimensional vision-driven subway bogie bolt loosening detection method proposed in the present invention;

[0075] Figure 2 This is a schematic diagram of the structure of the BOLT-YOLO bolt detection model;

[0076] Figure 3 This is a schematic diagram of the bolt key part identification results of the BOLT-YOLO bolt detection model;

[0077] Figure 4 This is a schematic diagram of the bolt anti-loosening line recognition results of the Unet semantic segmentation network model;

[0078] Figure 5 This is a schematic diagram of the skeleton refinement of the bolt anti-loosening line;

[0079] Figure 6 This is a schematic diagram of the result of point cloud registration of local point cloud data in the world coordinate system;

[0080] Figure 7 This is a schematic diagram of the bolt point cloud height. DETAILED DESCRIPTION

[0081] The specific embodiments of the present invention are described below to facilitate understanding of the present invention by those skilled in the art. However, it should be clear that the present invention is not limited to the scope of the specific embodiments. For those skilled in the art, as long as various changes are within the spirit and scope of the present invention as defined and determined by the appended claims, these changes are obvious, and all inventions and creations utilizing the concepts of the present invention are protected.

[0082] like Figure 1 As shown, a method for detecting bolt loosening of a subway bogie driven by multi-dimensional vision includes the following steps S1-S7:

[0083] S1. Obtaining two-dimensional image data and three-dimensional image data of bolts at key locations of a subway bogie.

[0084] Specifically, step S1 includes:

[0085] A monocular structured light camera module was installed on the robotic arm of a subway operation and maintenance robot, and tested in a trench in a vehicle depot parking lot. The phase shift method combined with Gray code technology was used to collect two-dimensional image data and three-dimensional image data of bolts at key parts of subway bogies.

[0086] In this embodiment, a monocular structured light camera module consists of a high-resolution color area array camera and a projector. Its principle is to capture a 2D image of the object being measured using a phase-shift method combined with a Gray code method. The projector then projects a 2D image onto the object being measured. Because the coded image is modulated by the object's surface shape, it deforms. This deformed structured light is captured by the fixed-position color area array camera. The positional relationship between the color area array camera and the projector light source, as well as the degree of structured light deformation, allows for 3D reconstruction of the object being measured. Specifically, the monocular structured light camera module uses a phase-shift method combined with Gray code technology to capture 2D and 3D data of bolts in key areas of a subway bogie. A full-white Gray code method is used to obtain 2D image information of the bolts in these key bogie components, while a phase-shift method combined with Gray code is used to generate 3D data. 2,000 full-white Gray code images with a resolution of 1440×1080 were collected, and 1,000 corresponding 3D data sets in xyz format were generated using the phase-shift method combined with Gray code.

[0087] S2. Data enhancement is performed on the bolt 2D image data, and the enhanced bolt 2D image data is input into the BOLT-YOLO bolt detection model for recognition. It is determined whether the key parts of the subway bogie are recognized. If so, the bolt image containing the key parts is output and each key part is numbered. Otherwise, the bolt is lost.

[0088] Specifically, step S2 includes S21-S26:

[0089] S21 . Perform rotation transformation and mirror transformation on the two-dimensional image data of the bolt to obtain transformed two-dimensional image data of the bolt.

[0090] S22 , performing brightness transformation, grayscale generation, and salt and pepper noise addition on the transformed two-dimensional bolt image data to obtain enhanced two-dimensional bolt image data.

[0091] In this embodiment, steps S21-S22 are a data enhancement process, and after the data enhancement process, the two-dimensional image data is expanded to 2000 images, i.e., enhanced bolt two-dimensional image data, so that the BOLT-YOLO bolt detection model can be trained using the enhanced bolt two-dimensional image data.

[0092] S23. Divide the enhanced two-dimensional bolt image data into a training set and a test set in proportion, and mark the label data of the training set to obtain a labeled training set.

[0093] In this embodiment, the training set and the test set are divided into a ratio of 8:2; the training set is labeled using LabelImg data labeling software to generate label information, and the labels include the M24 bolt label as bolt 0, the M12 bolt label as bolt 1, the bolt loss position label as bolt loss, and the shaft end label as shaft end; in addition, the test set needs to be serialized and numbered here to be used for feedback of bolt failure location information.

[0094] S24. Introduce a dual convolution module at the backbone network end and a content-aware feature reconstruction module at the neck network end to build the BOLT-YOLO bolt detection model.

[0095] In this embodiment, the BOLT-YOLO bolt detection model is an improved YOLOv8 model, and its structure and connection relationship are as follows: Figure 2 As shown, it includes a backbone network end, a neck network end, and a head network end; among them, the backbone network end is used for feature extraction, the neck network end is used for feature fusion, and the head network end is responsible for target detection tasks, including predicting bounding boxes, categories, and confidence levels.

[0096] Among them, the backbone network end includes the first convolutional layer, the first double convolutional layer, the first residual connection module, the second double convolutional layer, the second residual connection module, the third double convolutional layer, the third residual connection module, the fourth double convolutional layer, the fourth residual connection module and the multi-scale feature aggregation pyramid; among them, the first double convolutional layer, the second double convolutional layer, the third double convolutional layer and the fourth double convolutional layer are all double convolution modules, that is, efficient convolution blocks including group convolution and heterogeneous convolution, which are used to improve feature extraction efficiency and reduce computational complexity. In addition, the purpose of setting up a dual convolution module at the backbone network end is: although the standard convolution performs feature extraction and channel fusion at the same time, its computational cost is high; although the depth-wise separable convolution significantly reduces the number of parameters by separating the two steps, first using 3×3 depth-wise convolution (DW convolution) to extract features for each channel independently, and then using 1×1 point-wise convolution (PW convolution) to fuse channels, it also weakens the information flow between channels; in addition, although the group convolution further reduces the amount of computation by grouping the convolution kernel and the channels of the input feature map, the feature extraction capability is limited due to the lack of information sharing between different groups; at the same time, although the heterogeneous convolution alternates between 3×3 and 1×1 convolutions in the convolution kernel to reduce complexity, there is discontinuity in the integration of cross-channel information, which affects the accuracy of the model. Therefore, based on the defects of the original YOLOv8 model and the advantages of group convolution and heterogeneous convolution, an efficient convolution block that includes group convolution and heterogeneous convolution is introduced. That is, dual convolution combines the working methods of group convolution and heterogeneous convolution. It uses some convolution kernels to perform 3×3 and 1×1 joint convolution, which is responsible for both local feature extraction and channel fusion, while the other convolution kernels only perform 1×1 convolution, further improving channel fusion efficiency. In this way, dual convolution not only reduces computational complexity, but also maintains efficient feature extraction capabilities, improves cross-channel information exchange, and effectively improves the overall performance and accuracy of the network.

[0097] Among them, the neck network end includes a first content-aware feature reorganization module, a first connection module, a fifth residual connection module, a second content-aware feature reorganization module, a second connection module, a sixth residual connection module, a fifth double convolutional layer, a third connection module, a seventh residual connection module, a sixth double convolutional layer, a fourth connection module, and an eighth residual connection module; among them, the first content-aware feature reorganization module and the second content-aware feature reorganization module are both content-aware feature reorganization modules, which are used to realize feature upsampling of dense prediction tasks; in addition, the advantages of introducing the content-aware feature reorganization module at the neck network end are: the content-aware feature reorganization (CARAFE) module is an efficient and lightweight feature upsampling method designed to improve the performance of dense prediction tasks (such as target detection, semantic segmentation and instance segmentation); compared with traditional upsampling methods, the content-aware feature reorganization module has significant advantages, specifically: first, it can aggregate contextual information through a larger receptive field, which solves the limitation of traditional bilinear interpolation that only relies on sub-pixel neighborhoods. This large-field feature aggregation enables more to be obtained during the upsampling process. The content-aware feature reconstruction module dynamically generates adaptive convolution kernels based on the content of the input feature map, which can perform specific processing according to the specific situation of each instance. This design enhances the flexibility of the model and enables it to make more accurate feature extraction for different contextual information. In addition, the content-aware feature reconstruction module also performs well in computational efficiency. Despite the introduction of adaptive convolution kernels, its computational overhead is extremely small and has almost no impact on the overall complexity of the network. Therefore, the content-aware feature reconstruction module can be easily integrated into the existing convolutional neural network architecture to improve the overall performance. In the benchmark tests of object detection, semantic segmentation, and instance segmentation, the content-aware feature reconstruction module brings 1.2% AP, 1.3% AP, and 1.8% mIoU improvements, respectively, while the computational overhead is almost negligible. These features make CARAFE an upsampling module that is both efficient and flexible.

[0098] Among them, the head network end includes a large detection head, a medium detection head and a small detection head.

[0099] S25. Input the labeled training set into the BOLT-YOLO bolt detection model for training to obtain a trained BOLT-YOLO bolt detection model.

[0100] In this embodiment, the training parameters of the BOLT-YOLO bolt detection model are: the number of training rounds n is set to 300 times, the batch-size is set to m = 4, and the learning rate is set to c = 0.001. After the training is completed, the optimal weight model is automatically selected and used as the model for bolt detection in key parts of the bogie. In addition, the specific training is as follows:

[0101] Specifically, step S25 includes S251-S253:

[0102] S251. Input the labeled training set into the backbone network of the BOLT-YOLO bolt detection model for feature extraction to obtain feature maps of different scales.

[0103] S252. Input feature maps of different scales into the neck network end of the BOLT-YOLO bolt detection model for feature fusion to obtain fused features of different scales.

[0104] S253. Input the fused features of different scales into the head network of the BOLT-YOLO bolt detection model for bolt identification, and obtain the trained BOLT-YOLO bolt detection model and the key parts of the subway bogie.

[0105] S26. Input the test set into the trained BOLT-YOLO bolt detection model to identify the key parts of the bolts and determine whether the key parts of the subway bogie are identified. If so, output the bolt image containing the key parts and number each key part. Otherwise, the bolt is lost.

[0106] In this embodiment, the test set is input into the trained BOLT-YOLO bolt detection model to identify the key parts of the bolts, such as the axle end. In addition, the specific operation process of the bolt image containing the key parts and numbering each key part is as follows: according to the position of the detected bolt detection frame, the bolt is judged in which key component of the bogie, and the identified bolt area is saved independently as an image through the region expansion technology; each bolt in the image is uniquely numbered according to the component to which it belongs, and the specific numbering process is sorted according to the pixel coordinate position of the center point of the bolt detection frame, that is: first, the upper left corner of the image is used to identify the bolt. As a benchmark, extract the pixel coordinates of the center points of all bolt detection frames; arrange them in order from small to large according to the x-axis, first process the detection frame closest to the upper left corner of the image, and arrange them to the right in sequence until the bolts in the current row are detected, and then process the next row downwards; scan the entire image line by line in this way, and assign a unique number to each bolt in turn; for bolts that do not belong to specific parts, number them directly according to their detection order, for example, the bolt closest to the upper left corner is numbered Bolt_0, and then increases in sequence; and for bolts located in specific parts (such as shaft end covers), number them according to the sorting rules within the area. Similarly, start numbering from the bolt closest to the upper left corner of the area to ensure that the bolts in the area have an independent numbering method; for example, the bolt closest to the upper left corner of the shaft end cover is numbered Axle_box_end_cover_bolt_0, and then increases in sequence. Therefore, the results of recognition using the BOLT-YOLO bolt detection model are as follows: Figure 3 As shown, Figure 3 The blue box in the middle is the shaft end, the red part is the identified bolt 0, and the green part is the identified bolt 1.

[0107] S3. Crop each numbered key part in the bolt image containing the key parts to obtain an independent bolt area image.

[0108] In this embodiment, the bolt image containing the key parts identified by the BOLT-YOLO bolt detection model, that is, the bolt image containing the accurate position, is cropped, that is, each numbered key part is cropped out as an independent bolt area image, so that it can be input into the trained Unet semantic segmentation network in the subsequent steps to identify the bolt anti-loosening line.

[0109] S4. Input the independent bolt area image into the trained Unet semantic segmentation network model for recognition, and determine whether two sections of bolt anti-loosening lines are recognized. If so, execute step S5; otherwise, execute step S6; wherein the two sections of bolt anti-loosening lines are the bolt surface anti-loosening line and the fixed reference surface anti-loosening line.

[0110] In this embodiment, Labelme data annotation software is used to annotate independent bolt area images, generate bolt-related label information, and train the Unet semantic segmentation network model so as to use the trained Unet semantic segmentation network model to identify the bolt anti-loosening line; wherein, the model training setting parameter information is: the number of training rounds n is set to 100 times, the batch-size is set to m=8, the learning rate is set to c=0.01, and the optimal weight after the training is completed is used for the semantic segmentation of the bolt anti-loosening line. Therefore, the recognition results of the bolt surface anti-loosening line line1 and the fixed reference surface anti-loosening line line2 are identified by the trained Unet semantic segmentation network model. Figure 4 As shown, Figure 4 The white part in the middle is the identified bolt anti-loosening wire. In summary, this step avoids the noise interference caused by traditional image processing and the defect that image binarization may not be able to effectively extract the anti-loosening wire position.

[0111] S5. After skeleton refinement processing is performed on the two sections of the bolt anti-loosening line, fitting is performed to obtain the best fitting straight line of the two thinned sections of the bolt anti-loosening line, and the angle between the best fitting straight lines of the two thinned sections of the bolt anti-loosening line is calculated. It is determined whether the angle is greater than the bolt detection threshold. If so, the bolt is loose. Otherwise, execute step S6.

[0112] In this embodiment, the skeleton refinement operation is performed on the identified bolt anti-loosening lines so as to more clearly fit the two anti-loosening lines and calculate their angle. If the angle is greater than 1°, it is judged to be loose. If the angle of the anti-loosening line cannot be effectively calculated or its state cannot be judged, the pixel coordinates of the independent bolt area image are extracted and mapped to the dictionary data for cropping, so as to obtain the bolt point cloud data of the area to further determine whether the bolt is loose.

[0113] Specifically, step S5 includes S51-S55:

[0114] S51. Binarize the two sections of bolt anti-loosening lines to obtain two binarized sections of bolt anti-loosening lines.

[0115] In this embodiment, the purpose of binarization is to facilitate the subsequent steps of skeleton refinement and extraction of the bolt anti-loosening line; wherein, skeleton refinement is an operation of a binary image, which aims to extract key structures, retain center pixels and remove edge pixels. Through this refinement processing, the skeleton structure of the anti-loosening line can be gradually extracted while maintaining its original shape, thereby providing an accurate basis for subsequent angle calculation.

[0116] S52. Divide the two binary segments of the pixel points to be processed of the bolt anti-loosening line, and simultaneously obtain the eight neighborhood pixel points centered on the pixel point to be processed. And according to the refinement rule, determine whether to retain the pixel point to be processed, and obtain the refined bolt surface anti-loosening line and the refined fixed reference surface anti-loosening line.

[0117] In this embodiment, the thinning process of the anti-loosening line image gradually thins the anti-loosening line by removing the edge portion of the image to reduce redundant information, and finally represents the anti-loosening line with a single-pixel wide line. The goal of the thinning is to ensure that the topological structure and geometric relationship of the anti-loosening line remain unchanged, while making it have the simplest skeleton. Therefore, if the processed skeleton is located at the center of the anti-loosening line and maintains the original topological relationship and detailed structure, the thinning process can be regarded as an idealized result. In addition, the core principle of the thinning process is similar to the corrosion operation. By analyzing the 8-neighborhood information of the pixel point, it is determined whether a certain point should be retained or eliminated. Ideally, the thinning should be performed without changing the connectivity and overall structure of the anti-loosening line skeleton. The specific thinning rules are as follows:

[0118] If the pixel to be processed is an internal point, it is retained; if the pixel to be processed is a key internal point, it is retained; if the pixel to be processed is a boundary point, determine whether eliminating the pixel to be processed affects the connectivity of the anti-loosening line. If so, it is retained, otherwise, it is eliminated; if the pixel to be processed is an irrelevant boundary point and does not affect the positional relationship of the anti-loosening line after elimination, it is eliminated, otherwise, it is retained; if the pixel to be processed is the endpoint of the anti-loosening line and affects the integrity of the anti-loosening line after elimination, it is retained, otherwise, it is eliminated.

[0119] like Figure 5 As shown, Figure 5 The two sections of bolt anti-loosening lines after skeleton refinement are shown. That is, through the skeleton refinement operation, it is ensured that the extracted anti-loosening line skeleton not only retains the original shape, but also simplifies its structure, while avoiding the loss of key connectivity and structural integrity.

[0120] S53. Use the least squares method to perform best straight line fitting on the refined bolt surface anti-loosening line and the refined fixed reference surface anti-loosening line to obtain a first best fitting straight line of the refined bolt surface anti-loosening line and a second best fitting straight line of the refined fixed reference surface anti-loosening line.

[0121] In this embodiment, when performing straight line fitting and angle calculation on the two thinned anti-loosening wires, each line must first be fitted independently. Since the thinned line is usually composed of a series of discrete pixel points, the least squares method can be used for fitting to extract the linear features of these points. Therefore, the least squares method can be used to calculate the best fitting line so that the sum of the squares of the distances from all points to the line is minimized. The specific fitting process is as follows:

[0122] Specifically, step S53 includes:

[0123] S531. Extract all pixel coordinates from the refined bolt surface anti-loosening line and the refined fixed reference surface anti-loosening line respectively to obtain two sets of point sets; each set of point sets corresponds to a straight line.

[0124] S532. Fit the straight lines corresponding to each set of points using the least squares method to obtain a first best-fitting straight line for the refined bolt surface anti-loosening line and a second best-fitting straight line for the refined fixed reference surface anti-loosening line.

[0125] S54. Calculate the angle between the first best-fit straight line and the second best-fit straight line, that is:

[0126]

[0127] Wherein, θ represents the angle between the first best-fit straight line and the second best-fit straight line, m1 represents the slope of the first best-fit straight line, and m2 represents the slope of the second best-fit straight line.

[0128] S55 , determining whether the angle between the first best fitting straight line and the second best fitting straight line is greater than a bolt detection threshold; if so, the bolt is loose; otherwise, executing step S6 .

[0129] In this embodiment, if the angle exceeds the set looseness detection threshold, the bolt is determined to be loose, and the coordinates of the bolt are saved and output; if the angle is less than the threshold, it is determined that the bolt cannot be accurately detected by the two-dimensional anti-loosening line and further three-dimensional detection is required; therefore, this step can exclude bolts that do not require three-dimensional detection, effectively reducing the computational burden of the system and improving efficiency.

[0130] S6. Map and preprocess the pixel coordinates corresponding to the bolt three-dimensional image data and the bolt two-dimensional image data to generate dictionary data. At the same time, extract the pixel coordinates of the independent bolt area image, crop and align them with the dictionary data to obtain matching bolt point cloud data.

[0131] Specifically, step S6 includes S61-S65:

[0132] S61. Map the three-dimensional image data of the bolt to the corresponding pixel coordinates in the two-dimensional image data of the bolt, and perform filtering, downsampling and denoising processing to obtain dictionary data; wherein the dictionary data is point cloud data of bolts including several models.

[0133] In this embodiment, in terms of three-dimensional point cloud processing, the point cloud model is first restored according to the three-dimensional coordinate information corresponding to the pixel points, and the model is subjected to point cloud filtering, downsampling, and denoising to reduce the computational complexity of point cloud processing; since the components of the subway bogie contain various types of bolts, it is necessary to produce various types of bolt point cloud templates (dictionary data) for subsequent point cloud alignment, that is, the point cloud is cropped using CloudCompare software to produce bolt point cloud templates. The specific process is as follows: first, the required bolt point cloud portion is cropped out to form a template for subsequent alignment; next, through the rigid body transformation of the bolt point cloud, the geometric model of the bolt is adjusted so that the reference plane and the upper plane are both parallel to the xoy plane; wherein the reference plane coincides with the xoy plane to ensure standardized alignment of the model; the above operation not only provides a unified coordinate reference for subsequent precise measurement and analysis, but also greatly simplifies the coordinate transformation and position comparison process in the detection, thereby improving the overall alignment accuracy and operational efficiency.

[0134] S62 , extracting pixel coordinates of the independent bolt area image, and cropping the pixel coordinates of the independent bolt area image along the x-axis and the y-axis of the dictionary data to obtain cropped local point cloud data.

[0135] In this embodiment, during the cropping process, only the x- and y-axis coordinates are cropped, while the z-axis values ​​are retained to ensure that the bolt height information is not affected. This cropping operation ensures that only the bolt area is included in the subsequent point cloud registration and height calculation, reducing the computational effort of point cloud processing.

[0136] S63 , performing a world coordinate system conversion on the cropped local point cloud data to obtain local point cloud data in the world coordinate system.

[0137] In this embodiment, the clipped local point cloud data is converted into a world coordinate system using the graduation value to ensure that the point cloud data can accurately correspond to the position in the actual space.

[0138] S64. Calculate the covariance matrix of the local point cloud data in the world coordinate system using a principal component analysis method based on neighborhood search, and extract the principal component direction as the normal vector.

[0139] In this embodiment, normal vector estimation is performed on the local point cloud data in the world coordinate system, where the normal vector is an important description of the local geometric features of the point cloud surface. The direction of the normal of the point cloud surface can be determined by normal vector estimation, providing the necessary geometric information for subsequent point cloud alignment and feature extraction.

[0140] S65. Based on the normal vector, perform point cloud registration on the local point cloud data in the world coordinate system and the dictionary data to obtain registered bolt point cloud data.

[0141] In this embodiment, the purpose of point cloud registration is to ensure that the posture of the registered point cloud is consistent with the template, to achieve the unity of position and posture, and to complete the conversion of the coordinate system to prepare for the subsequent bolt status analysis and evaluation. Figure 6 The process of point cloud registration of local point cloud data in the world coordinate system is demonstrated. The goal of point cloud registration is to match and align the local point cloud data in the world coordinate system to be detected with multiple pre-built bolt point cloud templates in the dictionary data. Since the template dictionary stores a variety of different types of bolt point cloud templates, they provide reference and comparison objects in the registration process. In addition, registration not only requires the precise alignment of the target point cloud (local point cloud data in the world coordinate system) to the template point cloud (dictionary data), but also requires a rigid body transformation to convert its coordinate system to the coordinate system of the template point cloud to achieve standardized representation.

[0142] In addition, to improve the accuracy and efficiency of registration, a coarse registration step based on feature point matching is added to the point cloud registration process. That is, by extracting the feature descriptor (FPFH) of the point cloud for preliminary alignment, a better initial pose is provided for the ICP point cloud registration algorithm to avoid falling into a local optimal solution (i.e., step S63). Secondly, in the specific ICP point cloud registration iterative process, a weighted closest point calculation strategy is also introduced to give higher weights to key areas of the point cloud (such as bolt heads or important edges), so that these areas are matched first during the registration process, thereby improving the registration accuracy (i.e., steps S64-S65).

[0143] S7. Align the registered bolt point cloud data with the dictionary data, extract the upper plane and the reference plane of the aligned bolt point cloud data and perform fitting to generate an upper plane model and a reference plane model, calculate the distance between the upper plane model and the reference plane model, and determine whether the distance exceeds the error threshold. If so, the bolt is loose, otherwise, the bolt is normal.

[0144] Specifically, step S7 includes:

[0145] S71. Performing rigid body transformation on the registered bolt point cloud data and aligning it with the coordinates of the dictionary data to obtain aligned bolt point cloud data; wherein the aligned bolt point cloud data includes bolt model and bolt height information.

[0146] In this embodiment, after the alignment in the above steps is completed, it is necessary to perform a rigid body transformation on the aligned bolt point cloud data so that its coordinate system is aligned with the coordinate system of the template point cloud. The specific operation is: calculate the rigid body transformation matrix, and convert the coordinate system of the aligned bolt point cloud data into the template coordinate system based on the rotation matrix and translation vector obtained during the alignment process; the transformed point cloud data is uniformly represented in the coordinate system of the template to achieve coordinate standardization and posture consistency; at the same time, the transformation operation needs to be optimized to ensure that the posture of the bolt point cloud is accurate and consistent with the template.

[0147] S72 . Extracting the upper plane and the reference plane of the aligned bolt point cloud data according to the z-axis range information of the dictionary data; wherein the z-axis range information of the dictionary data includes a standard value of the bolt height.

[0148] In this embodiment, after point cloud registration, the registered bolt point cloud data has been aligned to the template point cloud coordinate system through rigid body transformation. At this time, the reference plane of the point cloud is parallel to the xoy plane. Therefore, in this state, the upper plane and reference plane of the aligned bolt point cloud data (bolt) can be effectively screened using the z-axis range information included in the dictionary data. The reference plane is usually located at the bottom or contact surface of the bolt, while the upper plane is located at the top of the bolt head. The purpose of extracting the upper plane of the aligned bolt point cloud data is to ensure that the selected point set can accurately represent the actual geometric shape of the bolt.

[0149] S73. Fit the upper plane and the reference plane of the aligned bolt point cloud data using the least squares method to generate an upper plane model and a reference plane model.

[0150] In this embodiment, the upper plane of the aligned bolt point cloud data is fitted into a plane model through fitting techniques such as the least squares method, and the reference plane is fitted into the plane equation of the reference plane (reference plane model); the purpose of this is to facilitate the subsequent steps to calculate the vertical distance between the two planes, so as to determine the height status of the bolt. Among them, under ideal laboratory conditions, the measurement of this distance can directly reflect the state of the bolt; by measuring the vertical distance of the bolt plane relative to the reference plane, it can be determined whether the bolt is in a loose state; among them, the distance between the upper plane model and the reference plane model, that is, the height difference of the bolt point cloud, is as follows: Figure 7 shown.

[0151] S74. Calculate the distance between the upper plane model and the reference plane model, that is:

[0152]

[0153] Where d represents the distance between the upper plane model and the reference plane model, x0, y0, and z0 represent the coordinates of the 0th point cloud data of the upper plane model on the x, y, and z axes, respectively, and A, B, C, and D represent the equation coefficients of the reference plane model, respectively.

[0154] S75. Calculate the error between the distance between the upper plane model and the reference plane model and the standard value of the bolt height in the dictionary data, and determine whether the error is greater than an error threshold. If so, the bolt is loose; otherwise, the bolt is normal.

[0155] In this embodiment, since the coordinate information of the bolt will be stored in the overall point cloud of the dictionary data, by recording its coordinate information and converting it back into the corresponding two-dimensional image, and showing the faulty bolt with a red mark box, the system will combine the two-dimensional and three-dimensional detection results for comprehensive output, which can help maintenance personnel quickly locate the problem bolt and improve maintenance efficiency. In order to further enhance the accuracy and reliability of the detection, the system will also combine the two-dimensional and three-dimensional detection results to verify the status of the bolt from multiple dimensions and ensure the accuracy of loosening detection. Therefore, by combining the preliminary detection of two-dimensional images with the precise analysis of three-dimensional point clouds, the efficiency and accuracy of bolt loosening detection can be greatly improved. Especially under complex actual working conditions, dynamic reference plane technology and historical data comparison methods can effectively reduce the false alarm rate and ensure the detection system's rapid response and accurate judgment of loose bolts.

[0156] In summary, the present invention proposes a method for detecting loose bolts of subway bogies driven by multi-dimensional vision. First, the monocular structured light camera module carried by the robot continuously collects two-dimensional images and three-dimensional images of bolts at key parts of the subway bogie, and combines them with the bolt detection method to detect abnormal status of train bolts in real time; secondly, in the two-dimensional detection of bolts, the deep learning method is used not only to detect the bolts of subway bogies in multiple scenarios and multiple models, but also to perform comprehensive judgment of two-dimensional and three-dimensional detection information. If the two-dimensional image detects that the bolt status is abnormal, resources will not be wasted for three-dimensional detection, thereby improving detection accuracy and reducing resource utilization; in addition, by processing and feature extraction of the two-dimensional image of the bolt, the key features of the bolt can be extracted, and accurate analysis and judgment of the bolt status can be achieved; finally, the method is applied to subway train inspection operations, that is, deployed in the operation and maintenance robot to realize comprehensive detection of bogie bolt loosening, thereby improving the safety and reliability of train operation.

[0157] Specific embodiments are used in the present invention to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core ideas. At the same time, for those skilled in the art, according to the ideas of the present invention, there may be changes in the specific implementation methods and application scopes. In summary, the contents of this specification should not be understood as limiting the present invention.

[0158] Those skilled in the art will appreciate that the embodiments described herein are intended to help readers understand the principles of the present invention, and it should be understood that the scope of protection of the present invention is not limited to such specific descriptions and embodiments. Those skilled in the art can make various other specific variations and combinations based on the technical teachings disclosed in the present invention without departing from the essence of the present invention, and such variations and combinations are still within the scope of protection of the present invention.

Claims

1. A multi-dimensional vision-driven method for detecting loose bolts on subway bogies, characterized in that: The following steps are involved: S1. Acquire 2D image data and 3D image data of bolts at key locations of a subway bogie; S2. Data enhancement is performed on the bolt 2D image data, and the enhanced bolt 2D image data is input into the BOLT-YOLO bolt detection model for recognition. It is determined whether the key parts of the subway bogie are recognized. If so, the bolt image containing the key parts is output and each key part is numbered. Otherwise, the bolt is lost. S3, cropping each numbered key part in the bolt image containing the key parts to obtain an independent bolt area image; S4. Input the independent bolt area image into the trained Unet semantic segmentation network model for recognition, and determine whether two bolt anti-loosening lines are recognized. If so, execute step S5; otherwise, execute step S6; Among them, the two sections of bolt anti-loosening lines are the bolt surface anti-loosening line and the fixed reference surface anti-loosening line; S5. After skeleton thinning, the two segments of the bolt anti-loosening line are fitted to obtain the best fitting straight line of the two thinned segments of the bolt anti-loosening line, calculate the angle between the best fitting straight lines of the two thinned segments of the bolt anti-loosening line, and determine whether the angle is greater than the bolt detection threshold. If so, the bolt is loose; otherwise, proceed to step S6; S6. Mapping and preprocessing the pixel coordinates corresponding to the three-dimensional image data of the bolt and the two-dimensional image data of the bolt to generate dictionary data, and extracting the pixel coordinates of the independent bolt area image, cropping and performing point cloud registration with the dictionary data to obtain matching bolt point cloud data; S7. Align the registered bolt point cloud data with the dictionary data, extract the upper plane and the reference plane of the aligned bolt point cloud data and perform fitting to generate an upper plane model and a reference plane model, calculate the distance between the upper plane model and the reference plane model, and determine whether the distance exceeds the error threshold. If so, the bolt is loose, otherwise, the bolt is normal.

2. The multi-dimensional vision-driven subway bogie bolt loosening detection method according to claim 1 is characterized in that: Step S1 specifically includes: A monocular structured light camera module was installed on the robotic arm of a subway operation and maintenance robot, and tested in a trench in a vehicle depot parking lot. The phase shift method combined with Gray code technology was used to collect two-dimensional image data and three-dimensional image data of bolts at key parts of subway bogies.

3. The multi-dimensional vision-driven subway bogie bolt loosening detection method according to claim 2 is characterized in that: Step S2 specifically includes: S21, performing rotation transformation and mirror transformation on the bolt two-dimensional image data to obtain transformed bolt two-dimensional image data; S22, performing brightness conversion, grayscale generation, and salt and pepper noise addition on the transformed bolt two-dimensional image data to obtain enhanced bolt two-dimensional image data; S23, dividing the enhanced bolt two-dimensional image data into a training set and a test set in proportion, and marking label data of the training set to obtain a labeled training set; S24. Introduce a dual convolution module at the backbone network end and a content-aware feature reconstruction module at the neck network end to build a BOLT-YOLO bolt detection model. Among them, the dual convolution module is an efficient convolution block that includes group convolution and heterogeneous convolution, which is used to improve feature extraction efficiency and reduce computational complexity; Content-aware feature reorganization module for feature upsampling in dense prediction tasks; S25. Input the labeled training set into the BOLT-YOLO bolt detection model for training to obtain a trained BOLT-YOLO bolt detection model; S26. Input the test set into the trained BOLT-YOLO bolt detection model to identify the key parts of the bolts and determine whether the key parts of the subway bogie are identified. If so, output the bolt image containing the key parts and number each key part. Otherwise, the bolt is lost.

4. The multi-dimensional vision-driven subway bogie bolt loosening detection method according to claim 3 is characterized in that: In step S24, the BOLT-YOLO bolt detection model includes a backbone network end, a neck network end, and a head network end; The backbone network includes a first convolutional layer, a first double convolutional layer, a first residual connection module, a second double convolutional layer, a second residual connection module, a third double convolutional layer, a third residual connection module, a fourth double convolutional layer, a fourth residual connection module, and a multi-scale feature aggregation pyramid. The neck network end includes a first content-aware feature reorganization module, a first connection module, a fifth residual connection module, a second content-aware feature reorganization module, a second connection module, a sixth residual connection module, a fifth double convolutional layer, a third connection module, a seventh residual connection module, a sixth double convolutional layer, a fourth connection module, and an eighth residual connection module; The head network end includes a large detection head, a medium detection head and a small detection head.

5. The multi-dimensional vision-driven subway bogie bolt loosening detection method according to claim 4 is characterized in that: Step S25 specifically includes: S251. Input the labeled training set into the backbone network of the BOLT-YOLO bolt detection model for feature extraction to obtain feature maps of different scales. S252. Input the feature maps of different scales into the neck network end of the BOLT-YOLO bolt detection model to perform feature fusion to obtain fused features of different scales. S253. Input the fused features of different scales into the head network of the BOLT-YOLO bolt detection model for bolt identification, and obtain the trained BOLT-YOLO bolt detection model and the key parts of the subway bogie.

6. The multi-dimensional vision-driven subway bogie bolt loosening detection method according to claim 5 is characterized in that: Step S5 specifically includes: S51, performing binarization processing on the two sections of bolt anti-loosening lines to obtain two binarized sections of bolt anti-loosening lines; S52, dividing the two binary segments of the bolt anti-loosening line to be processed pixel points, and simultaneously obtaining eight neighborhood pixel points centered on the pixel to be processed, and determining whether to retain the pixel to be processed based on a refinement rule, thereby obtaining a refined bolt surface anti-loosening line and a refined fixed reference surface anti-loosening line; S53, performing best straight line fitting on the refined bolt surface anti-loosening line and the refined fixed reference surface anti-loosening line using the least squares method, to obtain a first best fitting straight line for the refined bolt surface anti-loosening line and a second best fitting straight line for the refined fixed reference surface anti-loosening line; S54. Calculate the angle between the first best-fit straight line and the second best-fit straight line, that is: Wherein, θ represents the angle between the first best-fit straight line and the second best-fit straight line, m1 represents the slope of the first best-fit straight line, and m2 represents the slope of the second best-fit straight line; S55 , determining whether the angle between the first best fitting straight line and the second best fitting straight line is greater than a bolt detection threshold; if so, the bolt is loose; otherwise, executing step S6 .

7. The multi-dimensional vision-driven subway bogie bolt loosening detection method according to claim 6 is characterized in that: The refinement rules in step S52 are: If the pixel to be processed is an internal point, it will be retained; If the pixel to be processed is a key internal point, it will be retained; If the pixel to be processed is a boundary point, determine whether removing the pixel to be processed will affect the connectivity of the anti-loosening line. If so, retain it; otherwise, remove it. If the pixel to be processed is an irrelevant boundary point and its removal does not affect the positional relationship of the anti-loosening line, it is removed; otherwise, it is retained; If the pixel to be processed is the endpoint of the anti-loosening line and its removal affects the integrity of the anti-loosening line, it is retained; otherwise, it is removed.

8. The multi-dimensional vision-driven subway bogie bolt loosening detection method according to claim 7 is characterized in that: Step S53 specifically includes: S531, extracting all pixel coordinates from the refined bolt surface anti-loosening line and the refined fixed reference surface anti-loosening line respectively to obtain two sets of point sets; Among them, each set of points corresponds to a straight line; S532. Fit the straight lines corresponding to each set of points using the least squares method to obtain a first best-fitting straight line for the refined bolt surface anti-loosening line and a second best-fitting straight line for the refined fixed reference surface anti-loosening line.

9. The multi-dimensional vision-driven subway bogie bolt loosening detection method according to claim 8 is characterized in that: Step S6 specifically includes: S61, mapping the three-dimensional image data of the bolt to the corresponding pixel coordinates in the two-dimensional image data of the bolt, and performing filtering, downsampling and denoising processing to obtain dictionary data; The dictionary data includes point cloud data of several types of bolts; S62, extracting pixel coordinates of the independent bolt area image, and cropping the pixel coordinates of the independent bolt area image along the x-axis and y-axis of the dictionary data to obtain cropped local point cloud data; S63, converting the cropped local point cloud data into a world coordinate system to obtain local point cloud data in the world coordinate system; S64, using a principal component analysis method based on neighborhood search to calculate the covariance matrix of the local point cloud data in the world coordinate system, and extracting the principal component direction as the normal vector; S65. Based on the normal vector, perform point cloud registration on the local point cloud data in the world coordinate system and the dictionary data to obtain registered bolt point cloud data.

10. The multi-dimensional vision-driven subway bogie bolt loosening detection method according to claim 9 is characterized in that: Step S7 specifically includes: S71, performing rigid body transformation on the registered bolt point cloud data, and aligning the coordinates with the dictionary data to obtain aligned bolt point cloud data; The aligned bolt point cloud data includes the bolt model and bolt height information; S72. Extracting the upper plane and the reference plane of the aligned bolt point cloud data according to the z-axis range information of the dictionary data; The z-axis range information of the dictionary data includes the standard value of the bolt height; S73, fitting the upper plane and the reference plane of the aligned bolt point cloud data using the least squares method to generate an upper plane model and a reference plane model; S74. Calculate the distance between the upper plane model and the reference plane model, that is: Where d represents the distance between the upper plane model and the reference plane model, x0, y0, and z0 represent the coordinates of the 0th point cloud data of the upper plane model on the x, y, and z axes, respectively, and A, B, C, and D represent the equation coefficients of the reference plane model, respectively; S75. Calculate the error between the distance between the upper plane model and the reference plane model and the standard value of the bolt height in the dictionary data, and determine whether the error is greater than an error threshold. If so, the bolt is loose; otherwise, the bolt is normal.

Images