Multidimensional vision combined driving subway bogie bolt looseness detection method

Abstract

The invention relates to the technical field of train monitoring, and discloses a metro bogie bolt looseness detection method based on multi-dimensional visual combined driving, which comprises the following steps: acquiring bolt two-dimensional image data and bolt three-dimensional image data of a key part of a metro bogie; performing data enhancement on the two-dimensional image data of the bolt, inputting the two-dimensional image data into a BOLT-YOLO bolt detection model to identify key parts of the subway bogie, outputting a bolt image containing the key parts, and numbering each key part; obtaining an independent bolt area image, inputting the image into the Unet semantic segmentation network model to identify two sections of bolt anti-loosening lines, and judging the loosening state through the angle change of the anti-loosening lines and the bolts; if the two-dimensional image information is insufficient, a three-dimensional point cloud model of the bolt is generated, and whether the bolt exceeds an error range or not is judged through accurate registration and datum plane height difference calculation; according to the method, the abnormal state of the train bolt can be detected in real time, the detection precision is improved, and the resource utilization rate is reduced.

Description

Technical Field

[0001] The invention relates to the technical field of train monitoring, and in particular to a method for detecting bolt loosening of a subway bogie driven by multi-dimensional vision. Background Art

[0002] With the rapid development of rail transit, rail trains play an increasingly important role in daily life. Trains not only undertake a large number of transportation tasks, but also ensure their safe operation. Especially in train inspection and maintenance work, it is crucial to ensure the safety of trains during operation. Among them, loose bolts may cause serious problems such as equipment failure or even wheel falling off, which seriously threatens the safe operation of trains. Therefore, loose bolt detection has become a key link in train maintenance.

[0003] Traditional bolt detection mainly relies on manual inspection. Workers identify the anti-loosening mark line on the bolt with the naked eye to determine whether the bolt is loose. However, this manual detection method is not only inefficient, but also highly subjective and easily affected by human factors. Therefore, there is an urgent need for a technology that can detect bolt loosening in real time and automatically to improve the accuracy and efficiency of bolt detection, thereby improving the safety and reliability of train operation. At present, some bolt loosening detection methods have been proposed, including those based on acoustic elasticity and torsion. The acoustic elasticity method reflects the change of the bolt stress state by measuring the change of the sound velocity in the solid with stress, while the torsion method detects the change of the preload force according to the approximate linear relationship between the bolt preload force and the torque to determine the looseness of the bolt. Although these two methods can detect bolt loosening, they have high requirements on the external environment and the measurement process is complicated, which is not easy to promote in complex practical application scenarios. In contrast, the bolt loosening detection method based on deep learning detects the anti-loosening mark line on the bolt and semantically segments the anti-loosening line to determine the looseness of the bolt. This method refines the skeleton of the anti-loosening marking line and extracts it, and quickly identifies and judges whether the bolt is loose by combining deep learning methods with image processing technology. Its advantage is that it has a fast detection speed and can process a large amount of data more efficiently. However, this method also has certain limitations, that is, when the camera angle is not appropriate, it may lead to a poor shooting angle, which in turn affects the recognition accuracy of the anti-loosening marking line and thus affects the judgment result. Summary of the invention

[0004] In view of the above-mentioned deficiencies in the prior art, the present invention provides a subway bogie bolt loosening detection method driven by a multi-dimensional vision system, which solves the problem of low detection accuracy caused by the detection limitations of two-dimensional detection or three-dimensional detection in the prior art of subway bogie bolt abnormality detection.

[0005] In order to achieve the above-mentioned object of the invention, the technical solution adopted by the present invention is:

[0006] A multi-dimensional vision-coordinated driven method for detecting bolt loosening of a subway bogie comprises the following steps:

[0007] S1. Obtaining two-dimensional image data and three-dimensional image data of bolts at key parts of a subway bogie;

[0008] S2, perform data enhancement on the bolt 2D image data, and input the enhanced bolt 2D image data into the BOLT-YOLO bolt detection model for identification, and determine whether the key parts of the subway bogie are identified. If so, output the bolt image containing the key parts and number each key part. Otherwise, the bolt is lost;

[0009] S3, cropping each numbered key part in the bolt image containing the key part to obtain an independent bolt area image;

[0010] S4, inputting the independent bolt area image into the trained Unet semantic segmentation network model for recognition, and determining whether two sections of bolt anti-loosening lines are recognized. If so, executing step S5, otherwise, executing step S6;

[0011] Among them, the two sections of bolt anti-loosening lines are the bolt surface anti-loosening line and the fixed reference surface anti-loosening line;

[0012] S5, performing skeleton thinning processing on the two sections of bolt anti-loosening lines and then fitting them to obtain the best fitting straight line of the two thinned sections of bolt anti-loosening lines, calculating the angle of the best fitting straight line of the two thinned sections of bolt anti-loosening lines, and judging whether the angle is greater than the bolt detection threshold, if so, the bolt is loose, otherwise, executing step S6;

[0013] S6, mapping and preprocessing the pixel coordinates corresponding to the bolt three-dimensional image data and the bolt two-dimensional image data to generate dictionary data, and extracting the pixel coordinates of the independent bolt area image, and performing cropping and point cloud registration with the dictionary data to obtain matching bolt point cloud data;

[0014] S7. Align the registered bolt point cloud data with the dictionary data, extract the upper plane and the reference plane of the aligned bolt point cloud data and fit them, generate the upper plane model and the reference plane model, calculate the distance between the upper plane model and the reference plane model, and determine whether the distance exceeds the error threshold. If so, the bolt is loose, otherwise, the bolt is normal.

[0015] Furthermore, step S1 specifically includes:

[0016] A monocular structured light camera module was installed on the mechanical arm of the subway operation and maintenance robot, and tested in the trench of the depot parking lot. The phase shift method combined with Gray code technology was used to collect two-dimensional image data and three-dimensional image data of bolts at key parts of the subway bogie.

[0017] Furthermore, step S2 specifically includes:

[0018] S21, performing rotation transformation and mirror transformation on the bolt two-dimensional image data to obtain transformed bolt two-dimensional image data;

[0019] S22, performing brightness transformation, grayscale image generation and salt and pepper noise addition on the transformed bolt two-dimensional image data to obtain enhanced bolt two-dimensional image data;

[0020] S23, dividing the enhanced bolt two-dimensional image data into a training set and a test set in proportion, and marking label data of the training set to obtain a labeled training set;

[0021] S24. Introduce a dual convolution module at the backbone network end and a content-aware feature reconstruction module at the neck network end to build a BOLT-YOLO bolt detection model.

[0022] Among them, the dual convolution module is an efficient convolution block that includes group convolution and heterogeneous convolution, which is used to improve feature extraction efficiency and reduce computational complexity;

[0023] Content-aware feature reorganization module for feature upsampling of dense prediction tasks;

[0024] S25, inputting the labeled training set into the BOLT-YOLO bolt detection model for training, thereby obtaining a trained BOLT-YOLO bolt detection model;

[0025] S26. Input the test set into the trained BOLT-YOLO bolt detection model to identify the key parts of the bolts, and determine whether the key parts of the subway bogie are identified. If so, output the bolt image containing the key parts and number each key part. Otherwise, the bolt is lost.

[0026] Further, in step S24, the BOLT-YOLO bolt detection model includes a backbone network end, a neck network end, and a head network end;

[0027] The backbone network includes a first convolutional layer, a first double convolutional layer, a first residual connection module, a second double convolutional layer, a second residual connection module, a third double convolutional layer, a third residual connection module, a fourth double convolutional layer, a fourth residual connection module and a multi-scale feature aggregation pyramid;

[0028] The neck network end includes a first content-aware feature reorganization module, a first connection module, a fifth residual connection module, a second content-aware feature reorganization module, a second connection module, a sixth residual connection module, a fifth double convolutional layer, a third connection module, a seventh residual connection module, a sixth double convolutional layer, a fourth connection module, and an eighth residual connection module;

[0029] The head network end includes a large detection head, a medium detection head and a small detection head.

[0030] Furthermore, step S25 specifically includes:

[0031] S251, inputting the labeled training set into the backbone network of the BOLT-YOLO bolt detection model for feature extraction to obtain feature maps of different scales;

[0032] S252, inputting feature maps of different scales into the neck network end of the BOLT-YOLO bolt detection model for feature fusion to obtain fusion features of different scales;

[0033] S253. Input the fused features of different scales into the head network of the BOLT-YOLO bolt detection model for bolt recognition, and obtain the trained BOLT-YOLO bolt detection model and the key parts of the subway bogie.

[0034] Furthermore, step S5 specifically includes:

[0035] S51, performing binarization processing on the two sections of bolt anti-loosening lines to obtain two binarized sections of bolt anti-loosening lines;

[0036] S52, dividing the two binary segments of the pixel points to be processed of the bolt anti-loosening line, and obtaining the eight neighborhood pixel points centered on the pixel point to be processed, and judging whether to retain the pixel point to be processed according to the refinement rule, to obtain the refined bolt surface anti-loosening line and the refined fixed reference surface anti-loosening line;

[0037] S53, using the least square method to perform best straight line fitting on the refined bolt surface anti-loosening line and the refined fixed reference surface anti-loosening line, to obtain a first best fitting straight line of the refined bolt surface anti-loosening line and a second best fitting straight line of the refined fixed reference surface anti-loosening line;

[0038] S54, calculating the angle between the first best fitting straight line and the second best fitting straight line, that is:

[0039]

[0040] Where θ represents the angle between the first best fit straight line and the second best fit straight line, m 1 represents the slope of the first best fit straight line, m 2 represents the slope of the second best fit straight line;

[0041] S55, determining whether the angle between the first best fitting straight line and the second best fitting straight line is greater than a bolt detection threshold, if so, the bolt is loose, otherwise, executing step S6.

[0042] Furthermore, the detailed rules in step S52 are:

[0043] If the pixel to be processed is an internal point, it will be retained;

[0044] If the pixel to be processed is a key internal point, it is retained;

[0045] If the pixel to be processed is a boundary point, determine whether removing the pixel to be processed will affect the connectivity of the anti-loosening line. If so, keep it; otherwise, remove it.

[0046] If the pixel to be processed is an irrelevant boundary point and does not affect the positional relationship of the anti-loosening line after removal, it will be removed, otherwise, it will be retained;

[0047] If the pixel to be processed is the endpoint of the anti-loosening line and will affect the integrity of the anti-loosening line after removal, it will be retained; otherwise, it will be removed.

[0048] Further, step S53 specifically includes:

[0049] S531, extracting all pixel point coordinates from the refined bolt surface anti-loosening line and the refined fixed reference surface anti-loosening line respectively to obtain two groups of point sets;

[0050] Among them, each set of points corresponds to a straight line;

[0051] S532. Fitting the straight lines corresponding to each set of points by the least square method to obtain the first best fitting straight line of the refined bolt surface anti-loosening line and the second best fitting straight line of the refined fixed reference surface anti-loosening line.

[0052] Further, step S6 specifically includes:

[0053] S61, mapping the three-dimensional image data of the bolt to the corresponding pixel coordinates in the two-dimensional image data of the bolt, and performing filtering, downsampling and denoising processing to obtain dictionary data;

[0054] The dictionary data includes point cloud data of several types of bolts;

[0055] S62, extracting pixel coordinates of the independent bolt area image, and cutting the pixel coordinates of the independent bolt area image along the x-axis and y-axis of the dictionary data to obtain cut local point cloud data;

[0056] S63, converting the cropped local point cloud data into a world coordinate system to obtain local point cloud data in the world coordinate system;

[0057] S64, using a principal component analysis method based on neighborhood search to calculate the covariance matrix of the local point cloud data in the world coordinate system, and extracting the principal component direction as the normal vector;

[0058] S65. Based on the normal vector, perform point cloud registration on the local point cloud data in the world coordinate system and the dictionary data to obtain registered bolt point cloud data.

[0059] Further, step S7 specifically includes:

[0060] S71, performing rigid body transformation on the registered bolt point cloud data, and aligning the coordinates with the dictionary data to obtain aligned bolt point cloud data;

[0061] Among them, the aligned bolt point cloud data includes the bolt model and bolt height information;

[0062] S72, extracting the upper plane and the reference plane of the aligned bolt point cloud data according to the z-axis range information of the dictionary data;

[0063] The z-axis range information of the dictionary data includes the standard value of the bolt height;

[0064] S73, fitting the upper plane and the reference plane of the aligned bolt point cloud data by using the least square method to generate an upper plane model and a reference plane model;

[0065] S74. Calculate the distance between the upper plane model and the reference plane model, that is:

[0066]

[0067] Where, d represents the distance between the upper plane model and the reference plane model, x 0 ,y 0 、z 0 They represent the coordinates of the 0th point cloud data of the upper plane model on the x, V, and z axes, respectively. A, B, C, and D represent the equation coefficients of the reference plane model, respectively.

[0068] S75. Calculate the error between the distance between the upper plane model and the reference plane model and the standard value of the bolt height in the dictionary data, and determine whether the error is greater than an error threshold. If so, the bolt is loose, otherwise, the bolt is normal.

[0069] The present invention has the following beneficial effects:

[0070] 1. The present invention proposes a method for detecting the looseness of subway bogie bolts driven by multi-dimensional vision. The monocular structured light camera module carried by the robot continuously collects the two-dimensional image and three-dimensional image of the bolts at the key parts of the subway bogie, and combines the bolt detection method to detect the abnormal state of the train bolts in real time;

[0071] 2. In the two-dimensional detection of bolts, the deep learning method can not only detect the bolts of subway bogies in multiple scenarios and models, but also make comprehensive judgments on two-dimensional and three-dimensional detection information. If the two-dimensional image detects that the bolt state is abnormal, resources will not be wasted on three-dimensional detection, which improves the detection accuracy and reduces resource utilization;

[0072] 3. By processing and extracting features from the two-dimensional image of the bolt, the key features of the bolt can be extracted, thus achieving accurate analysis and identification of the bolt status;

[0073] 4. Applying this method to subway train inspection operations, that is, deploying it in the operation and maintenance robot, can realize the comprehensive detection of the looseness of the bogie bolts, thereby improving the safety and reliability of train operation. BRIEF DESCRIPTION OF THE DRAWINGS

[0074] Figure 1 A schematic flow chart of a method for detecting bolt loosening of a subway bogie driven by multi-dimensional vision combined with the present invention;

[0075] Figure 2 This is a schematic diagram of the structure of the BOLT-YOLO bolt detection model;

[0076] Figure 3 This is a schematic diagram of the bolt key part recognition results of the BOLT-YOLO bolt detection model;

[0077] Figure 4 This is a schematic diagram of the bolt anti-loosening line recognition results of the Unet semantic segmentation network model;

[0078] Figure 5 This is a schematic diagram of the skeleton refinement of the bolt anti-loosening line;

[0079] Figure 6 It is a schematic diagram of the result of point cloud registration of local point cloud data in the world coordinate system;

[0080] Figure 7 This is a schematic diagram of the bolt point cloud height. DETAILED DESCRIPTION

[0081] The specific implementation modes of the present invention are described below so that those skilled in the art can understand the present invention. However, it should be clear that the present invention is not limited to the scope of the specific implementation modes. For those of ordinary skill in the art, as long as various changes are within the spirit and scope of the present invention as defined and determined by the attached claims, these changes are obvious, and all inventions and creations utilizing the concept of the present invention are protected.

[0082] like Figure 1 As shown, a method for detecting bolt loosening of a subway bogie driven by multi-dimensional vision includes the following steps S1-S7:

[0083] S1. Obtaining two-dimensional image data and three-dimensional image data of bolts at key positions of a subway bogie.

[0084] Specifically, step S1 includes:

[0085] A monocular structured light camera module was installed on the mechanical arm of the subway operation and maintenance robot, and tested in the trench of the depot parking lot. The phase shift method combined with Gray code technology was used to collect two-dimensional image data and three-dimensional image data of bolts at key parts of the subway bogie.

[0086] In this embodiment, the monocular structured light camera module is composed of a high-resolution color array camera and a projection optical machine. The principle is as follows: the projection optical machine projects a two-dimensional image of the object to be measured by the phase shift method combined with the Gray code method. Since the coded image is modulated by the shape of the object surface and deformed, the deformed structured light is captured by the color array camera at a fixed position. Therefore, the object to be measured can be reconstructed in three dimensions according to the positional relationship between the color array camera and the projection light source and the degree of deformation of the structured light. That is, the monocular structured light camera module is used to collect the two-dimensional and three-dimensional data of the bolts of the key parts of the subway bogie by using the phase shift method combined with the Gray code technology, wherein the full white Gray code is used to obtain the two-dimensional image information of the bolts of the key parts of the bogie, and the phase shift method combined with the Gray code method is used to solve the three-dimensional data. In addition, 2000 full white Gray code images were collected with an image resolution of 1440×1080, and 1000 corresponding xyz format three-dimensional data were generated by using the phase shift method combined with the Gray code method.

[0087] S2. Perform data enhancement on the bolt 2D image data, and input the enhanced bolt 2D image data into the BOLT-YOLO bolt detection model for identification, to determine whether the key parts of the subway bogie are identified. If so, output the bolt image containing the key parts and number each key part. Otherwise, the bolt is lost.

[0088] Specifically, step S2 specifically includes S21-S26:

[0089] S21, performing rotation transformation and mirror transformation on the two-dimensional image data of the bolt to obtain transformed two-dimensional image data of the bolt.

[0090] S22, performing brightness transformation, grayscale image generation and salt and pepper noise addition operations on the transformed bolt two-dimensional image data to obtain enhanced bolt two-dimensional image data.

[0091] In this embodiment, steps S21-S22 are a data enhancement process, and after the data enhancement process, the two-dimensional image data is expanded to 2000 images, that is, enhanced bolt two-dimensional image data, so that the BOLT-YOLO bolt detection model can be trained using the enhanced bolt two-dimensional image data.

[0092] S23, dividing the enhanced bolt two-dimensional image data into a training set and a test set in proportion, and marking the label data of the training set to obtain a labeled training set.

[0093] In this embodiment, the training set and the test set are divided in a ratio of 8:2; wherein, the training set is labeled using LabelImg data labeling software to generate label information, and the labels include the M24 bolt label as bolt 0, the M12 bolt label as bolt 1, the bolt loss position label as bolt loss, and the shaft end label as shaft end; in addition, the test set needs to be serialized and numbered here so as to be used to feedback the bolt failure position information.

[0094] S24. Introduce a dual convolution module at the backbone network end and a content-aware feature reconstruction module at the neck network end to build a BOLT-YOLO bolt detection model.

[0095] In this embodiment, the BOLT-YOLO bolt detection model is an improved YOLOv8 model, and its structure and connection relationship are as follows: Figure 2 As shown, it includes a backbone network end, a neck network end and a head network end; wherein, the backbone network end is used for feature extraction, the neck network end is used for feature fusion, and the head network end is responsible for target detection tasks, including predicting bounding boxes, categories and confidence levels.

[0096] Among them, the backbone network end includes a first convolutional layer, a first double convolutional layer, a first residual connection module, a second double convolutional layer, a second residual connection module, a third double convolutional layer, a third residual connection module, a fourth double convolutional layer, a fourth residual connection module and a multi-scale feature aggregation pyramid; among them, the first double convolutional layer, the second double convolutional layer, the third double convolutional layer and the fourth double convolutional layer are all double convolutional modules, that is, efficient convolution blocks including group convolution and heterogeneous convolution, which are used to improve feature extraction efficiency and reduce computational complexity. In addition, the purpose of setting up a dual convolution module at the backbone network end is: although the standard convolution performs feature extraction and channel fusion at the same time, its computational cost is high; although the depthwise separable convolution significantly reduces the number of parameters by separating the two steps, first using 3×3 depthwise convolution (DW convolution) to extract features for each channel independently, and then using 1×1 point convolution (PW convolution) to fuse channels, it also weakens the information flow between channels; in addition, although the group convolution further reduces the amount of calculation by grouping the channels of the convolution kernel and the input feature map, the feature extraction capability is limited due to the lack of information sharing between different groups; at the same time, although the heterogeneous convolution alternately uses 3×3 and 1×1 convolutions in the convolution kernel to reduce the complexity, there is discontinuity in the integration of cross-channel information, which affects the accuracy of the model. Therefore, based on the defects of the original YOLOv8 model and the advantages of group convolution and heterogeneous convolution, an efficient convolution block containing group convolution and heterogeneous convolution is introduced. That is, double convolution combines the working mode of group convolution and heterogeneous convolution, and uses part of the convolution kernel to perform 3×3 and 1×1 joint convolution, which is responsible for local feature extraction and channel fusion, while the other part of the convolution kernel only performs 1×1 convolution, further improving the channel fusion efficiency. In this way, double convolution not only reduces the computational complexity, but also maintains efficient feature extraction capabilities, improves cross-channel information exchange, and effectively improves the overall performance and accuracy of the network.

[0097] Among them, the neck network end includes a first content-aware feature reorganization module, a first connection module, a fifth residual connection module, a second content-aware feature reorganization module, a second connection module, a sixth residual connection module, a fifth double convolutional layer, a third connection module, a seventh residual connection module, a sixth double convolutional layer, a fourth connection module, and an eighth residual connection module; wherein the first content-aware feature reorganization module and the second content-aware feature reorganization module are both content-aware feature reorganization modules, which are used to realize feature upsampling of dense prediction tasks; in addition, the advantages of introducing the content-aware feature reorganization module at the neck network end are: the content-aware feature reorganization (CARAFE) module is an efficient and lightweight feature upsampling method designed to improve the performance of dense prediction tasks (such as target detection, semantic segmentation, and instance segmentation); compared with traditional upsampling methods, the content-aware feature reorganization module has significant advantages, specifically: first, it can aggregate contextual information through a larger receptive field, which solves the limitation that traditional bilinear interpolation only relies on sub-pixel neighborhoods. This large-field feature aggregation enables more information to be obtained during the upsampling process. The content-aware feature reorganization module can capture more spatial information, thereby improving the effect of feature reorganization. Secondly, the content-aware feature reorganization module implements a content-aware processing method. Different from the deconvolution method with fixed convolution kernels, the content-aware feature reorganization module dynamically generates adaptive convolution kernels according to the content of the input feature map, and can perform specific processing according to the specific situation of each instance. This design enhances the flexibility of the model, enabling it to make more accurate feature extraction for different contextual information. In addition, the content-aware feature reorganization module also performs well in computational efficiency. Although the adaptive convolution kernel is introduced, its computational overhead is extremely small and has almost no effect on the overall complexity of the network. Therefore, the content-aware feature reorganization module can be easily integrated into the existing convolutional neural network architecture to improve the overall performance. In the benchmark tests of target detection, semantic segmentation, and instance segmentation, the content-aware feature reorganization module brings 1.2% AP, 1.3% AP, and 1.8% mIoU improvements respectively, while the computational overhead is almost negligible. These features make CARAFE an upsampling module with both high efficiency and flexibility.

[0098] Among them, the head network end includes a large detection head, a medium detection head and a small detection head.

[0099] S25. Input the labeled training set into the BOLT-YOLO bolt detection model for training to obtain a trained BOLT-YOLO bolt detection model.

[0100] In this embodiment, the training parameters of the BOLT-YOLO bolt detection model are: the number of training rounds n is set to 300 times, the batch-size is set to m=4, the learning rate is set to c=0.001, and the optimal weight model is automatically selected after the training is completed, and the model is used as the model for bolt detection in key parts of the bogie. In addition, the specific training is as follows:

[0101] Specifically, step S25 specifically includes S251-S253:

[0102] S251. Input the labeled training set into the backbone network of the BOLT-YOLO bolt detection model for feature extraction to obtain feature maps of different scales.

[0103] S252. Input feature maps of different scales into the neck network end of the BOLT-YOLO bolt detection model for feature fusion to obtain fusion features of different scales.

[0104] S253. Input the fused features of different scales into the head network of the BOLT-YOLO bolt detection model for bolt recognition, and obtain the trained BOLT-YOLO bolt detection model and the key parts of the subway bogie.

[0105] S26. Input the test set into the trained BOLT-YOLO bolt detection model to identify the key parts of the bolts, and determine whether the key parts of the subway bogie are identified. If so, output the bolt image containing the key parts and number each key part. Otherwise, the bolt is lost.

[0106] In this embodiment, the test set is input into the trained BOLT-YOLO bolt detection model to identify the key parts of the bolts, such as identifying the shaft ends; in addition, the specific operation process of including the bolt image of the key parts and numbering each key part is as follows: according to the detected position of the bolt detection frame, determine which key component of the bogie the bolt is in, and through the region expansion technology, save the identified bolt area independently as an image; in the image, each bolt is uniquely numbered according to the component to which it belongs, and the specific numbering process is sorted according to the pixel coordinate position of the center point of the bolt detection frame, that is: first, the upper left corner of the image is used to identify the bolt. Take the image as the reference, extract the pixel coordinates of the center points of all bolt detection frames; arrange them in order from small to large according to the x-axis, first process the detection frame closest to the upper left corner of the image, and arrange them to the right in sequence until the bolts in the current row are detected, and then process the next row downwards; scan the entire image line by line in this way, and assign unique numbers to each bolt in turn; for bolts that do not belong to specific parts, number them directly according to their detection order, for example, the bolt closest to the upper left corner is numbered Bolt_0, and then increases in sequence; and for bolts located in specific parts (such as shaft end covers), they are numbered according to the sorting rules in the area. Similarly, start numbering from the bolt closest to the upper left corner of the area to ensure that the bolts in the area have independent numbering methods; for example, the bolt closest to the upper left corner of the shaft end cover is numbered Axle_box_end_cover_bolt_0, and then increases in sequence. Therefore, the results of recognition using the BOLT-YOLO bolt detection model are as follows: Figure 3 As shown, Figure 3 The blue box in the middle is the shaft end, the red part is the identified bolt 0, and the green part is the identified bolt 1.

[0107] S3. Crop each numbered key part in the bolt image containing the key parts to obtain an independent bolt area image.

[0108] In this embodiment, the bolt image containing the key parts identified by the BOLT-YOLO bolt detection model, that is, the bolt image containing the accurate position is cropped, that is, each numbered key part is cropped out as an independent bolt area image, so that it can be input into the trained Unet semantic segmentation network in the subsequent steps to identify the bolt anti-loosening line.

[0109] S4. Input the independent bolt area image into the trained Unet semantic segmentation network model for recognition, and determine whether two sections of bolt anti-loosening lines are recognized. If so, execute step S5, otherwise, execute step S6; wherein the two sections of bolt anti-loosening lines are the bolt surface anti-loosening line and the fixed reference surface anti-loosening line.

[0110] In this embodiment, Labelme data annotation software is used to annotate independent bolt area images, generate bolt-related label information, and train the Unet semantic segmentation network model so as to use the trained Unet semantic segmentation network model to identify the bolt anti-loosening line; wherein, the model training setting parameter information is: the number of training rounds n is set to 100 times, the batch-size is set to m=8, the learning rate is set to c=0.01, and the optimal weight after the training is completed is used for the semantic segmentation of the bolt anti-loosening line. Therefore, the recognition results of the bolt surface anti-loosening line line1 and the fixed reference surface anti-loosening line line2 are identified by the trained Unet semantic segmentation network model as shown in the figure. Figure 4 As shown, Figure 4 The white part in the middle is the identified bolt anti-loosening line. In summary, this step avoids the noise interference caused by traditional image processing and the defect that image binarization may not be able to effectively extract the anti-loosening line position.

[0111] S5. After skeleton refinement processing is performed on the two sections of bolt anti-loosening lines, fitting is performed to obtain the best fitting straight line of the two sections of the bolt anti-loosening lines, and the angle between the best fitting straight lines of the two sections of the bolt anti-loosening lines is calculated to determine whether the angle is greater than the bolt detection threshold. If so, the bolt is loose, otherwise, execute step S6.

[0112] In this embodiment, the skeleton refinement operation is performed on the identified bolt anti-loosening lines so as to fit the two anti-loosening lines more clearly and calculate their angle. If the angle is greater than 1°, it is judged to be loose. If it is impossible to effectively calculate the angle of the anti-loosening line or judge its state, the pixel coordinates of the independent bolt area image are extracted and mapped to the dictionary data for cropping, so as to obtain the bolt point cloud data of the area, so as to further judge whether the bolt is loose.

[0113] Specifically, step S5 specifically includes S51-S55:

[0114] S51, binarization processing is performed on the two sections of bolt anti-loosening lines to obtain two binarized sections of bolt anti-loosening lines.

[0115] In this embodiment, the purpose of binarization is to facilitate the subsequent steps of skeleton refinement and extraction of the bolt anti-loosening line; wherein, skeleton refinement is an operation of a binary image, which aims to extract key structures, retain center pixels and remove edge pixels. Through this refinement processing, the skeleton structure of the anti-loosening line can be gradually extracted while maintaining its original shape, thereby providing an accurate basis for subsequent angle calculations.

[0116] S52, dividing the two segments of the binary bolt anti-loosening line pixel points to be processed, and at the same time obtaining the 8-neighborhood pixel points centered on the pixel point to be processed, and judging whether to retain the pixel point to be processed according to the refinement rule, to obtain the refined bolt surface anti-loosening line and the refined fixed reference surface anti-loosening line.

[0117] In this embodiment, the thinning process of the anti-loosening line image removes the edge of the image, gradually thins the anti-loosening line, reduces redundant information, and finally represents the anti-loosening line with a single-pixel wide line; wherein, the goal of thinning is to ensure that the topological structure and geometric relationship of the anti-loosening line remain unchanged, while making it have the simplest skeleton; therefore, if the processed skeleton is located at the center of the anti-loosening line and maintains the original topological relationship and detailed structure, the thinning process can be regarded as an idealized result. In addition, the core principle of the thinning process is similar to the corrosion operation. By analyzing the 8-neighborhood information of the pixel point, it is determined whether a point should be retained or eliminated. Ideally, the thinning should be performed without changing the connectivity and overall structure of the anti-loosening line skeleton. The specific thinning rules are as follows:

[0118] If the pixel to be processed is an internal point, it is retained; if the pixel to be processed is a key internal point, it is retained; if the pixel to be processed is a boundary point, determine whether the elimination of the pixel to be processed affects the connectivity of the anti-loosening line, if so, retain it, otherwise, eliminate it; if the pixel to be processed is an irrelevant boundary point and does not affect the positional relationship of the anti-loosening line after elimination, then eliminate it, otherwise, retain it; if the pixel to be processed is the endpoint of the anti-loosening line and affects the integrity of the anti-loosening line after elimination, then retain it, otherwise, eliminate it.

[0119] like Figure 5 As shown, Figure 5 Two sections of bolt anti-loosening lines are shown after skeleton refinement. That is, through the skeleton refinement operation, it is ensured that the extracted anti-loosening line skeleton not only retains the original form, but also simplifies its structure, while avoiding the loss of key connectivity and structural integrity.

[0120] S53, using the least squares method to perform best straight line fitting on the refined bolt surface anti-loosening line and the refined fixed reference surface anti-loosening line, to obtain a first best fitting straight line for the refined bolt surface anti-loosening line and a second best fitting straight line for the refined fixed reference surface anti-loosening line.

[0121] In this embodiment, when performing straight line fitting and angle calculation on the two thinned anti-loosening lines, each line needs to be fitted independently first; since the thinned line is usually composed of a series of discrete pixel points, in order to extract the straight line features of these points, the least squares method can be used for fitting; therefore, the best fitting straight line can be calculated using the least squares method, so that the sum of the squares of the distances from all points to the straight line is minimized, and the specific fitting process is as follows:

[0122] Specifically, step S53 specifically includes:

[0123] S531. Extract all pixel point coordinates from the refined bolt surface anti-loosening line and the refined fixed reference surface anti-loosening line respectively to obtain two groups of point sets; wherein each group of point sets corresponds to a straight line.

[0124] S532. Fitting the straight lines corresponding to each set of points by the least square method to obtain the first best fitting straight line of the refined bolt surface anti-loosening line and the second best fitting straight line of the refined fixed reference surface anti-loosening line.

[0125] S54, calculating the angle between the first best fitting straight line and the second best fitting straight line, that is:

[0126]

[0127] Where θ represents the angle between the first best fit straight line and the second best fit straight line, m 1 represents the slope of the first best fit straight line, m 2 Represents the slope of the second best fit line.

[0128] S55, determining whether the angle between the first best fitting straight line and the second best fitting straight line is greater than a bolt detection threshold, if so, the bolt is loose, otherwise, executing step S6.

[0129] In this embodiment, if the angle exceeds the set looseness detection threshold, the bolt is determined to be loose, and the coordinates of the bolt are saved and output; if the angle is less than the threshold, it is determined that the bolt cannot be accurately detected by the two-dimensional anti-loosening line, and further three-dimensional detection is required; therefore, this step can exclude bolts that do not require three-dimensional detection, effectively reducing the computational burden of the system and improving efficiency.

[0130] S6. Map and preprocess the pixel coordinates corresponding to the bolt three-dimensional image data and the bolt two-dimensional image data to generate dictionary data, and extract the pixel coordinates of the independent bolt area image, and cut and align the dictionary data with the point cloud to obtain matching bolt point cloud data.

[0131] Specifically, step S6 specifically includes S61-S65:

[0132] S61, mapping the three-dimensional image data of the bolt to the corresponding pixel coordinates in the two-dimensional image data of the bolt, and performing filtering, down-sampling and denoising processing to obtain dictionary data; wherein the dictionary data is point cloud data of bolts including several models.

[0133] In this embodiment, in terms of three-dimensional point cloud processing, the point cloud model is first restored according to the three-dimensional coordinate information corresponding to the pixel points, and the model is subjected to point cloud filtering, downsampling, and denoising to reduce the amount of calculation for point cloud processing; since the components of the subway bogie include bolts of various types, it is necessary to make various types of bolt point cloud templates (dictionary data) for subsequent point cloud alignment, that is, the point cloud is cut using the cloudcompare software to make a bolt point cloud template, and the specific process is as follows: first, the required bolt point cloud portion is cut out to form a template for subsequent alignment; next, through the rigid body transformation of the bolt point cloud, the geometric model of the bolt is adjusted so that the reference plane and the upper plane are parallel to the xoy plane; wherein the reference plane coincides with the xoy plane to ensure standardized alignment of the model; the above operation not only provides a unified coordinate reference for subsequent precise measurement and analysis, but also greatly simplifies the coordinate transformation and position comparison process in the detection, thereby improving the overall alignment accuracy and operating efficiency.

[0134] S62 , extracting pixel coordinates of the independent bolt area image, and cropping the pixel coordinates of the independent bolt area image along the x-axis and the y-axis of the dictionary data to obtain cropped local point cloud data.

[0135] In this embodiment, during the cutting process, only the coordinates of the x-axis and y-axis are cut, while the values ​​of the z-axis are all retained to ensure that the height information of the bolt is not affected. Through the cutting operation, it is ensured that only the bolt area is included in the subsequent point cloud registration and height calculation process, reducing the amount of point cloud processing calculation.

[0136] S63, converting the cropped local point cloud data into a world coordinate system to obtain local point cloud data in the world coordinate system.

[0137] In this embodiment, the cut local point cloud data is converted into a world coordinate system using the graduation value to ensure that the point cloud data can accurately correspond to the position in the actual space.

[0138] S64. Calculate the covariance matrix of the local point cloud data in the world coordinate system using a principal component analysis method based on neighborhood search, and extract the principal component direction as a normal vector.

[0139] In this embodiment, normal vector estimation is performed on the local point cloud data in the world coordinate system, wherein the normal vector is an important description of the local geometric features of the point cloud surface. The direction of the normal of the point cloud surface can be determined by normal vector estimation, providing necessary geometric information for subsequent point cloud alignment and feature extraction.

[0140] S65. Based on the normal vector, perform point cloud registration on the local point cloud data in the world coordinate system and the dictionary data to obtain registered bolt point cloud data.

[0141] In this embodiment, the purpose of point cloud registration is to ensure that the posture of the registered point cloud is consistent with the template, achieve the unity of position and posture, and complete the conversion of the coordinate system to prepare for the subsequent bolt status analysis and evaluation. Figure 6 The process of point cloud registration of local point cloud data in the world coordinate system is demonstrated, where the goal of point cloud registration is to match and align the local point cloud data in the world coordinate system to be detected with multiple pre-built bolt point cloud templates in the dictionary data; since the template dictionary stores a variety of different types of bolt point cloud templates, they can provide reference and comparison objects in the registration process; in addition, registration not only requires the target point cloud (local point cloud data in the world coordinate system) to be accurately aligned to the template point cloud (dictionary data), but also requires a rigid body transformation to convert its coordinate system into the coordinate system of the template point cloud to achieve standardized representation.

[0142] In addition, in order to improve the accuracy and efficiency of registration, a coarse registration step based on feature point matching is added to the point cloud registration process, that is, by extracting the feature descriptor (FPFH) of the point cloud for preliminary alignment, a better initial pose is provided for the ICP point cloud registration algorithm to avoid falling into the local optimal solution (i.e., step S63). Secondly, in the specific ICP point cloud registration iteration process, a weighted closest point calculation strategy is also introduced to give higher weights to key areas of the point cloud (such as bolt heads or important edges), so that these areas are matched first during the registration process, thereby improving the registration accuracy (i.e., steps S64-S65).

[0143] S7. Align the registered bolt point cloud data with the dictionary data, extract the upper plane and the reference plane of the aligned bolt point cloud data and fit them, generate the upper plane model and the reference plane model, calculate the distance between the upper plane model and the reference plane model, and determine whether the distance exceeds the error threshold. If so, the bolt is loose, otherwise, the bolt is normal.

[0144] Specifically, step S7 specifically includes:

[0145] S71. Performing a rigid body transformation on the registered bolt point cloud data, and aligning the data with the coordinates of the dictionary data to obtain aligned bolt point cloud data; wherein the aligned bolt point cloud data includes bolt model and bolt height information.

[0146] In this embodiment, after the alignment in the above steps is completed, it is necessary to perform a rigid body transformation on the aligned bolt point cloud data so that its coordinate system is aligned with the coordinate system of the template point cloud. The specific operation is: calculate the rigid body transformation matrix, and convert the coordinate system of the aligned bolt point cloud data into the template coordinate system based on the rotation matrix and translation vector obtained during the alignment process; the transformed point cloud data is uniformly represented in the coordinate system of the template to achieve coordinate standardization and posture consistency; at the same time, the transformation operation needs to be optimized to ensure that the posture of the bolt point cloud is accurate and consistent with the template.

[0147] S72. Extracting the upper plane and the reference plane of the aligned bolt point cloud data according to the z-axis range information of the dictionary data; wherein the z-axis range information of the dictionary data includes the standard value of the bolt height.

[0148] In this embodiment, after the point cloud registration, the registered bolt point cloud data has been aligned to the template point cloud coordinate system through rigid body transformation. At this time, the reference plane of the point cloud is parallel to the xoy plane. Therefore, in this state, the upper plane and reference plane of the aligned bolt point cloud data (bolt) can be effectively screened out using the z-axis range information included in the dictionary data. Among them, the reference plane is usually located at the bottom or contact surface of the bolt, and the upper plane is located at the top of the bolt head. Among them, the purpose of extracting the upper plane of the aligned bolt point cloud data is to ensure that the selected point set can accurately represent the actual geometric shape of the bolt.

[0149] S73. Fitting the upper plane and the reference plane of the aligned bolt point cloud data by using the least square method to generate an upper plane model and a reference plane model.

[0150] In this embodiment, the upper plane of the aligned bolt point cloud data is fitted into a plane model through fitting techniques such as the least squares method, and the reference plane is fitted into the plane equation of the reference plane (reference plane model); the purpose of this is to facilitate the calculation of the vertical distance between the two planes in the subsequent steps, so as to determine the height state of the bolt. Among them, under ideal laboratory conditions, the measurement of this distance can directly reflect the state of the bolt; by measuring the vertical distance of the bolt plane relative to the reference plane, it can be determined whether the bolt is in a loose state; among them, the distance between the upper plane model and the reference plane model, that is, the height difference of the bolt point cloud, is as follows Figure 7 shown.

[0151] S74. Calculate the distance between the upper plane model and the reference plane model, that is:

[0152]

[0153] Where, d represents the distance between the upper plane model and the reference plane model, x 0 ,y 0 、z 0 They represent the coordinates of the 0th point cloud data of the upper plane model on the x, y, and z axes, respectively. A, B, C, and D represent the equation coefficients of the reference plane model, respectively.

[0154] S75. Calculate the error between the distance between the upper plane model and the reference plane model and the standard value of the bolt height in the dictionary data, and determine whether the error is greater than an error threshold. If so, the bolt is loose, otherwise, the bolt is normal.

[0155] In this embodiment, since the coordinate information of the bolt will be stored in the overall point cloud of the dictionary data, by recording its coordinate information and converting it back into the corresponding two-dimensional image, and using a red mark box to display the faulty bolt, the system will combine the two-dimensional and three-dimensional detection results for comprehensive output, which can help maintenance personnel quickly locate the problem bolt and improve maintenance efficiency. In order to further enhance the accuracy and reliability of the detection, the system will also combine the two-dimensional and three-dimensional detection results to verify the status of the bolt from multiple dimensions to ensure the accuracy of loose detection. Therefore, by combining the preliminary detection of the two-dimensional image with the precise analysis of the three-dimensional point cloud, the efficiency and accuracy of bolt loosening detection can be greatly improved. Especially under complex actual working conditions, the dynamic reference surface technology and the historical data comparison method can effectively reduce the false alarm rate and ensure the rapid response and accurate judgment of the detection system to loose bolts.

[0156] In summary, the present invention proposes a method for detecting the loosening of subway bogie bolts driven by multi-dimensional vision. First, the monocular structured light camera module carried by the robot continuously collects the two-dimensional image and three-dimensional image of the bolts at the key parts of the subway bogie, and combines the bolt detection method to detect the abnormal state of the train bolts in real time; secondly, in the two-dimensional detection of bolts, the deep learning method is used to not only detect the bolts of the subway bogies in multiple scenes and multiple models, but also to perform a comprehensive judgment of the two-dimensional and three-dimensional detection information. If the two-dimensional image detects that the bolt state is abnormal, resources will not be wasted for three-dimensional detection, which improves the detection accuracy and reduces the resource utilization; in addition, by processing and extracting the features of the two-dimensional image of the bolt, the key features of the bolt can be extracted, and the accurate analysis and judgment of the bolt state can be realized; finally, the method is applied to the subway train inspection operation, that is, deployed in the operation and maintenance robot to realize the comprehensive detection of the loosening of the bogie bolts, thereby improving the safety and reliability of the train operation.

[0157] The present invention uses specific embodiments to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

[0158] Those skilled in the art will appreciate that the embodiments described herein are intended to help readers understand the principles of the present invention, and should be understood that the protection scope of the present invention is not limited to such specific statements and embodiments. Those skilled in the art can make various other specific variations and combinations that do not deviate from the essence of the present invention based on the technical revelations disclosed by the present invention, and these variations and combinations are still within the protection scope of the present invention.

Claims

1. A method for detecting bolt loosening of subway bogies driven by multi-dimensional vision, characterized in that: The following steps are involved: S1. Obtaining two-dimensional image data and three-dimensional image data of bolts at key parts of a subway bogie; S2, perform data enhancement on the bolt 2D image data, and input the enhanced bolt 2D image data into the BOLT-YOLO bolt detection model for identification, and determine whether the key parts of the subway bogie are identified. If so, output the bolt image containing the key parts and number each key part. Otherwise, the bolt is lost; S3, cropping each numbered key part in the bolt image containing the key part to obtain an independent bolt area image; S4, inputting the independent bolt area image into the trained Unet semantic segmentation network model for recognition, and determining whether two sections of bolt anti-loosening lines are recognized. If so, executing step S5, otherwise, executing step S6; Among them, the two sections of bolt anti-loosening lines are the bolt surface anti-loosening line and the fixed reference surface anti-loosening line; S5, performing skeleton thinning processing on the two sections of bolt anti-loosening lines and then fitting them to obtain the best fitting straight line of the two thinned sections of bolt anti-loosening lines, calculating the angle of the best fitting straight line of the two thinned sections of bolt anti-loosening lines, and judging whether the angle is greater than the bolt detection threshold, if so, the bolt is loose, otherwise, executing step S6; S6, mapping and preprocessing the pixel coordinates corresponding to the bolt three-dimensional image data and the bolt two-dimensional image data to generate dictionary data, and extracting the pixel coordinates of the independent bolt area image, and performing cropping and point cloud registration with the dictionary data to obtain matching bolt point cloud data; S7. Align the registered bolt point cloud data with the dictionary data, extract the upper plane and the reference plane of the aligned bolt point cloud data and fit them, generate the upper plane model and the reference plane model, calculate the distance between the upper plane model and the reference plane model, and determine whether the distance exceeds the error threshold. If so, the bolt is loose, otherwise, the bolt is normal.

2. The method for detecting bolt loosening of subway bogies driven by multi-dimensional vision combination according to claim 1 is characterized in that: Step S1 specifically includes: A monocular structured light camera module was installed on the mechanical arm of the subway operation and maintenance robot, and tested in the trench of the depot parking lot. The phase shift method combined with Gray code technology was used to collect two-dimensional image data and three-dimensional image data of bolts at key parts of the subway bogie.

3. The method for detecting bolt loosening of subway bogies driven by multi-dimensional vision combination according to claim 2 is characterized in that: Step S2 specifically includes: S21, performing rotation transformation and mirror transformation on the bolt two-dimensional image data to obtain transformed bolt two-dimensional image data; S22, performing brightness transformation, grayscale image generation and salt and pepper noise addition on the transformed bolt two-dimensional image data to obtain enhanced bolt two-dimensional image data; S23, dividing the enhanced bolt two-dimensional image data into a training set and a test set in proportion, and marking label data of the training set to obtain a labeled training set; S24. Introduce a dual convolution module at the backbone network end and a content-aware feature reconstruction module at the neck network end to build a BOLT-YOLO bolt detection model. Among them, the dual convolution module is an efficient convolution block that includes group convolution and heterogeneous convolution, which is used to improve feature extraction efficiency and reduce computational complexity; Content-aware feature reorganization module for feature upsampling of dense prediction tasks; S25, inputting the labeled training set into the BOLT-YOLO bolt detection model for training, thereby obtaining a trained BOLT-YOLO bolt detection model; S26. Input the test set into the trained BOLT-YOLO bolt detection model to identify the key parts of the bolts, and determine whether the key parts of the subway bogie are identified. If so, output the bolt image containing the key parts and number each key part. Otherwise, the bolt is lost.

4. The method for detecting bolt loosening of subway bogies driven by multi-dimensional vision according to claim 3 is characterized in that: In step S24, the BOLT-YOLO bolt detection model includes a backbone network end, a neck network end, and a head network end; The backbone network includes a first convolutional layer, a first double convolutional layer, a first residual connection module, a second double convolutional layer, a second residual connection module, a third double convolutional layer, a third residual connection module, a fourth double convolutional layer, a fourth residual connection module and a multi-scale feature aggregation pyramid; The neck network end includes a first content-aware feature reorganization module, a first connection module, a fifth residual connection module, a second content-aware feature reorganization module, a second connection module, a sixth residual connection module, a fifth double convolutional layer, a third connection module, a seventh residual connection module, a sixth double convolutional layer, a fourth connection module, and an eighth residual connection module; The head network end includes a large detection head, a medium detection head and a small detection head.

5. The method for detecting bolt loosening of subway bogies driven by multi-dimensional vision according to claim 4 is characterized in that: Step S25 specifically includes: S251, inputting the labeled training set into the backbone network of the BOLT-YOLO bolt detection model for feature extraction to obtain feature maps of different scales; S252, inputting feature maps of different scales into the neck network end of the BOLT-YOLO bolt detection model for feature fusion to obtain fusion features of different scales; S253. Input the fused features of different scales into the head network of the BOLT-YOLO bolt detection model for bolt recognition, and obtain the trained BOLT-YOLO bolt detection model and the key parts of the subway bogie.

6. The method for detecting bolt loosening of subway bogies driven by multi-dimensional vision combination according to claim 5 is characterized in that: Step S5 specifically includes: S51, performing binarization processing on the two sections of bolt anti-loosening lines to obtain two binarized sections of bolt anti-loosening lines; S52, dividing the two binary segments of the pixel points to be processed of the bolt anti-loosening line, and obtaining the eight neighborhood pixel points centered on the pixel point to be processed, and judging whether to retain the pixel point to be processed according to the refinement rule, to obtain the refined bolt surface anti-loosening line and the refined fixed reference surface anti-loosening line; S53, using the least square method to perform best straight line fitting on the refined bolt surface anti-loosening line and the refined fixed reference surface anti-loosening line, to obtain a first best fitting straight line of the refined bolt surface anti-loosening line and a second best fitting straight line of the refined fixed reference surface anti-loosening line; S54, calculating the angle between the first best fitting straight line and the second best fitting straight line, that is: Wherein, θ represents the angle between the first best fit straight line and the second best fit straight line, m1 represents the slope of the first best fit straight line, and m2 represents the slope of the second best fit straight line; S55, determining whether the angle between the first best fitting straight line and the second best fitting straight line is greater than a bolt detection threshold, if so, the bolt is loose, otherwise, executing step S6.

7. The method for detecting bolt loosening of subway bogies driven by multi-dimensional vision according to claim 6 is characterized in that: The refinement rule in step S52 is: If the pixel to be processed is an internal point, it will be retained; If the pixel to be processed is a key internal point, it is retained; If the pixel to be processed is a boundary point, determine whether removing the pixel to be processed will affect the connectivity of the anti-loosening line. If so, keep it; otherwise, remove it. If the pixel to be processed is an irrelevant boundary point and does not affect the positional relationship of the anti-loosening line after removal, it will be removed, otherwise, it will be retained; If the pixel to be processed is the endpoint of the anti-loosening line and will affect the integrity of the anti-loosening line after removal, it will be retained; otherwise, it will be removed.

8. The method for detecting bolt loosening of subway bogies driven by multi-dimensional vision combination according to claim 7 is characterized in that: Step S53 specifically includes: S531, extracting all pixel point coordinates from the refined bolt surface anti-loosening line and the refined fixed reference surface anti-loosening line respectively to obtain two groups of point sets; Among them, each set of points corresponds to a straight line; S532. Fitting the straight lines corresponding to each set of points by the least square method to obtain the first best fitting straight line of the refined bolt surface anti-loosening line and the second best fitting straight line of the refined fixed reference surface anti-loosening line.

9. The method for detecting bolt loosening of subway bogies driven by multi-dimensional vision combination according to claim 8 is characterized in that: Step S6 specifically includes: S61, mapping the three-dimensional image data of the bolt to the corresponding pixel coordinates in the two-dimensional image data of the bolt, and performing filtering, downsampling and denoising processing to obtain dictionary data; The dictionary data includes point cloud data of several types of bolts; S62, extracting pixel coordinates of the independent bolt area image, and cutting the pixel coordinates of the independent bolt area image along the x-axis and y-axis of the dictionary data to obtain cut local point cloud data; S63, converting the cropped local point cloud data into a world coordinate system to obtain local point cloud data in the world coordinate system; S64, using a principal component analysis method based on neighborhood search to calculate the covariance matrix of the local point cloud data in the world coordinate system, and extracting the principal component direction as the normal vector; S65. Based on the normal vector, perform point cloud registration on the local point cloud data in the world coordinate system and the dictionary data to obtain registered bolt point cloud data.

10. The method for detecting bolt loosening of subway bogies driven by multi-dimensional vision according to claim 9 is characterized in that: Step S7 specifically includes: S71, performing rigid body transformation on the registered bolt point cloud data, and aligning the coordinates with the dictionary data to obtain aligned bolt point cloud data; Among them, the aligned bolt point cloud data includes the bolt model and bolt height information; S72, extracting the upper plane and the reference plane of the aligned bolt point cloud data according to the z-axis range information of the dictionary data; The z-axis range information of the dictionary data includes the standard value of the bolt height; S73, fitting the upper plane and the reference plane of the aligned bolt point cloud data by using the least square method to generate an upper plane model and a reference plane model; S74. Calculate the distance between the upper plane model and the reference plane model, that is: Where d represents the distance between the upper plane model and the reference plane model, x0, y0, and z0 represent the coordinates of the 0th point cloud data of the upper plane model on the x, y, and z axes, respectively, and A, B, C, and D represent the equation coefficients of the reference plane model, respectively; S75. Calculate the error between the distance between the upper plane model and the reference plane model and the standard value of the bolt height in the dictionary data, and determine whether the error is greater than an error threshold. If so, the bolt is loose, otherwise, the bolt is normal.

Images