Bolt looseness detection method and device, electronic equipment and storage medium

Abstract

The invention provides a bolt looseness detection method and device, electronic equipment and a storage medium. The method comprises the steps of obtaining three-dimensional point cloud data, collected by an inspection robot, of a bolt area of a railway vehicle; mapping the three-dimensional point cloud data into a two-dimensional plane image, and performing bolt target detection on the two-dimensional plane image to obtain a bolt target detection result in a two-dimensional image space; mapping a bolt target detection result to a three-dimensional point cloud space, and extracting corresponding point cloud region features; performing point cloud instance segmentation on the point cloud region to obtain segmented point cloud data; and performing bolt looseness detection according to the segmented point cloud data to obtain a bolt looseness detection result. According to the invention, full-automatic rapid detection of the looseness condition of the vehicle bottom bolt can be realized, manual intervention is not needed, the accuracy and robustness of bolt looseness detection are significantly improved, and the operation and maintenance cost of the railway vehicle is reduced at the same time.

Description

Technical Field

[0001] The present invention relates to the field of detection technologies, and provides a bolt loosening detection method, device, electronic device and storage medium. Background Art

[0002] With the rapid development of the rail transit industry, the demand for the detection and operation and maintenance of high-speed trains is increasing day by day. At present, the daily inspection of the bottom of high-speed trains mainly covers checking whether bolts are loose, whether there are serious wear, missing corners or scratches on components, etc. As a key link in the operation and maintenance of the railway system, daily inspection still faces many challenges at the present stage. In bad weather and complex environments, it is often difficult for manual inspection to detect and record potential damage areas in a timely manner, thus affecting the timeliness and effectiveness of maintenance work. In addition, the traditional inspection method relies on a large amount of manpower, which not only has a high working intensity, but also has low efficiency and high costs. Summary of the Invention

[0003] The present invention provides a bolt loosening detection method, device, electronic device and storage medium, so as to solve the defects that the existing inspection method relies on a large amount of manpower, has a high working intensity, low efficiency and high costs. The present invention can realize the full-automatic and rapid detection of the loosening condition of the bolts at the bottom of the vehicle without manual intervention, significantly improve the accuracy and robustness of bolt loosening detection, and at the same time reduce the operation and maintenance costs of rail vehicles.

[0004] The present invention provides a bolt loosening detection method, including: acquiring three-dimensional point cloud data of the bolt area of a rail vehicle collected by an inspection robot; mapping the three-dimensional point cloud data into a two-dimensional planar image, and performing bolt target detection on the two-dimensional planar image to obtain a bolt target detection result in the two-dimensional image space; mapping the bolt target detection result into the three-dimensional point cloud space, and extracting the corresponding point cloud region features; performing point cloud instance segmentation on the point cloud region to obtain the segmented point cloud data; and performing bolt loosening detection according to the segmented point cloud data to obtain a bolt loosening detection result.

[0005] According to the bolt loosening detection method provided by the present invention, after mapping the three-dimensional point cloud data into a two-dimensional planar image, it further includes: performing image enhancement on the two-dimensional planar image by using a preset image enhancement algorithm, so as to perform bolt target detection based on the image after image enhancement; the preset image enhancement algorithm includes a contrast stretching algorithm, a histogram equalization algorithm, a sharpening algorithm and a filtering algorithm.

[0006] A bolt loosening detection method provided by the present invention, the bolt target detection of the two-dimensional plane image to obtain the bolt target detection result in the two-dimensional image space, includes: inputting the two-dimensional plane image into a pre-trained YOLO model to obtain a plurality of detection bounding boxes output by the YOLO model; the detection bounding boxes are used to mark the positions and sizes of bolts in the two-dimensional plane image; the confidence levels corresponding to the plurality of detection bounding boxes are greater than a preset confidence threshold; inputting the detection bounding boxes into a pre-trained Faster RCNN model to obtain the bolt target detection result in the two-dimensional image space output by the Faster RCNN model; wherein, the pre-trained YOLO model is trained based on a YOLO object detection data set, and the Faster RCNN model is trained based on a Faster RCNN object detection data set.

[0007] A bolt loosening detection method provided by the present invention, the inputting the two-dimensional plane image into a pre-trained YOLO model to obtain a plurality of detection bounding boxes output by the YOLO model, includes: inputting the two-dimensional plane image into a first feature extraction layer for feature extraction to obtain the image features of the two-dimensional plane image output by the first feature extraction layer; inputting the image features into a candidate bounding box generation layer to obtain candidate bounding boxes output by the candidate bounding box generation layer; inputting the candidate bounding boxes into a first classification and regression layer for classification operation and regression operation to obtain bolt target positioning bounding boxes output by the first classification and regression layer; inputting the bolt target positioning bounding boxes into a first non-maximum suppression layer to remove duplicate bounding boxes to obtain a plurality of the detection bounding boxes output by the first non-maximum suppression layer.

[0008] A bolt loosening detection method provided by the present invention, the inputting the detection bounding boxes into a pre-trained Faster RCNN model to obtain the bolt target detection result in the two-dimensional image space output by the Faster RCNN model, includes: inputting the detection bounding boxes into a second feature extraction layer for feature extraction to obtain an extracted feature map output by the second feature extraction layer; inputting the extracted feature map into a region proposal layer for screening and adjustment to obtain candidate regions output by the region proposal layer; inputting the candidate regions into a region of interest pooling layer for feature map mapping to obtain region feature vectors output by the region of interest pooling layer; inputting the region feature vectors into a second classification and regression layer for classification operation and regression operation to obtain bolt target positioning regions output by the second classification and regression layer; inputting the bolt target positioning regions into a second non-maximum suppression layer to remove duplicate regions to obtain the bolt target detection result in the two-dimensional image space output by the second non-maximum suppression layer.

[0009] A bolt loosening detection method provided by the present invention, the point cloud instance segmentation of the point cloud region to obtain the segmented point cloud data includes: inputting the point cloud region features into a pre-trained Mask R-CNN model to obtain the segmentation mask of the bolt output by the Mask R-CNN model; inputting the segmentation mask of the bolt into a pre-trained U-Net model to obtain the segmented point cloud data output by the U-Net model; wherein the Mask R-CNN model is trained based on a Mask R-CNN point cloud instance segmentation dataset, and the U-Net model is trained based on a U-Net point cloud instance segmentation dataset.

[0010] A bolt loosening detection method provided by the present invention, the inputting the point cloud region features into a pre-trained Mask R-CNN model to obtain the segmentation mask of the bolt output by the Mask R-CNN model includes: inputting the point cloud region features into a pre-trained Faster RCNN model to obtain the bolt target detection result in the three-dimensional point cloud space output by the Faster RCNN model; inputting the bolt target detection result in the three-dimensional point cloud space into a segmentation mask generation layer to obtain the segmentation mask of the bolt output by the segmentation mask generation layer.

[0011] The present invention also provides a bolt loosening detection device, including: an acquisition module for acquiring three-dimensional point cloud data of the bolt region of the rail vehicle collected by an inspection robot; a target detection module for mapping the three-dimensional point cloud data into a two-dimensional plane image and performing bolt target detection on the two-dimensional plane image to obtain a bolt target detection result in the two-dimensional image space; an extraction module for mapping the bolt target detection result into the three-dimensional point cloud space and extracting corresponding point cloud region features; a segmentation module for performing point cloud instance segmentation on the point cloud region to obtain segmented point cloud data; a loosening detection module for performing bolt loosening detection based on the segmented point cloud data to obtain a bolt loosening detection result.

[0012] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein when the processor executes the computer program, the bolt loosening detection method described in any one of the above is implemented.

[0013] The present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the bolt loosening detection method described in any one of the above is implemented.

[0014] A method, device, electronic device, and storage medium for detecting bolt loosening provided by the present invention. The method includes: acquiring three-dimensional point cloud data of a bolt area of a rail vehicle collected by an inspection robot; mapping the three-dimensional point cloud data into a two-dimensional plane image, and performing bolt target detection on the two-dimensional plane image to obtain a bolt target detection result in the two-dimensional image space; mapping the bolt target detection result into the three-dimensional point cloud space, and extracting corresponding point cloud region features; performing point cloud instance segmentation on the point cloud region to obtain segmented point cloud data; and performing bolt loosening detection based on the segmented point cloud data to obtain a bolt loosening detection result. The present invention can achieve fully automatic and rapid detection of the loosening condition of the bolts under the vehicle, without manual intervention, significantly improving the accuracy and robustness of bolt loosening detection, and at the same time reducing the operation and maintenance costs of the rail vehicle. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] To more clearly illustrate the technical solutions in the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0016] Figure 1 It is a flowchart of the bolt loosening detection method provided by the present invention.

[0017] Figure 2 It is a structural diagram of the bolt loosening detection device provided by the present invention.

[0018] Figure 3 It is a structural diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0019] The following further describes in detail the embodiments of the present invention in conjunction with the drawings and embodiments. The following embodiments are used to illustrate the present invention, but cannot be used to limit the scope of the present invention.

[0020] With the rapid development of the rail transit industry, the demand for high-speed train inspection, operation, and maintenance is increasing day by day. However, considering the limitations of factors such as the operation scenario, lighting conditions, data clarity, and judgment criteria, the following several main problems will exist in the traditional manual inspection method for troubleshooting bolt faults in train operation and maintenance: At present, most of the troubleshooting of bolt faults still relies on manual inspection, and the reliability depends on the working state and proficiency of the maintenance personnel, with subjectivity taking the dominant position; there are a large number of bolts, with different specifications, sizes, shapes, and perspectives, making automatic detection difficult; the structure of the components under the vehicle is complex, and the background is messy when detecting bolt images, further increasing the difficulty of detecting key bolts; there is a high requirement for the real-time detection of bolt faults.

[0021] Please refer to Figure 1 , Figure 1 which is a schematic flow chart of the bolt loosening detection method provided by the present invention.

[0022] To solve the technical problems existing in the prior art, the present invention provides a bolt loosening detection method, including: 101: Obtain the three-dimensional point cloud data of the bolt area of the rail vehicle collected by the inspection robot.

[0023] In the inspection task at the bottom of the train, due to the narrow working space and complex environment, accurate positioning and navigation are the keys to ensuring the efficient completion of the inspection robot's tasks. For this reason, in this embodiment, the inspection robot is equipped with an advanced navigation system, which realizes precise autonomous positioning and navigation along the railway line or under the vehicle body in the depot by integrating technologies such as built-in global positioning system (GPS, Global Positioning System), inertial navigation system (INS, Inertial Navigation System), and visual simultaneous localization and mapping (SLAM, Simultaneous Localization and Mapping). This process not only ensures that the robot can accurately reach the preset inspection points, but also lays a solid foundation for the subsequent scanning and photographing work.

[0024] When the robot reaches the designated position, the high-definition cameras (including 3D cameras, depth cameras, laser line scan cameras, etc.) carried by it are immediately activated to perform a preliminary scan of the bottom of the vehicle. With the help of advanced image recognition algorithms, the robot can quickly identify the approximate position of the bolts. On this basis, further combined with the image recognition data such as 3D cameras, the three-dimensional coordinates of the bolts are accurately locked, making full preparations for the subsequent detailed scan.

[0025] After the bolt position is determined, the inspection robot accurately activates the relevant cameras to perform a fine scan of the bolt and its surrounding environment. During this process, the light emitted by the camera (such as laser or structured light) interacts with the bottom surface of the vehicle, and the reflected light is accurately captured and processed by the camera, and finally three-dimensional point cloud data containing detailed information such as the shape, size, and position of the bolt is generated. These data are presented in the form of a set of points, and each point accurately records its coordinate information in three-dimensional space.

[0026] The collected 3D point cloud data is transmitted to a remote server or cloud platform in real time via a high-speed wireless network. On the server side, professional software performs a series of pre-processing operations on the data, including key steps such as denoising, registration, and segmentation, to significantly improve data quality and ensure the accuracy of subsequent analysis. At the same time, with the help of advanced algorithms, feature extraction is performed on the processed point cloud data to accurately obtain key information such as the diameter, height, and tightening status of the bolts, providing solid and reliable data support for subsequent bolt status detection.

[0027] The inspection robot body adopts wireless communication, and the communication station and the grassroots station adopt optical fiber communication. Through digital encryption, identity authentication, address binding, terminal isolation, power control, hidden SSID and other related technical means, the security risks of wireless communication are eliminated. The robot body control signal and video signal are both network interfaces, which can realize the communication between the robot and the control background.

[0028] As a preferred embodiment, after mapping the three-dimensional point cloud data into a two-dimensional plane image, it also includes: using a preset image enhancement algorithm to enhance the two-dimensional plane image, so as to perform bolt target detection based on the image after image enhancement; the preset image enhancement algorithm includes a contrast stretching algorithm, a histogram equalization algorithm, a sharpening algorithm and a filtering algorithm.

[0029] Since the original 3D point cloud data has the characteristics of high density, high dimension and irregular distribution, directly processing it is not only computationally complex, but also extremely demanding on hardware resources. In order to effectively reduce the computational complexity and improve the processing efficiency, in this embodiment, the 3D point cloud data is first projected and mapped into a 2D plane image, while retaining the key information required for bolt detection as much as possible.

[0030] Specifically, we choose to project along the Z axis, that is, ignore the Z axis coordinates and only keep the X and Y axis coordinates. In this way, each 3D point (x, y, z) will be mapped to a 2D point (x, y). Of course, in order to retain the depth information, the Z axis coordinates can also be encoded into the grayscale value in some way, or the information content of the 2D image can be enhanced by calculating properties such as normal vectors.

[0031] After projection, a grayscale value needs to be assigned to each 2D point to generate a grayscale image. The grayscale value can be determined based on the Z-axis coordinate, normal vector, color, reflectivity, or other attributes of the 3D point. For example, if depth information is crucial for bolt detection, the Z-axis coordinate (or the depth value after a certain transformation) can be directly used as the grayscale value; if the normal vector can reflect the surface characteristics of the bolt, a certain component of the normal vector or the angle between it and a reference direction can be used as the grayscale value. The generation of the grayscale image is the basis for subsequent object detection, so it should be ensured that the grayscale value can accurately reflect the characteristic information of the 3D point. By comparing the detection effects under different grayscale value definition methods, the optimal grayscale value definition method can be selected.

[0032] During the projection process, record the mapping parameters of each point from 3D to 2D. These parameters include the coordinates of the original 3D point, the coordinates of the projected 2D point, and any attributes used to generate the grayscale value. The reason for recording these parameters is that they will be used in subsequent steps to restore the point cloud coordinates, perform depth estimation, or carry out other related processing.

[0033] The grayscale image generated by projection may have problems such as insufficient contrast, uneven brightness, or noise interference, resulting in the bolt part not being prominent enough in the image, thus affecting the subsequent object detection effect. To solve this problem, a preset image enhancement algorithm needs to be used to enhance the grayscale image to improve the contrast and visibility of the bolt in the image. The preset image enhancement algorithms include contrast stretching algorithm, histogram equalization algorithm, sharpening algorithm, and filtering algorithm.

[0034] The contrast stretching algorithm is specifically as follows: First, by adjusting the contrast of the image, the bright areas in the image become brighter and the dark areas become darker, thereby enhancing the contrast between different parts of the image. For bolt detection, contrast stretching can effectively improve the visibility of the bolt part in the image, making it more prominent and facilitating subsequent object detection. Contrast stretching can be achieved through methods such as linear transformation, piecewise linear transformation, or non-linear transformation. Linear transformation adjusts the brightness range of the image by multiplying by a constant factor. Piecewise linear transformation allows different linear transformations to be applied in different brightness intervals to achieve more refined contrast adjustment. Non-linear transformation provides a more flexible way to adjust contrast, but it may also cause image distortion or information loss.

[0035] The histogram equalization algorithm is specifically as follows: By adjusting the histogram of the image, the gray-scale distribution of the image is made more uniform, thereby enhancing the contrast of the image. For bolt detection, histogram equalization helps to reduce the uneven brightness in the image, improve the overall quality of the image, and make the bolt part more prominent in the image. Histogram equalization is achieved by calculating the gray-scale histogram of the image, finding the intervals where the gray-scale values are most concentrated, and redistributing the gray-scale values in these intervals across the entire gray-scale range, so as to make the gray-scale distribution of the image more uniform. For example, histogram equalization is implemented by calculating the cumulative distribution function (CDF, Cumulative Distribution Function) of the image. The CDF maps the gray-scale values of the image to new gray-scale values, making the distribution of the new gray-scale values more uniform.

[0036] The sharpening algorithm can enhance the edge and detail information in the image, making the contour of the bolt clearer. The filtering algorithm can be used to remove the noise and interference information in the image and improve the signal-to-noise ratio of the image.

[0037] Specifically, for example, the image can be first sharpened to enhance the edge information of the bolt; then filtered to remove the noise and interference information in the image; and finally, contrast stretching or histogram equalization can be performed to improve the contrast and visibility of the bolt in the image.

[0038] 102: Map the three-dimensional point cloud data into a two-dimensional planar image, and perform bolt target detection on the two-dimensional planar image to obtain the bolt target detection result in the two-dimensional image space.

[0039] As a preferred embodiment, performing bolt target detection on the two-dimensional planar image to obtain the bolt target detection result in the two-dimensional image space includes: inputting the two-dimensional planar image into a pre-trained YOLO model to obtain several detection bounding boxes output by the YOLO model; the detection bounding boxes are used to mark the position and size of the bolt in the two-dimensional planar image; the confidence levels corresponding to the several detection bounding boxes are greater than a preset confidence threshold; inputting the detection bounding boxes into a pre-trained Faster RCNN model to obtain the bolt target detection result in the two-dimensional image space output by the Faster RCNN model; wherein, the pre-trained YOLO model is trained based on the YOLO object detection dataset, and the Faster RCNN model is trained based on the Faster RCNN object detection dataset.

[0040] As a preferred embodiment, a two-dimensional planar image is input into a pre-trained YOLO model to obtain a number of detected bounding boxes output by the YOLO model, including: inputting the two-dimensional planar image into a first feature extraction layer for feature extraction to obtain the image features of the two-dimensional planar image output by the first feature extraction layer; inputting the image features into a candidate bounding box generation layer to obtain the candidate bounding boxes output by the candidate bounding box generation layer; inputting the candidate bounding boxes into a first classification and regression layer for classification and regression operations to obtain the bolt target positioning bounding boxes output by the first classification and regression layer; and inputting the bolt target positioning bounding boxes into a first non-maximum suppression layer to remove duplicate bounding boxes, thereby obtaining a number of detected bounding boxes output by the first non-maximum suppression layer.

[0041] In the bolt target detection task, in order to balance the detection speed and accuracy, in this embodiment, the YOLO model is used to improve the bolt target detection speed, and the Faster RCNN model is used to improve the bolt target detection accuracy.

[0042] The YOLO model transforms the target detection problem into a regression problem, and can output the position and category information of the target through a single forward propagation, which gives the YOLO model a significant advantage in detection speed and is especially suitable for scenarios with high real-time requirements.

[0043] Before training the YOLO model, it is first necessary to annotate the two-dimensional planar image generated by the projection and mark the position and size of the bolt. Open-source annotation software such as LabelImg and VIA can be selected as the annotation tool. During the annotation process, it is necessary to ensure that each bolt is accurately marked, and the size and shape of the annotation box can accurately reflect the actual size and shape of the bolt. After annotation, the YOLO target detection dataset needs to be divided into a training set, a validation set, and a test set. Generally, the training set is used to train the model, the validation set is used to adjust the model parameters and perform model selection, and the test set is used to evaluate the model performance. A reasonable dataset division is crucial for model training and evaluation.

[0044] Use the annotated training set to train the YOLO model. During the YOLO model training process, the YOLO model will learn how to identify bolt targets in the image and output detected bounding boxes and confidence levels. Through continuous iteration and optimization, the model will gradually improve the detection accuracy and generalization ability.

[0045] It is necessary to adjust hyperparameters such as model parameters and learning rate to obtain the best detection effect. By selecting hyperparameter settings, the detection performance and training efficiency of the model can be effectively improved.

[0046] The YOLO model includes a first feature extraction layer, a candidate bounding box generation layer, a first classification and regression layer, and a first non-maximum suppression layer.

[0047] In the target detection stage, the two-dimensional planar image to be detected is input into the trained YOLO model. The input image should be preprocessed and enhanced to improve the detection accuracy and robustness. The YOLO model will perform forward propagation on the input image and output the detection bounding box and confidence of the bolt. The detection bounding box will mark the position and size of the bolt in the image, and the confidence will represent the confidence level of the YOLO model in the detection result. The detection results are filtered according to the preset confidence threshold, and only the detection bounding boxes with confidence higher than the threshold are retained. This can reduce the cases of false detection and missed detection and improve the detection accuracy. The selection of the confidence threshold should be flexibly adjusted according to the actual needs and the performance of the YOLO model.

[0048] Specifically, the two-dimensional planar image first undergoes feature extraction through the first feature extraction layer. The first feature extraction layer usually consists of multiple convolutional layers and pooling layers, which are used to extract hierarchical feature representations from the input image. These feature representations will be used for subsequent candidate bounding box generation and classification regression tasks.

[0049] The candidate bounding box generation layer uses the image features of the two-dimensional planar image output by the first feature extraction layer to generate a series of candidate bounding boxes. These candidate bounding boxes will cover different regions in the image and are used for subsequent bolt detection. The generation of candidate bounding boxes is usually based on the concept of anchor boxes, that is, a series of bounding boxes with different sizes and aspect ratios are preset as candidate bounding boxes.

[0050] The first classification and regression layer performs classification operations and regression operations on each candidate bounding box to obtain the bolt target localization bounding box. The classification operation is used to determine whether the bolt target is contained in the candidate bounding box, and the regression operation is used to adjust the position and size of the candidate bounding box to more accurately locate the bolt target. The classification operation and regression operation are usually implemented based on a convolutional neural network, and the model is trained to learn how to classify and regress the candidate bounding boxes.

[0051] Finally, the first non-maximum suppression layer is used to remove duplicate bounding boxes. Since there are overlapping and competing relationships between the bolt target localization bounding boxes, the first non-maximum suppression layer will select the bounding box with the highest confidence as the detection bounding box and remove other bounding boxes with a high degree of overlap with it. This can reduce the cases of false detection and missed detection and improve the detection accuracy.

[0052] Through the processing of the YOLO model, the rough position and category information of the bolt can be obtained. After the YOLO stage, it is also necessary to introduce the Faster RCNN model for further fine detection.

[0053] As a preferred embodiment, the detected bounding box is input into a pre-trained Faster RCNN model to obtain the bolt target detection result in the two-dimensional image space output by the Faster RCNN model, including: inputting the detected bounding box into the second feature extraction layer for feature extraction to obtain the extracted feature map output by the second feature extraction layer; inputting the extracted feature map into the region proposal layer for screening and adjustment to obtain the candidate regions output by the region proposal layer; inputting the candidate regions into the region of interest pooling layer for feature map mapping to obtain the region feature vector output by the region of interest pooling layer; inputting the region feature vector into the second classification and regression layer for classification and regression operations to obtain the bolt target localization region output by the second classification and regression layer; inputting the bolt target localization region into the second non-maximum suppression layer to remove duplicate regions, and obtaining the bolt target detection result in the two-dimensional image space output by the second non-maximum suppression layer.

[0054] In this embodiment, in the Faster RCNN stage, there is no need to re-annotate the dataset. Since the images have been annotated in the YOLO stage, and this annotation information can be used for the training and testing of Faster RCNN. However, it should be noted that since the input sizes and formats of YOLO and Faster RCNN may be different, appropriate cropping, scaling, or padding operations need to be performed on the images to meet the input requirements of the second feature extraction layer of Faster RCNN.

[0055] Use the labeled training set (or the data processed through the YOLO stage) to train the Faster RCNN model. During the training process of the Faster RCNN model, the Faster RCNN model will learn how to generate high-quality candidate regions and perform classification and regression operations on these candidate regions. Hyperparameters such as model parameters and learning rate need to be adjusted to obtain the best detection effect. At the same time, attention should also be paid to the parameter sharing problem between the RPN layer and the Fast RCNN layer.

[0056] The Faster RCNN model includes a second feature extraction layer, a region proposal layer, a region of interest pooling layer, a second classification and regression layer, and a second non-maximum suppression layer.

[0057] In the target detection stage, the detected bounding box output in the YOLO stage is used as the input and input into the trained Faster RCNN model. The Faster RCNN model will perform further processing and fine detection on each detected bounding box.

[0058] Specifically, the detected bounding box first undergoes feature extraction through the second feature extraction layer.

[0059] The region proposal layer further filters and refines using the extracted feature map output by the second feature extraction layer. The region proposal layer will generate a series of more accurate candidate regions, which will cover the more likely bolt target positions in the image.

[0060] Then, the region of interest pooling layer maps the candidate regions onto the feature map and extracts fixed-size feature vectors. These regional feature vectors will be used for subsequent classification and regression operations. The region of interest pooling layer achieves the precise alignment of the feature map and the candidate regions, further improving the detection accuracy.

[0061] Next, the second classification and regression layer performs classification and regression operations on the regional feature vectors. The classification operation is used to determine whether the candidate region contains a bolt target, and the regression operation is used to adjust the position and size of the candidate region to more accurately locate the bolt target.

[0062] Finally, the second non-maximum suppression layer is used to remove duplicate regions. Due to the overlapping and competing relationships between the bolt target localization regions, the second non-maximum suppression layer will screen the bolt target localization regions according to the confidence threshold, and only retain the bolt target localization regions with confidence higher than the threshold. This can further improve the detection accuracy and output the bolt target detection results in the two-dimensional image space, including the precise position and category information of the bolt.

[0063] 103: Map the bolt target detection result to the three-dimensional point cloud space and extract the corresponding point cloud region features.

[0064] In this embodiment, the bolt target detection result can be mapped into the three-dimensional point cloud space, and then the corresponding point cloud region features can be extracted. It specifically includes the coordinate transformation and region extraction processes.

[0065] Among them, the coordinate transformation is a key step to accurately map the bolt target detection result in the two-dimensional image space to the three-dimensional point cloud space. This process relies on the previously recorded mapping parameters, which describe the correspondence between the original three-dimensional point cloud and the projected two-dimensional image.

[0066] The mapping parameter restoration is to convert the coordinates of the detection box on the two-dimensional image (usually pixel coordinates) back to the coordinates in the three-dimensional space. This process involves inverse projection calculation, that is, according to the model and mapping parameters, the two-dimensional points are inversely mapped into the three-dimensional space.

[0067] For each two-dimensional point in the detection box, its corresponding point in the original three-dimensional point cloud needs to be found. This is usually achieved by traversing the point cloud data and calculating the position of each point projected onto the image plane according to the mapping parameters. When the points within the two-dimensional detection box are projected back into the three-dimensional space, the approximate position and size of the detection box in the three-dimensional space can be determined according to the positions of these points.

[0068] After the coordinate transformation is completed, it is necessary to extract the corresponding point cloud region in the three-dimensional point cloud space. This process includes two main steps: point cloud cropping and region filtering.

[0069] According to the transformed three-dimensional coordinates, it is necessary to crop a point cloud region containing the object of interest (such as a bolt) from the original point cloud. According to the position and size of the detection box in the three-dimensional space, a cropping region containing all the points within the detection box is defined. Then, traverse the original point cloud data and retain the points located within the cropping region to form the cropped point cloud.

[0070] The cropped point cloud may still contain some noise points or outliers, which will affect the subsequent point cloud segmentation and processing effects. Therefore, it is necessary to perform filtering on the cropped point cloud.

[0071] When performing filtering, it is necessary to select appropriate filtering parameters according to the specific situation of the point cloud and the requirements of subsequent processing, such as the neighborhood size, distance threshold, etc.

[0072] In the object detection step, the position of the bolt has been successfully located, which lays a solid foundation for subsequent point cloud processing and analysis. Next, the region where the bolt is located will be accurately extracted from the preprocessed point cloud data for more in-depth operations such as point cloud segmentation, feature extraction, and recognition.

[0073] 104: Perform point cloud instance segmentation on the point cloud region to obtain the segmented point cloud data.

[0074] As a preferred embodiment, performing point cloud instance segmentation on the point cloud region to obtain the segmented point cloud data includes: inputting the point cloud region features into a pre-trained Mask R-CNN model to obtain the segmentation mask of the bolt output by the Mask R-CNN model; inputting the segmentation mask of the bolt into a pre-trained U-Net model to obtain the segmented point cloud data output by the U-Net model; where the Mask R-CNN model is trained based on the Mask R-CNN point cloud instance segmentation dataset, and the U-Net model is trained based on the U-Net point cloud instance segmentation dataset.

[0075] As a preferred embodiment, inputting the point cloud region features into a pre-trained Mask R-CNN model to obtain the segmentation mask of the bolt output by the Mask R-CNN model includes: inputting the point cloud region features into a pre-trained Faster RCNN model to obtain the bolt object detection result in the three-dimensional point cloud space output by the Faster RCNN model; inputting the bolt object detection result in the three-dimensional point cloud space into the segmentation mask generation layer to obtain the segmentation mask of the bolt output by the segmentation mask generation layer.

[0076] In the three-dimensional point cloud bolt loosening detection, point cloud instance segmentation is a crucial step. It requires the algorithm to accurately separate the bolt target from the complex background and provide high-quality input data for subsequent detection and evaluation. To achieve this goal, in this embodiment, the Mask R-CNN model and the U-Net model are used to improve the accuracy and robustness of point cloud instance segmentation.

[0077] In the bolt loosening detection task, the Mask R-CNN model can accurately identify the position and category of the bolt and generate a high-quality segmentation mask, providing strong support for subsequent processing.

[0078] In the Mask R-CNN stage, the input point cloud region features are subjected to bolt target detection by a pre-trained Faster RCNN model to obtain the bolt target detection results in the three-dimensional point cloud space. Specifically, first, a feature extraction network is used for deep feature extraction. The feature extraction network usually adopts a deep convolutional neural network (CNN), such as ResNet, VGG, etc. These networks gradually extract the spatial features and semantic information of the point cloud data by stacking multiple convolutional layers and pooling layers. After feature extraction, a region proposal layer is used to generate candidate regions. The region proposal layer is a lightweight convolutional neural network. It receives the output of the feature extraction network as input and generates a series of candidate regions that may contain the target through a sliding window and anchor mechanism. These candidate regions not only contain the approximate position and size information of the target but also have a certain class confidence, providing important clues for subsequent target detection and segmentation. After obtaining the candidate regions, the feature map is aligned with the candidate regions through the region of interest pooling layer. The region of interest pooling layer maps each candidate region to the feature map and converts candidate regions of different sizes into a fixed-size feature map through pooling operations. In this way, the candidate regions can be classified and regressed using the classification and regression layers. Through the processing of the classification and regression layers, the accurate position and category information of the bolt can be obtained.

[0079] The Mask R-CNN model also introduces a segmentation mask generation layer to generate the segmentation mask of the bolt. The segmentation mask generation layer receives the bolt target detection results in the three-dimensional point cloud space as input and restores the feature map to the original image size through a series of convolutional layers and transposed convolutional layers, and outputs the probability value of each pixel belonging to the bolt. Through threshold processing, the accurate segmentation mask of the bolt can be obtained. This mask not only contains the position and shape information of the bolt but also has high accuracy and robustness, providing strong support for subsequent point cloud processing and bolt loosening detection.

[0080] To further improve the segmentation accuracy, in this embodiment, the segmentation mask of the bolt output by the Mask R-CNN model is used as the input of the U-Net model for refinement processing. The U-Net model adopts an image segmentation network. Through the encoder-decoder structure and skip connections, it can capture the detailed information in the image and achieve precise segmentation of the target.

[0081] In the U-Net stage, the encoder part gradually extracts the deep features of the input image through convolutional layers and pooling layers. The convolutional layer extracts the spatial features and semantic information of the image through convolution operations, while the pooling layer reduces the resolution of the feature map through downsampling operations, reduces the computational amount, and extracts more representative features. The encoder part usually adopts a stacked structure of multiple convolutional layers and pooling layers, gradually converting the input image into a high-dimensional feature representation.

[0082] The decoder part then gradually restores the image resolution and detailed information through deconvolutional layers and skip connections. The deconvolutional layer gradually restores the resolution of the feature map to the size of the original image through upsampling operations, while the skip connections combine the feature maps of the encoder part with the feature maps of the decoder part to achieve multi-scale feature fusion.

[0083] Skip connections are a key component in the U-Net model. By combining the feature maps of the encoder part with the feature maps of the decoder part, they achieve multi-scale feature fusion. In the U-Net stage, skip connections usually use simple concatenation operations or summation operations to achieve feature fusion, and the specific choice depends on the network structure and task requirements.

[0084] Through the encoder-decoder structure and skip connections of the U-Net model, more accurate and detailed segmented point cloud data can be obtained. Compared with the segmentation mask output by the Mask R-CNN model, the segmentation result of the U-Net model is more refined and complete, and can better retain the detailed structure and edge information of the bolt. This is of great significance for subsequent bolt loosening detection and analysis.

[0085] 105: Perform bolt loosening detection based on the segmented point cloud data to obtain the bolt loosening detection result.

[0086] Further processing and analysis of the segmented point cloud data are the key links for judging whether the bolt is loose. This process not only requires fine feature extraction of the point cloud data, but also needs to use advanced machine learning algorithms to accurately judge the loosening state of the bolt and display the detection result in an intuitive form to the user. In this embodiment, bolt loosening detection includes point cloud feature extraction, loosening judgment, and result visualization.

[0087] Among them, point cloud feature extraction aims to extract key information that can characterize the state of the bolt from the segmented point cloud data. These features usually include geometric features and texture features, which together constitute a comprehensive description of the bolt state.

[0088] Geometric features mainly focus on spatial attributes such as the shape, size, and position of the bolt. In the point cloud data, these features can be extracted by calculating various statistics, geometric measures, and spatial distribution characteristics of the point cloud. For example, geometric quantities such as the centroid, principal axis direction, surface area, and volume of the bolt point cloud can be calculated, as well as features such as the spatial distribution density and curvature of the point cloud. These geometric features can reflect the overall shape and spatial position of the bolt, providing an important basis for subsequent loosening judgment.

[0089] Texture features focus on the detailed information on the bolt surface, such as surface roughness and color changes. In the point cloud data, texture features can be extracted by calculating attributes such as the normal vector, curvature, and color histogram of the point cloud. These features can reflect the microscopic structure and material properties of the bolt surface, and are of great significance for identifying abnormal changes on the bolt surface (such as small displacements or deformations caused by loosening).

[0090] To improve the accuracy and robustness of feature extraction, in this embodiment, geometric features and texture features are fused to improve the description ability and discrimination of features. Feature selection algorithms (such as recursive feature elimination, principal component analysis, etc.) are used to reduce the dimensionality and select features. Feature selection algorithms can identify the features that contribute the most to the classification task and remove redundant and noisy features.

[0091] After extracting the key features of the bolt, the next step is to use machine learning algorithms to judge whether the bolt is loose. This step requires selecting a suitable classifier and training and optimizing it to achieve accurate identification of the bolt loosening state.

[0092] In the loosening judgment task, the gradient boosting decision tree (GBDT) machine learning algorithm is used. By iteratively constructing multiple weak classifiers for ensemble learning, it can handle non-linear relationships and feature interactions.

[0093] To improve the accuracy of looseness judgment, the following optimizations are made to the training process of the classifier in this embodiment: The model integration strategy (such as Bagging, Boosting, etc.) is adopted to fuse the results of multiple classifiers. Model integration can make full use of the advantages of different classifiers to improve the accuracy and robustness of classification. By combining the prediction results of multiple classifiers, a more stable and reliable judgment result can be obtained. The hyperparameters of the classifier are optimized by the grid search method. Hyperparameters are the parameters that need to be adjusted during the classifier training process, such as the penalty coefficient C and kernel function parameter gamma of SVM, the number of trees and maximum depth of the random forest, etc. By adjusting these parameters, the classifier can be more adapted to the characteristics of bolt looseness detection and improve the accuracy and efficiency of classification. Aiming at the problem of imbalance between positive and negative samples in bolt looseness detection, techniques such as resampling and synthetic minority over-sampling technique (SMOTE) are used for processing. The resampling technique can increase the number of minority class samples to balance them with the majority class samples; while the SMOTE technique increases the sample diversity by synthesizing new minority class samples.

[0094] Displaying the detection results in a visual form is the last step in the post-processing step. Through visual display, users can intuitively understand the looseness state of the bolt and the reliability of the detection results.

[0095] During the result visualization process, the segmented point cloud data can be displayed in the form of a three-dimensional image, and loose bolts and normal bolts can be distinguished by different colors or marks. In addition, the detection results can also be displayed in the form of a two-dimensional image, such as overlaying the segmentation mask of the bolt on the original image and marking the position and range of the loose bolt with different colors or lines. These methods can all visually display the detection results, facilitating users to view and analyze.

[0096] During the visual display process, detailed annotations and explanations are made for the detection results. Information such as the position, size, and looseness degree of the loose bolt can be marked on the image, and the corresponding judgment basis and confidence score are given.

[0097] The performance of the detection results is evaluated, and the evaluation results are displayed in a visual form. Metrics such as the accuracy, recall rate, and F1 score of the classifier can be calculated and displayed in the form of bar charts, line charts, etc. Through performance evaluation, users can understand the performance of the classifier and optimize and improve the classifier according to the evaluation results.

[0098] The present invention has high positioning accuracy and strong feature extraction ability, and can effectively identify various bolts and extract bolt contours. It has a high degree of automation and can achieve full-automatic detection without manual intervention. The detection speed is fast. The deep learning model can be embeddedly developed using a module with supercomputer performance and accelerated by deploying TensorRT, enabling real-time detection.

[0099] Through the method of the present invention, both the efficiency and accuracy are higher than manual detection. It can achieve panoramic rapid scanning and intelligent analysis of the underbody components of the vehicle. The overall maintenance time for an 8-car train set can reach 40 - 50 minutes, greatly improving the maintenance efficiency.

[0100] The bolt loosening detection device provided by the present invention will be described below. The bolt loosening detection device described below can be correspondingly referred to the bolt loosening detection method described above.

[0101] Please refer to Figure 2 , Figure 2 which is a schematic structural diagram of the bolt loosening detection device provided by the present invention.

[0102] The present invention also provides a bolt loosening detection device, including: an acquisition module 201 for acquiring three-dimensional point cloud data of the bolt area of the rail vehicle collected by the inspection robot; a target detection module 202 for mapping the three-dimensional point cloud data into a two-dimensional plane image and performing bolt target detection on the two-dimensional plane image to obtain the bolt target detection result in the two-dimensional image space; an extraction module 203 for mapping the bolt target detection result into the three-dimensional point cloud space and extracting the corresponding point cloud region features; a segmentation module 204 for performing point cloud instance segmentation on the point cloud region to obtain the segmented point cloud data; a loosening detection module 205 for performing bolt loosening detection based on the segmented point cloud data to obtain the bolt loosening detection result.

[0103] Figure 3 Illustrates a schematic structural diagram of an electronic device, such as Figure 3As shown in the figure, the electronic device may include: a processor 301, a communications interface 302, a memory 303, and a communication bus 304. Among them, the processor 301, the communications interface 302, and the memory 303 communicate with each other through the communication bus 304. The processor 301 can call the logical instructions in the memory 303 to execute the bolt loosening detection method, which includes: obtaining three-dimensional point cloud data of the bolt area of the rail vehicle collected by the inspection robot; mapping the three-dimensional point cloud data into a two-dimensional plane image, and performing bolt target detection on the two-dimensional plane image to obtain the bolt target detection result in the two-dimensional image space; mapping the bolt target detection result into the three-dimensional point cloud space, and extracting the corresponding point cloud area features; performing point cloud instance segmentation on the point cloud area to obtain the segmented point cloud data; performing bolt loosening detection based on the segmented point cloud data to obtain the bolt loosening detection result.

[0104] In addition, when the logical instructions in the above-mentioned memory 303 are implemented in the form of software functional units and sold or used as an independent product, they can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the related technology, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods of the various embodiments of the present invention. The foregoing storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), magnetic disks, or optical discs that can store program codes.

[0105] An embodiment of the present invention discloses a computer program product. The computer program product includes a computer program stored on a non-transitory computer-readable storage medium. The computer program includes program instructions. When the program instructions are executed by a computer, the computer can execute the bolt loosening detection method provided by each of the above method embodiments. The method includes: obtaining three-dimensional point cloud data of the bolt area of the rail vehicle collected by the inspection robot; mapping the three-dimensional point cloud data into a two-dimensional plane image, and performing bolt target detection on the two-dimensional plane image to obtain the bolt target detection result in the two-dimensional image space; mapping the bolt target detection result into the three-dimensional point cloud space, and extracting the corresponding point cloud area features; performing point cloud instance segmentation on the point cloud area to obtain the segmented point cloud data; performing bolt loosening detection based on the segmented point cloud data to obtain the bolt loosening detection result.

[0106] On the other hand, an embodiment of the present invention further provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it is configured to execute the bolt loosening detection method provided in each of the above embodiments. The method includes: obtaining three-dimensional point cloud data of the bolt area of a rail vehicle collected by an inspection robot; mapping the three-dimensional point cloud data into a two-dimensional planar image, and performing bolt target detection on the two-dimensional planar image to obtain a bolt target detection result in the two-dimensional image space; mapping the bolt target detection result into the three-dimensional point cloud space, and extracting corresponding point cloud region features; performing point cloud instance segmentation on the point cloud region to obtain segmented point cloud data; performing bolt loosening detection based on the segmented point cloud data to obtain a bolt loosening detection result.

[0107] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place, or may be distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art can understand and implement it without creative efforts.

[0108] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the essence of the above technical solution, or the part that contributes to the related technology, can be embodied in the form of a software product. The computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0109] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements for some of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for detecting bolt loosening, characterized in that: include: Acquire three-dimensional point cloud data of the bolt area of ​​the rail vehicle collected by the inspection robot; Mapping the three-dimensional point cloud data into a two-dimensional plane image, and performing bolt target detection on the two-dimensional plane image to obtain a bolt target detection result in the two-dimensional image space; Mapping the bolt target detection result to a three-dimensional point cloud space and extracting corresponding point cloud area features; Performing point cloud instance segmentation on the point cloud area to obtain segmented point cloud data; Bolt looseness detection is performed based on the segmented point cloud data to obtain a bolt looseness detection result.

2. The bolt loosening detection method according to claim 1, characterized in that: After mapping the three-dimensional point cloud data into a two-dimensional plane image, the method further includes: The two-dimensional plane image is enhanced by using a preset image enhancement algorithm, so as to detect bolt targets based on the enhanced image; the preset image enhancement algorithm includes a contrast stretching algorithm, a histogram equalization algorithm, a sharpening algorithm and a filtering algorithm.

3. The bolt loosening detection method according to claim 1, characterized in that: The performing bolt target detection on the two-dimensional plane image to obtain a bolt target detection result in the two-dimensional image space includes: Input the two-dimensional plane image into a pre-trained YOLO model to obtain a plurality of detection bounding boxes output by the YOLO model; the detection bounding boxes are used to mark the position and size of the bolts in the two-dimensional plane image; the confidences corresponding to a plurality of the detection bounding boxes are greater than a preset confidence threshold; Inputting the detection bounding box into a pre-trained Faster RCNN model to obtain a bolt target detection result in the two-dimensional image space output by the Faster RCNN model; The pre-trained YOLO model is trained based on the YOLO target detection data set, and the FasterRCNN model is trained based on the Faster RCNN target detection data set.

4. The bolt loosening detection method according to claim 3, characterized in that: The step of inputting the two-dimensional plane image into a pre-trained YOLO model and obtaining a plurality of detection bounding boxes output by the YOLO model comprises: Inputting the two-dimensional plane image into a first feature extraction layer to perform feature extraction, and obtaining image features of the two-dimensional plane image output by the first feature extraction layer; Inputting the image features into a candidate bounding box generation layer to obtain a candidate bounding box output by the candidate bounding box generation layer; Inputting the candidate bounding box into a first classification and regression layer for classification and regression operations to obtain a bolt target positioning bounding box output by the first classification and regression layer; The bolt target positioning bounding box is input into the first non-maximum suppression layer to remove duplicate bounding boxes, so as to obtain a plurality of detection bounding boxes output by the first non-maximum suppression layer.

5. The bolt loosening detection method according to claim 3, characterized in that: The step of inputting the detection bounding box into a pre-trained Faster RCNN model to obtain a bolt target detection result in the two-dimensional image space output by the Faster RCNN model includes: Inputting the detection bounding box into a second feature extraction layer for feature extraction, and obtaining an extracted feature map output by the second feature extraction layer; Inputting the extracted feature map into the region proposal layer for screening and adjustment to obtain a candidate region output by the region proposal layer; Input the candidate region into the region of interest pooling layer for feature map mapping, and obtain the region feature vector output by the region of interest pooling layer; Inputting the regional feature vector into the second classification and regression layer to perform classification and regression operations, and obtaining the bolt target positioning area output by the second classification and regression layer; The bolt target positioning area is input into the second non-maximum suppression layer to remove repeated areas, and the bolt target detection result in the two-dimensional image space output by the second non-maximum suppression layer is obtained.

6. The method for detecting bolt loosening according to any one of claims 1 to 5, characterized in that: The step of performing point cloud instance segmentation on the point cloud region to obtain segmented point cloud data includes: Inputting the point cloud region features into a pre-trained Mask R-CNN model to obtain a segmentation mask of the bolt output by the Mask R-CNN model; Inputting the segmentation mask of the bolt into a pre-trained U-Net model to obtain the segmented point cloud data output by the U-Net model; The Mask R-CNN model is trained based on the Mask R-CNN point cloud instance segmentation dataset, and the U-Net model is trained based on the U-Net point cloud instance segmentation dataset.

7. The bolt loosening detection method according to claim 6, characterized in that: The step of inputting the point cloud region features into a pre-trained Mask R-CNN model to obtain a segmentation mask of the bolt output by the Mask R-CNN model includes: Inputting the point cloud region features into a pre-trained Faster RCNN model to obtain a bolt target detection result in a three-dimensional point cloud space output by the Faster RCNN model; The bolt target detection result in the three-dimensional point cloud space is input into a segmentation mask generation layer to obtain a segmentation mask of the bolt output by the segmentation mask generation layer.

8. A bolt loosening detection device, characterized in that: include: An acquisition module, used to acquire three-dimensional point cloud data of a bolt area of ​​a rail vehicle collected by an inspection robot; A target detection module, used for mapping the three-dimensional point cloud data into a two-dimensional plane image, and performing bolt target detection on the two-dimensional plane image to obtain a bolt target detection result in the two-dimensional image space; An extraction module, used to map the bolt target detection result to a three-dimensional point cloud space and extract corresponding point cloud area features; A segmentation module is used to perform point cloud instance segmentation on the point cloud area to obtain segmented point cloud data; The looseness detection module is used to perform bolt looseness detection according to the segmented point cloud data to obtain a bolt looseness detection result.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the program, the bolt loosening detection method according to any one of claims 1 to 7 is implemented.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the bolt loosening detection method according to any one of claims 1 to 7 is implemented.

Images