Automatic identification method for loose bolts of vehicle retarder

Abstract

The application discloses a kind of vehicle retarder bolt loosening identification method, belong to railway hump shunting equipment detection technical field, including, the bolt is the improved bolt of the rectangular mark component with thickness of screw cap top integrated formation.The method comprises: obtaining the retarder opening photo that unmanned aerial vehicle is photographed twice before and after in same flight point position;Opening photo is input retarder bolt identification model, and the center point of four bolts is identified and is cut and generates bolt subgraph;Bolt subgraph is input bolt angle entity identification model, and bolt center point, bolt mark and retarder edge reference are identified;According to bolt mark and edge reference, the counterclockwise rotation angle of bolt is calculated;The angle difference of same bolt before and after is compared, and it is determined to loosen if it exceeds threshold value.The application can automatically identify bolt loosening state and calculate rotation angle, replace artificial inspection.

Description

Technical Field

[0001] This invention relates to the field of railway hump yard shunting equipment detection technology, specifically an automatic identification method for loose bolts on vehicle reducers. Background Technology

[0002] The vehicle speed reducer is a key actuator in the automatic control system for hump yard shunting. Its function is to apply braking force to the shunting vehicles via brake rails, control the shunting speed, and ensure the safety of shunting operations. The brake rails are connected to the brake calipers by fixing bolts, and the tightness of these bolts directly affects the stability of the brake rail opening size and the reliability of braking and release actions. Once the bolts loosen, the brake rail opening size will shift, leading to abnormal braking force. This can range from affecting shunting efficiency to potentially causing accidents such as vehicle runaway or rear-end collisions.

[0003] Currently, the on-site inspection of the tightness of the brake rail fixing bolts relies entirely on manual labor. According to operating procedures, maintenance personnel must visually inspect and manually tighten each bolt at least once a day, resulting in a massive workload and high labor intensity. Moreover, this manual inspection method has significant limitations: the inspection results are highly dependent on the experience, sense of responsibility, and work condition of the operators, leading to frequent issues such as missed inspections, inadequate checks, and improper tightening operations. If loose bolts are not detected in time, they will directly affect the braking effect of the reducer, creating safety hazards. With the increasing demands for safety and efficiency in railway transportation, this manual inspection model is no longer sufficient to meet actual needs.

[0004] In recent years, drone inspection and image recognition technologies have been applied in the field of railway infrastructure inspection, such as the inspection of overhead contact line components and track fasteners. However, in the identification of loose bolts in vehicle reducers, two problems are faced: First, the top of ordinary bolts lacks visual features that can be reliably extracted by image recognition algorithms, and the manufacturer's markings and strength grade fonts on the nuts are too small to be clearly imaged when taken from above; second, bolt loosening manifests as slight rotation, and accurately calculating the rotation angle from the image is a technical challenge. Therefore, there is an urgent need for a solution that can automatically identify loose bolts to replace existing manual inspection operations.

[0005] Based on this, the present invention provides a method for identifying loose bolts in a vehicle reducer and an improved bolt. Summary of the Invention

[0006] The purpose of this invention is to provide an automatic identification method for loose bolts in vehicle reducers, so as to solve the problems mentioned in the background art.

[0007] To achieve the above objectives, the present invention provides a method for identifying loose bolts in a vehicle reducer, comprising the following steps: Step 1: Obtain two photos of the vehicle's speed reducer opening taken by the drone at the same waypoint, one before the other. The opening photos contain four bolts. Step 2: Input the open photo into the trained reducer bolt recognition model to identify the positions of the four bolts and their corresponding center points; Step 3: Based on the four identified bolt center points, four bolt sub-images are generated by cropping from the opening photo; Step 4: Input each bolt sub-image into the trained bolt angle entity recognition model to identify the bolt center point, bolt mark, and reducer edge reference. The bolt mark is a marking component set on the top of the bolt. Step 5: Calculate the counterclockwise rotation angle of the bolt relative to the edge of the reducer based on the bolt markings and the reducer edge reference; Step 6: Compare the counterclockwise rotation angles of the same bolt before and after two rotations, calculate the angle difference, and determine that the bolt is loose when the angle difference exceeds the preset threshold.

[0008] Furthermore, the bolt is an integrally formed bolt with a rectangular marking member with thickness on the top of the nut. One end of the rectangular marking member originates from the center point of the nut, and its center line coincides with the line connecting the center point of the nut and one corner of the nut.

[0009] Furthermore, the calculation method for the counterclockwise rotation angle mentioned in step 5 is as follows: The edge of the reducer reference object closest to the bolt center point is translated to the bolt center point to form the first ray; The line segment formed by connecting the center points of the two short sides of the bolt marking component is translated to the center point of the bolt to form a second ray; Calculate the angle that the bolt travels from the first ray to the second ray by rotating counterclockwise; this is the relative counterclockwise angle of the bolt.

[0010] Furthermore, both the reducer bolt recognition model and the bolt angle entity recognition model are trained using a target detection and segmentation network as the base model.

[0011] Furthermore, when training the object detection and segmentation network, model training parameters are set, including training image size and multi-scale training strategy, warm-up training strategy, anchor box calculation strategy, detection box loss weight, intersection-over-union threshold, data augmentation strategy and mask downsampling ratio; a custom callback function is set to track and record the model accuracy index after each round of validation, and dynamically adjust the learning rate and loss weight according to the changes in the accuracy index.

[0012] Furthermore, the method for cutting the bolt sub-drawing in step 3 is as follows: A rectangular area of ​​adaptive size is generated with the bolt center point as the center. The initial size is a preset pixel value. When it exceeds the image boundary, it is gradually reduced by a preset step size until it is completely within the image range.

[0013] Furthermore, the training datasets for the reducer bolt recognition model and the bolt angle entity recognition model are constructed in the following manner: Collect photos of reducer openings under different lighting conditions and divide them into training set, validation set and test set according to a preset ratio; The image was annotated using the annotation tool, with the center point of each of the four bolts, the corresponding bolt markings, and the edge reference of the reducer annotated. Data augmentation is performed on images in the training and validation sets by rotating and brightening transformations, and corresponding labels are generated simultaneously.

[0014] Furthermore, it also includes a bolt loosening trend early warning step: If the historical data of the counterclockwise rotation angle of the same bolt within a preset historical period is analyzed, and a warning message is triggered if there is a trend of continuous increase in the counterclockwise rotation angle.

[0015] Furthermore, the method for determining the drone flight path is as follows: The reference point is set manually by flying, and other waypoints are generated by displacement based on the reference point. The waypoint routes are then imported in batches through the configuration file. Edit the flight path and set the shooting parameters for each waypoint on the drone cloud platform; After the flight path is completed, the drone cloud platform acquires photos of the reducer openings at each waypoint and maps them to the actual opening locations according to the configuration.

[0016] Furthermore, it also includes alarm procedures: When a bolt is detected to be loose, a real-time alarm is triggered and a voice announcement is made.

[0017] The technical solutions provided by the embodiments of this disclosure bring at least the following beneficial effects: In this invention, a rectangular marking component with thickness is integrally formed on the top of the bolt, eliminating the need for on-site manual marking and avoiding human errors such as missed or incorrect marking. The marking component rotates synchronously with the bolt, and its positional change relative to the edge of the reducer reflects the bolt's rotation angle, providing a reliable image feature reference for visual inspection.

[0018] This invention employs a two-stage recognition strategy. First, the overall position of the bolt is located. Then, the marked components and edge references are finely identified in the sub-image. This decomposes the complex task into two relatively simple sub-tasks, reducing the difficulty of model training and improving the overall recognition accuracy. When calculating the counterclockwise rotation angle, the line connecting the center points of the short sides of the marked components and the inner side of the edge references are translated to the center point of the bolt to convert them into ray angles, avoiding the influence of irregular bolt contours on positioning accuracy. Attached Figure Description

[0019] Figure 1 This is a flowchart outlining the framework of the vehicle reducer bolt loosening identification method of the present invention.

[0020] Figure 2 This is a schematic diagram of a vehicle's speed reducer with four bolts at one opening.

[0021] Figure 3 This is the structural design drawing of the improved bolt of this invention.

[0022] Figure 4 This is a tagged rendering of a photo of a vehicle's speed reducer opening.

[0023] Figure 5 This is a comparison table of the evaluation results of the model of this invention.

[0024] Figure 6 This is a schematic diagram showing the calculation results of the bolt's relative counterclockwise angle.

[0025] Figure 7 This is a schematic diagram of the reasoning results of the bolt loosening identification model (loosening has occurred).

[0026] Figure 8 This is a schematic diagram of the reasoning results of the bolt loosening identification model (no loosening occurred). Detailed Implementation

[0027] The technical solution of the present invention will be clearly and completely described below with reference to the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0028] Please see Figure 1 This application provides an automatic identification method for loose bolts in a vehicle reducer, comprising the following steps.

[0029] The first step is to obtain two photos of the vehicle's speed reducer opening taken by the drone at the same waypoint, one before the other. The photos of the opening contain four bolts.

[0030] The "same waypoint location" here refers to the waypoints where the UAV hovers and takes pictures at the same spatial coordinates (including longitude, latitude, and altitude) when flying along the preset route. The consistency of the waypoint locations in two consecutive shots ensures that the captured images of the opening have the same shooting angle, altitude, and field of view. This is the basis for subsequent image comparison and angle calculation.

[0031] The drone flight path is determined as follows: First, the operator manually flies the drone over the decelerator to select several reference points, ensuring the spatial positions of these reference points are accurate. Then, other waypoints are generated based on the displacement of these reference points. Alternatively, batch importing waypoints and flight paths via configuration files is also supported. The configuration files are in CSV or JSON format and contain the longitude, latitude, altitude, and waypoint number for each waypoint.

[0032] After importing the flight path, edit it on the UAV cloud platform, setting parameters such as gimbal tilt angle, shooting action, shooting angle, and zoom ratio for each waypoint. The gimbal tilt angle controls the camera's downward shooting angle; in this embodiment, it's set to -90 degrees, meaning shooting vertically downwards, with the camera directly facing the top of the bolt. The shooting action controls whether the UAV takes a single shot or multiple shots in burst at each waypoint. The shooting angle controls the yaw angle to ensure consistent field of view at each waypoint. The zoom ratio controls the lens magnification; in this embodiment, it's set to 2 to 5x to ensure the bolts are of appropriate size in the image, with each bolt occupying approximately 150 x 150 to 250 x 250 pixels. Save the flight path after setting the parameters.

[0033] During flight, the drone automatically follows the pre-edited route, taking photos at each waypoint. The image quality at each waypoint is monitored on the cloud platform, and minor adjustments are made to waypoints with positional deviations. For example, if the bolt position is offset in a photo of a waypoint, the waypoint coordinates are adjusted in the opposite direction by 0.1 to 0.3 meters until all photos of each waypoint meet the criteria for clarity and usability—that is, the four bolts are clearly imaged, without motion blur, and with appropriate brightness. At this point, the flight path is saved as the final path.

[0034] After the flight path is completed, the photos of the slack reducer openings at each waypoint are downloaded to the local machine via the historical flight path plan photo acquisition interface of the UAV cloud platform. This interface can be deployed in the cloud or locally. After the photos are acquired, they are mapped to the actual opening positions according to the configuration. For example, "J1-1Z" represents the first left opening of the first slack reducer.

[0035] Each opening photograph contains four bolts, which are named nut1, nut2, nut3, and nut4 in a fixed spatial order (from left to right and from top to bottom) in this embodiment. For example, the four bolts corresponding to the reducer opening J1-1Z are named J1-1Z-nut1, J1-1Z-nut2, J1-1Z-nut3, and J1-1Z-nut4.

[0036] The second step is to input the open photo into the trained reducer bolt recognition model to identify the positions of the four bolts and their corresponding center points.

[0037] The reducer bolt recognition model is a deep learning-based object detection model. Its function is to detect the location of each bolt and determine its geometric center point from an open photograph containing four bolts. The model's input is the original open photograph, i.e., an RGB three-channel color image; in this embodiment, the input image resolution is 1920 pixels × 1080 pixels. The output is the coordinates of the detection box and the center point coordinates of the four bolts.

[0038] Each bolt detection box output by the model is represented by a rotated rectangle, in the format (center point x-coordinate, center point y-coordinate, rectangle width, rectangle height, rotation angle). The center point x-coordinate and y-coordinate are the pixel coordinates of the rectangle's center in the image, with width and height in pixels. The rotation angle is the angle of deflection of the rectangle's longer side relative to the horizontal axis of the image, in degrees. The bolt center point coordinates are the x-coordinate and y-coordinate of the detection box's center point. The training method for this model is described in Example 3.

[0039] The third step involves cropping four bolt sub-images from the opening photograph based on the identified center points of the four bolts.

[0040] A bolt sub-image is a local image cropped from the original open photograph, centered on a single bolt. Each sub-image contains the complete bolt along with surrounding marked components and reducer edge references.

[0041] The cropping method is as follows: Using the center point of the bolt obtained in the second step as the cropping center, a rectangular cropping area is generated. The size is determined using an adaptive method. The initial size is set to 350 pixels × 350 pixels. If the rectangle exceeds the boundary of the original photo, it is reduced by 50 pixels each time until it falls completely within the photo area.

[0042] For example, if the center point coordinates of the bolt are (100 pixels, 100 pixels) and the original photo size is 1920 pixels × 1080 pixels, then the initial cropping area is from (-75 pixels, -75 pixels) to (425 pixels, 425 pixels), completely within the photo's area, so a 350 pixel × 350 pixel image is directly used. If the center point coordinates are (50 pixels, 50 pixels), and the initial area is from (-125 pixels, -125 pixels) to (225 pixels, 225 pixels), exceeding the left and top boundaries, then it is successively shrunk to 300 pixels × 300 pixels, 250 pixels × 250 pixels, 200 pixels × 200 pixels, 150 pixels × 150 pixels, and 100 pixels × 100 pixels. When it reaches 100 pixels × 100 pixels, the area becomes from (0 pixels, 0 pixels) to (100 pixels, 100 pixels), completely within the photo, so a 100 pixel × 100 pixel image is used.

[0043] Once the size is determined, the cropping function of an image processing library is called, such as the crop function of the Python PIL library or the array slicing operation of OpenCV, to extract the region from the original photo and store it as a sub-image of the bolt.

[0044] By adopting the above adaptive cropping method, on the one hand, it can ensure that each sub-image contains the complete bolt and surrounding reference features, avoiding the loss of key information when the bolt is close to the edge of the image; on the other hand, it provides a standardized input for the subsequent angle recognition model, avoiding the impact of inconsistent sizes on recognition accuracy.

[0045] The fourth step is to input each bolt sub-image into the trained bolt angle entity recognition model to identify the bolt center point, bolt mark, and reducer edge reference. The bolt mark is a marking component set on the top of the bolt.

[0046] The bolt angle entity recognition model is also a deep learning-based object detection and segmentation model. Its function is to detect three target objects from a single bolt sub-image: the bolt itself (outputting the center point and outline), the marker component on top of the bolt (outputting the outline and direction), and the reducer edge reference (outputting the edge outline closest to the bolt). The model's input is a single bolt sub-image with the adaptive size determined in step three; the output is a rotated rectangle of the three target objects, in the same format as in step two. The training method for this model is described in Example 3.

[0047] Fifth step: Calculate the counterclockwise rotation angle of the bolt relative to the edge of the reducer based on the bolt markings and the reducer edge reference.

[0048] See calculation method Figure 6 The details are as follows: First, determine the center point O of the bolt rectangle. The bolt rectangle is the bolt detection box output from the model in step four. It is a rotated rectangle, and its geometric center is the bolt center point O, with coordinates (xO, yO).

[0049] Then, the inner edge (the side closest to the bolt center point O) of the reducer edge reference rectangle is translated to point O, forming ray L1. Specifically, the reducer edge reference rectangle has four vertices and four sides. The distance from each side to point O is calculated, and the closest side is taken as the inner edge. The two endpoints of the inner edge are denoted as P1 and P2. Starting from O, ray L1 is drawn along the direction from P1 to P2. The direction of L1 is the direction of the reducer edge at its current position.

[0050] Next, determine the center points M1 and M2 of the two short sides of the bolt mark rectangle. The bolt mark rectangle is the bolt mark segmentation result output from step four, and is also a rotated rectangle with two long sides and two short sides. The direction of the line connecting the center points M1 and M2 of the two short sides is the pointing direction of the bolt mark component. Connect M1 and M2 to form a line segment, translate it to point O, so that one endpoint of the line segment coincides with O, forming ray L2. During translation, follow the nearest distance principle, that is, ensure that point O falls on the extension line of M1M2 and is the closest point.

[0051] Finally, calculate the angle θ traversed by rotating counterclockwise from ray L1 to ray L2. Establish a coordinate system with O as the origin, calculate the angle α1 between L1 and the positive direction of the horizontal axis, and the angle α2 between L2 and the positive direction of the horizontal axis. Then θ is equal to (α2 minus α1) modulo 360 degrees, with the result between 0 and 360 degrees. "Counterclockwise" refers to the smaller angle of rotation from L1 to L2 in the image plane, opposite to the direction of rotation of a clock hand. This angle θ is the counterclockwise rotation angle of the bolt relative to the edge of the reducer.

[0052] The above angle calculation method constructs two rays with clear geometric meaning, transforming the rotational relationship of the bolt mark relative to the fixed reference object into the angle between the rays. This avoids the positioning error caused by the irregular shape of the bolt itself, significantly improving the accuracy of angle calculation. The maximum error can be controlled within 3 degrees according to the test.

[0053] The sixth step is to compare the counterclockwise rotation angles of the same bolt before and after two rotations, calculate the angle difference, and determine that the bolt is loose when the angle difference exceeds the preset threshold.

[0054] The angle measured this time is recorded as θ1, and the angle measured last time is recorded as θ0. The angle difference Δθ is equal to (θ1 minus θ0) modulo 360 degrees. In specific calculation, first find the difference d, which is equal to θ1 minus θ0, and then calculate the remainder when d is divided by 360 degrees. For example, when θ0 is 355 degrees and θ1 is 5 degrees, d is -350 degrees, and modulo 360 degrees gives 10 degrees, indicating that the bolt actually rotated 10 degrees counterclockwise. As another example, when θ0 is 10 degrees and θ1 is 355 degrees, d is 345 degrees, and modulo 360 degrees gives 345 degrees, indicating that the bolt actually rotated 15 degrees clockwise.

[0055] Regarding the determination of the preset threshold: In this embodiment, the preset threshold is set to 3 degrees. This threshold is determined based on the following considerations: Statistically, the maximum measurement error of the bolt angle entity recognition model on the test set is 2.1 degrees under sunny conditions and 2.9 degrees under cloudy conditions (see...). Figure 5 (As in Example 3), the worst-case scenario, i.e., the maximum error of 2.9 degrees, is taken as the upper limit of error. Therefore, the threshold is set to 3 degrees, which is slightly greater than this upper limit, to ensure that when the angle difference exceeds this threshold, the influence of model measurement error can be eliminated, and it can be determined that the bolt has indeed rotated. If the threshold is set to 2 degrees, model error may lead to false alarms; if it is set to 5 degrees, minor loosening may be missed. 3 degrees is a reasonable value that balances detection sensitivity and accuracy.

[0056] When a loose bolt is detected, the system triggers a real-time alarm and broadcasts a voice announcement. An alarm prompt box pops up on the interface, and the alarm record contains information including "gearbox number - opening number - bolt number - time of loosening - angle difference". At the same time, the speaker broadcasts "A bolt at a certain opening of a certain gearbox has become loose. Please handle it immediately" to remind the operator to deal with it in a timely manner.

[0057] In addition, this embodiment also includes a trend warning function: analyzing the historical angle data of the same bolt over the past 15 days. The specific judgment method is as follows: take the angle sequence [θ1, θ2, ..., θn] measured each time within the past 15 days, where n is the number of measurements. Assuming weekly inspections, there are approximately 2 to 3 measurements within 15 days. Calculate the adjacent difference Δθi, which is equal to θi+1 minus θi. If all adjacent differences are greater than 0 degrees, and the linear regression slope of the entire sequence is greater than 0.5 degrees per measurement (fitted using the least squares method), then a continuous increasing trend is determined. At this time, a warning is triggered and a voice broadcast is given, reminding the operator that the bolt may be slowly loosening and suggesting that it be tightened in advance.

[0058] The aforementioned trend warning mechanism can issue an early warning before the bolts reach the loosening threshold, transforming post-event alarms into pre-event warnings, giving maintenance personnel a window of time to take action in advance, and effectively avoiding safety accidents caused by sudden loosening.

[0059] 15 days is the preferred value determined based on the inspection frequency and bolt loosening characteristics. In actual applications, it can be adjusted to 7 days or 30 days depending on the specific circumstances.

[0060] It should be noted that the order of steps one through six in this embodiment is the preferred order. Alternatively, the two models from steps two and four can be merged into a single end-to-end model, directly outputting the angle of each bolt from the original opening photograph, without needing to first cut sub-images. However, experimental verification has shown that the two-stage method significantly reduces training difficulty and improves accuracy, therefore it is the preferred solution.

[0061] The two-stage recognition scheme described above first locates the overall position of the bolt in the original open image, and then refines the identification of marked components and edge references in the sub-image. This is equivalent to decomposing a complex task into two relatively simple sub-tasks. Each sub-model only needs to focus on a few target categories, reducing the training difficulty. Furthermore, the accuracy of each stage can be optimized separately, resulting in a significant improvement in overall recognition accuracy.

[0062] Example 2 This embodiment provides an improved bolt structure for the above method.

[0063] like Figure 2 As shown, there are four fixing bolts at each opening of the reducer. The existing bolt nuts only have the manufacturer's mark and strength grade on the top. When the drone takes an overhead shot, the font is too small and the contrast is low, making it impossible to capture a clear image and difficult to use as an image recognition feature.

[0064] Therefore, this embodiment provides an improved bolt. For example... Figure 3 As shown, the improved bolt is an integrally molded bolt with a thick rectangular marking component on the top of the nut. The rectangular marking component is integrally molded with the bolt nut during casting, eliminating the need for additional manual marking (such as affixing labels, painting, etc.) after bolt installation.

[0065] The positional relationship of the rectangular marking component is as follows: one end starts from the center point of the nut, and the center line of the rectangular marking component, the line connecting the center point of the nut and one corner of the nut coincide. That is to say, the rectangular marking component is arranged along the direction from the center of the nut towards a certain corner, and its center line coincides with the direction of extension.

[0066] Regarding dimensions: the length is 1.2 to 1.5 times the radius of the nut's circumcircle, the width is 0.2 to 0.3 times the length, and the thickness is 2 to 5 millimeters. This size range was determined based on the following considerations: when the length is less than 1.2 times the radius, the marking component is not obvious, making it difficult for image recognition algorithms to reliably extract features; when the length is greater than 1.5 times the radius, it extends beyond the nut's edge, affecting installation and disassembly operations; when the width is less than 0.2 times the length, the marking is too thin and elongated, easily confused with the background when photographed from above; when the width is greater than 0.3 times the length, the marking is too wide and flat, weakening its directional indicative power; when the thickness is less than 2 millimeters, the shadow is not obvious when photographed from above, making feature extraction difficult; when the thickness is greater than 5 millimeters, material costs increase and manufacturing processes become more complex. Experiments have verified that the above range achieves a balance between recognition effectiveness, installation convenience, and manufacturing cost.

[0067] The advantages of the improved bolt are as follows: the rectangular marking component and the bolt body are integrally formed, eliminating the need for manual marking on-site and avoiding human errors such as omissions or mismarking. Furthermore, the rectangular marking component's position relative to the bolt is fixed and unique; once the bolt rotates, the marking component rotates synchronously, and its orientation change relative to the edge of the reducer represents the bolt's rotation angle, ensuring accurate and reliable measurement results. Simultaneously, the marking component has a certain thickness, producing significant shadow and highlight contrast when photographed from above, greatly enhancing the robustness of image feature extraction.

[0068] The shape of the rectangular marker component is not limited to rectangles; other shapes that can indicate direction, such as triangles, circular pointers, and trapezoids, can also be used, as long as they can be clearly imaged when taken from above and can rotate synchronously with the bolt. However, rectangles have the clearest edges, the most defined corners, and are the easiest for algorithms to extract features from, making them the preferred shape.

[0069] Example 3 This embodiment provides a training method for the aforementioned reducer bolt recognition model and bolt angle entity recognition model.

[0070] First, training data was collected. A large number of photos of the reducer openings were taken by drones along the flight path within the designated skylight area. The photos covered different lighting conditions, including sunny days with ample light and significant shadows, and cloudy days with soft light and low contrast, to ensure the model's applicability in various environments. They also covered different locations (different reducers, different openings) and different reducer states (braking and deceleration). In this embodiment, each reducer has eight openings, and each opening has four bolts, resulting in a total of over 300 photos collected.

[0071] The dataset was divided into training, validation, and test sets in an 8:1:1 ratio. The training set (240 images) was used for model parameter learning, the validation set (30 images) for hyperparameter tuning and early stopping detection, and the test set (30 images) for final evaluation. The partitioning ensured that each subset contained images with different lighting conditions, locations, and states to guarantee the model's generalization ability.

[0072] The photos in the test set require the accurate angles of the four bolt markers to be measured manually beforehand for calculating model errors. The measurement method is as follows: using the edge of the reducer as a baseline, measure the counterclockwise angle of the marker relative to the baseline using a protractor or image measurement software, with an accuracy of 0.5 degrees. This manually measured value is used as the true value and compared with the model's calculated value.

[0073] Then, data annotation was performed. The LabelMe annotation tool was used to annotate the center points of the four bolts, labeling them in a fixed order from nut1 to nut4 to ensure consistency of bolt labels at the same location in different images. Each bolt also required an additional labeling of a marker patch (groove1) and an edge reference patch (groove2), with a bolt prefix, for example, "nut1-groove1" indicating the marker patch for the first bolt. See the annotation results below. Figure 4 .

[0074] After annotation, four corresponding bolt sub-images are generated based on each opening annotation image. Each sub-image contains three labels: anchor (center point), groove1 (marker), and groove2 (edge ​​reference). See [link / description]. Figure 4 The adaptive pruning method from step three of Example 1 is used during generation. This results in the formation of preliminary training, validation, and test sets.

[0075] To expand the data volume and improve generalization ability, data augmentation is performed on the training and validation sets. Rotation transformation: Randomly rotate the image by 5 to 10 degrees around its center. For example, rotate by 5 degrees, 8 degrees, and 10 degrees respectively to generate a new image. The label coordinates rotate synchronously; each point coordinate is multiplied by the rotation matrix. Brightness transformation: The overall brightness is randomly adjusted by -10% to +10%. For example, darken by 10%, brighten by 5%, and brighten by 10%. The pixel values ​​of the three RGB channels are uniformly multiplied by a random coefficient between 0.9 and 1.1. The training set receives more augmentation (generating 5 to 8 variants per image), while the validation set receives less augmentation (generating 2 to 3 variants per image).

[0076] By employing the aforementioned data augmentation methods, the training samples are expanded several times without increasing on-site shooting costs, covering more rotation angles and brightness variations. This makes the model more adaptable to different shooting conditions and improves its generalization performance.

[0077] Both models employ object detection and segmentation networks; this embodiment uses YOLOv8X-seg. This network is a single-stage object detection and instance segmentation model that can simultaneously output bounding boxes and segmentation masks, making it suitable for accurately locating bolt center points and marking component boundaries. The Ultralytics YOLOv8 official code repository (version 8.0.0 or higher) and the PyTorch framework (version 2.0.0 or higher) are used. Complete training and inference code and configuration files are submitted as attachments.

[0078] The training parameters are set as follows.

[0079] The initial image size is 1280×960 pixels. Multi-scale training is enabled, with each round randomly selecting a size between 640×480 pixels and 1920×1440 pixels to enhance scale adaptability. A warm-up training session is then initiated, consisting of five rounds. The learning rate for the first five rounds is set to one-tenth of the base learning rate of 0.01 (i.e., 0.001) to ensure a smooth start for the model and avoid oscillations. After the warm-up, the learning rate is restored to 0.01. Anchor boxes are automatically calculated using K-means clustering to generate nine anchor boxes based on the size distribution of the objects in the dataset, adapting to a 4:3 image ratio.

[0080] The bounding box loss weight is set to 1.2 (default is 1.0) to improve the localization accuracy of small targets. The intersection-union (IU) threshold is set to 0.7 (default is 0.5), and only samples with an IU greater than 0.7 are considered positive, thus raising the detection standard. The mosaic enhancement intensity is set to 0.3 (default is 1.0), appropriately reduced to protect the features of small targets from being destroyed by excessive stitching. The copy-paste enhancement ratio is set to 0.3, which allows small targets to be cropped from one image and pasted into another, increasing sample diversity. The scaling range is set to 0.3, i.e., random scaling from 0.7 to 1.3 times. The perspective transformation intensity is set to 0.001, with the transformation matrix translation, rotation, and scaling parameters randomly selected within a range of ±0.001. The mask downsampling ratio is set to 2 (default is 4), with the output resolution being half that of the input, preserving more contour details.

[0081] The training parameters described above are specifically designed for small target detection of bolts and boundary segmentation of marked components: increasing the bounding box loss weight and raising the intersection-union ratio (IU) threshold improves the detection rate of small targets, while reducing the mosaic intensity and decreasing the mask downsampling ratio protects the integrity of the small target contours. Experiments have verified that using these parameter configurations significantly improves the segmentation accuracy of bolt-marked components compared to using the default parameter configurations.

[0082] During training, a custom callback function is set: after each validation round, the detection mAP50, detection mAP50-95, and segmentation mAP50 are tracked, and the best mAP50-95 and its corresponding round are recorded; the current time and the time elapsed are also printed; when mAP50 exceeds 0.85 for 5 consecutive rounds, the learning rate is decayed (multiplied by 0.1), and the detection box loss weight is increased by 0.1, with a maximum detection box loss weight of 2.0; when the validation set loss increases for 3 consecutive rounds, the learning rate is restored to its pre-decay value, and the detection box loss weight is restored to its initial value of 1.2. The above parameters are used to adjust and balance the detection and segmentation accuracy.

[0083] Early stopping is triggered when training reaches 300 epochs or when the validation set loss does not decrease for 20 consecutive epochs, and the model parameters with the highest mAP50-95 on the validation set are saved.

[0084] After training, precision, recall, and mAP are calculated on the test set. Precision is equal to the number of correctly detected targets divided by the total number of detected targets; recall is equal to the number of correctly detected targets divided by the total number of true targets; mAP is the average AP across all categories. In this embodiment, all three metrics reach over 90%.

[0085] Next, the angle calculated by the model is compared with the actual value measured manually, and the maximum error and average error are calculated. The maximum error is the maximum absolute value of the difference between the model value and the actual value across all samples, and the average error is the average of the absolute values ​​of the differences across all samples. In this example, the maximum error is 2.9 degrees, and the average error is 1.5 degrees. The evaluation results are shown below. Figure 5 .

[0086] The above evaluation results show that the bolt loosening identification method of the present invention can maintain high identification accuracy under different lighting conditions, such as sunny and cloudy days. The maximum error of angle calculation does not exceed 3 degrees, and the average error is only 1.3 to 1.5 degrees, which is better than the average error of manual measurement (2.5 to 3.1 degrees). It can meet the accuracy requirements of replacing manual inspection.

[0087] Example 4 This embodiment provides an implementation method for the model inference interface.

[0088] To automate bolt loosening detection, this embodiment provides an HTTP interface for model inference. The interface uses the POST method, with the request path " / api / v1 / bolt-recognition". The request header must include an authentication token (Authorization: Bearer {token}) to ensure security. The request body uses multipart / form-data format and contains two fields: the previously captured image of the loose bolt and the currently captured image of the loose bolt, both in JPEG or PNG format, with a maximum size of 10MB per image.

[0089] The response body is in JSON format and includes a status code (200 for success, others for errors), status information ("success" for success, and a detailed description for errors), and an array of recognition results. Each recognition result includes the bolt number (nut1 to nut4), the counterclockwise rotation angle measured last time (in degrees, rounded to one decimal place), the counterclockwise rotation angle measured this time (in degrees, rounded to one decimal place), the angle difference (in degrees, rounded to one decimal place), and a judgment result on whether it is loose (true for loose, false for normal).

[0090] The interface call process is as follows: The client sends an HTTP POST request, carrying two photos of the openings, one before and one after. After receiving the data, the server loads the two trained models and executes steps two through six of Example 1 in sequence. It calculates the angle difference for each of the four bolts and determines whether they are loose. The server then encapsulates the results into JSON and returns them to the client.

[0091] The reasoning results can be presented as images, overlaid on the original open photograph: a green position marker box for each bolt, a blue bolt outline, a red reference line for the reducer edge, yellow arcs and black text indicating the counter-clockwise rotation angle, and red "loose" or green "normal" text labels, making the recognition results clear and intuitive. An example of the reasoning results is shown below. Figure 7 (Loosening) and Figure 8 (No loosening occurred) as shown.

[0092] The reasoning result image visually presents the angle calculation process and judgment conclusion, making it easy for maintenance personnel to quickly review and also providing visual traceability evidence in case of disputes.

[0093] Example 5 This embodiment provides a data display method.

[0094] During the weekly maintenance window (a period when the railway line is not in operation and maintenance work is available, usually once a week for 2 to 4 hours each time), the tightness of all reducer bolts is measured, the relative counterclockwise angle of each bolt is recorded, and the data is stored according to region (different reducer numbers, different opening positions).

[0095] Data storage uses a relational database (such as MySQL or PostgreSQL). The data table contains: measurement ID (auto-incrementing primary key), reducer number (e.g., "J1"), opening number (e.g., "1Z"), bolt number (e.g., "nut1"), measurement time (accurate to the second), counterclockwise rotation angle (in degrees, rounded to one decimal place), original photo file name, inference result photo file name, whether it is loose, and whether it has been processed.

[0096] In the system reducer-opening interface, users can view the latest measurement data, which is displayed in a table showing the counterclockwise rotation angle and looseness determination results of the four bolts for each opening. Normal bolts are displayed with a green background, while loose bolts are displayed with a red background and flashing as a reminder. Users can also view historical measurement data, which is displayed in a line graph showing the rotation angle change curve of each bolt at different time points. The horizontal axis represents the measurement date, and the vertical axis represents the rotation angle (in degrees). The 3-degree threshold line is marked with a dashed line in the graph. Users can also view the corresponding opening photos. Clicking "View Image" will show the original opening photo and the inference result photo for each measurement, supporting zooming and comparison.

[0097] The above data display method links the angle values ​​of each measurement, historical change trends, and original image data, making it easier for maintenance personnel to fully grasp the bolt condition change patterns. Once an anomaly occurs, historical situations can be quickly traced to assist in decision-making and handling. This realizes the transformation of vehicle reducer brake rail fixing bolt maintenance from manual periodic inspection to data-driven intelligent condition maintenance.

[0098] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. An automatic identification method for loose bolts in a vehicle reducer, characterized in that, include: The drone takes two photos of the vehicle's speed reducer opening, taken at the same waypoint, and the opening photos contain four bolts. The open photograph is input into the trained reducer bolt recognition model to identify the positions of the four bolts and their corresponding center points; Based on the four identified bolt center points, four bolt sub-images are generated by cropping from the opening photo; Each bolt sub-image is input into the trained bolt angle entity recognition model to identify the bolt center point, bolt mark, and reducer edge reference. The bolt mark is a marking component set on the top of the bolt. Based on the bolt markings and the reducer edge reference, calculate the counterclockwise rotation angle of the bolt relative to the reducer edge; Compare the counterclockwise rotation angles of the same bolt before and after two rotations, calculate the angle difference, and determine that the bolt is loose when the angle difference exceeds a preset threshold.

2. The automatic identification method for loose bolts in a vehicle reducer according to claim 1, characterized in that, The bolt is an integrally formed bolt with a rectangular marking member with thickness on the top of the nut. One end of the rectangular marking member starts from the center point of the nut, and its center line coincides with the line connecting the center point of the nut and one corner of the nut.

3. The automatic identification method for loose bolts in a vehicle reducer according to claim 1, characterized in that, The counterclockwise rotation angle is calculated as follows: The edge of the reducer reference object closest to the bolt center point is translated to the bolt center point to form the first ray; The line segment formed by connecting the center points of the two short sides of the bolt marking component is translated to the center point of the bolt to form a second ray; Calculate the angle that the bolt travels from the first ray to the second ray by rotating counterclockwise; this is the relative counterclockwise angle of the bolt.

4. The automatic identification method for loose bolts in a vehicle reducer according to claim 1, characterized in that, Both the reducer bolt recognition model and the bolt angle entity recognition model are trained using a target detection and segmentation network as the base model.

5. The automatic identification method for loose bolts in a vehicle reducer according to claim 4, characterized in that, When training the target detection and segmentation network, the model training parameters are set, including training image size and multi-scale training strategy, warm-up training strategy, anchor box calculation strategy, detection box loss weight, intersection-over-union threshold, data augmentation strategy and mask downsampling ratio; Set up a custom callback function to track and record the model accuracy metrics after each round of validation, and dynamically adjust the learning rate and loss weights based on changes in the accuracy metrics.

6. The automatic identification method for loose bolts in a vehicle reducer according to claim 1, characterized in that, The bolt sub-drawing is cut as follows: A rectangular area of ​​adaptive size is generated with the bolt center point as the center. The initial size is a preset pixel value. When it exceeds the image boundary, it is gradually reduced by a preset step size until it is completely within the image range.

7. The automatic identification method for loose bolts in a vehicle reducer according to claim 1, characterized in that, The training datasets for the reducer bolt recognition model and the bolt angle entity recognition model are constructed in the following way: Collect photos of reducer openings under different lighting conditions and divide them into training set, validation set and test set according to a preset ratio; The image was annotated using the annotation tool, with the center point of each of the four bolts, the corresponding bolt markings, and the edge reference of the reducer annotated. Data augmentation is performed on images in the training and validation sets by rotating and brightening transformations, and corresponding labels are generated simultaneously.

8. The automatic identification method for loose bolts in a vehicle reducer according to claim 1, characterized in that, It also includes a bolt loosening trend early warning step: If the historical data of the counterclockwise rotation angle of the same bolt within a preset historical period is analyzed, and a warning message is triggered if there is a trend of continuous increase in the counterclockwise rotation angle.

9. The automatic identification method for loose bolts in a vehicle reducer according to claim 1, characterized in that, The method for determining the drone flight path is as follows: The reference point is set manually by flying, and other waypoints are generated by displacement based on the reference point. The waypoint routes are then imported in batches through the configuration file. Edit the flight path and set the shooting parameters for each waypoint on the drone cloud platform; After the flight path is completed, the drone cloud platform acquires photos of the reducer openings at each waypoint and maps them to the actual opening locations according to the configuration.

10. The automatic identification method for loose bolts in a vehicle reducer according to claim 1, characterized in that, It also includes alarm procedures: When a bolt is detected to be loose, a real-time alarm is triggered and a voice announcement is made.

Images