Railway train split pin defect detection method and system

Abstract

The application discloses a kind of rail train split pin defect detection method and system, including quality evaluation step, image quality evaluation is carried out to the image to be detected to determine whether detection continues;Flexible positioning step, the positioning parameter of target candidate area is output;Structure analysis step, establish local coordinate system and extract the V vertex of split pin, left arm skeleton, right arm skeleton and endpoint;Double score calculation step, the existence score and visibility score of left and right arm are calculated;Fusion determination step, calculate comprehensive determination score and output split pin state;Shielding compensation step, when quality evaluation triggers to be reviewed due to shielding ratio, by judging whether left and right arm exist visual overlap or detection shadow area, revise to be reviewed state and continue detection;The present application can effectively distinguish real defects and pseudo defects caused by shielding, stains and reflection by V-shaped double-arm topology modeling, arm-level existence and visibility joint scoring and shielding compensation, reduce false positive rate.

Description

Technical Field

[0001] This invention relates to the field of cotter pin detection technology, and more specifically to a method and system for detecting defects in cotter pins of railway trains. Background Technology

[0002] Cotter pins are widely used in railway train bogies, braking mechanisms, connecting mechanisms, fork mechanisms, pin limit mechanisms, and various anti-loosening and anti-detachment assembly structures. Their function is to prevent axial loosening of related components, connection instability, or accidental unlocking during operation. For urban rail vehicles, although cotter pins are relatively small standard fasteners, their failure directly affects the reliability of adjacent connecting parts. Therefore, the condition of cotter pins is always a key inspection item during train maintenance and intelligent inspection.

[0003] In common assembly conditions, cotter pins typically exhibit a V-shaped double-arm unfolded structure. Their structural features are reflected in a single apex, two branch arms, two end points, and their relative positional relationships with adjacent components such as holes, nuts, washers, or pins. In actual maintenance, when one arm of the cotter pin is broken, missing, severely bent, insufficiently unfolded, or improperly assembled, its limiting function may be weakened or even completely failed. Therefore, cotter pin defect detection not only needs to answer "does the target exist?", but also needs to determine whether its double-arm structure is complete, whether its geometric relationship is reasonable, and whether there is a reliable basis for judgment under the current image conditions.

[0004] In existing technologies, visual inspection solutions for small fasteners on rail vehicles typically include several approaches such as manual visual inspection, template matching, edge detection, morphological analysis, general object detection networks, instance segmentation networks, or classification networks. While manual methods have certain advantages based on experience, they suffer from low efficiency, strong subjectivity, and susceptibility to fatigue and missed detections. Using only general object detection or classification networks, although they can identify the target area to some extent, they are unable to further distinguish between "a real broken or missing arm" and "an arm that is not visible due to occlusion, incomplete field of view, stains, excessive reflection, or local blurring." Using only traditional edge, area, or length threshold rules is prone to misjudgment when the background structure is complex, the cotter pin size is small, or the local contrast is insufficient, and it is also sensitive to changes in work position, lighting, and imaging posture.

[0005] Especially in actual image acquisition scenarios of train inspection robots, cotter pin targets often face the following complex conditions: First, the target size is small, and the width of the two arms in the image may only be a few pixels. Slight defocusing or motion blur can lead to degradation of end information. Second, cotter pins are often located near metal components, and there are similar textures such as rods, edges, holes, nuts, and oily boundaries in the background, which can easily form false V-shapes or false notches. Third, in special positive samples, there are often field-of-view truncation and occlusion phenomena. At this time, the cotter pin is still in an intact state, but the image only shows part of the structure, which can easily be mistaken for a break or missing part. Fourth, there are oil stains, rust, dust, shadows, and reflections on the surface of the cotter pin or adjacent areas. These factors can cause edge breakage, brightness saturation, and local texture submersion, thus significantly increasing the false alarm rate. Summary of the Invention

[0006] To address the shortcomings of existing technologies, the present invention aims to provide a method and system for detecting cotter pin defects in rail trains. The purpose is to solve the problem that existing technologies cannot reliably distinguish between genuine single-arm fractures and missing single-arm defects and false defects caused by obstruction, incomplete data acquisition, dirt, or reflection when detecting cotter pin defects. At the same time, it also solves the problems of existing solutions lacking local coordinate normalization, structural-level modeling, visibility quantification, and verification.

[0007] Therefore, this invention provides a method and system for detecting cotter pin defects in rail trains. It uses quality gating to determine whether an image meets the criteria for judgment, eliminates viewpoint and attitude differences through flexible matching positioning and local coordinate normalization, explicitly models the cotter pin as a structural component with a V-vertex and left and right arms through V-shaped double-arm topology modeling, distinguishes between real and false defects through joint modeling of arm-level existence scoring and visibility scoring, and forms a complete detection path through three-state output and verification.

[0008] To achieve the above objectives, the present invention provides the following technical solution: A method for detecting defects in cotter pins of railway trains includes the following steps: The quality assessment step involves evaluating the image quality of the image to be inspected and determining whether to continue the inspection based on the quality assessment results. The purpose is to determine whether the image meets the conditions for judgment before defect judgment, so as to avoid low-quality images directly entering the subsequent judgment and causing false alarms. The flexible positioning step involves performing flexible matching positioning on the image to be detected when it is determined that the detection will continue. The search is conducted to find the area where the cotter pin is located and the positioning parameters of the target candidate area are output. The purpose is to unify the cotter pin images of different models, different work positions and different shooting angles into a comparable coordinate space through flexible matching positioning and local coordinate normalization, so that the subsequent structural measurements and geometric constraints have consistent mathematical meaning. The structural analysis step involves establishing a local coordinate system based on the positioning parameters, mapping the target candidate region to a unified local standard pose, and extracting the structured representation of the cotter pin from the local standard pose image. The structured representation includes the V-vertex position, the left arm skeleton, the right arm skeleton, the left arm endpoint, and the right arm endpoint. The purpose is to explicitly model the cotter pin as a topological structural component with a V-vertex, left and right arms, endpoints, skeleton, and directional constraints, so that the detection algorithm can not only locate the target region, but also calculate the existence evidence of the left and right arms respectively around the double-arm topological structure. The dual-score calculation step calculates the existence score and visibility score of the left and right arms respectively based on the structured representation. The existence score is used to reflect whether the corresponding arm has a stable structural response, and the visibility score is used to reflect whether the corresponding arm has sufficient observation conditions. The purpose is to enable the system to distinguish between "real defects" and "false defects" through joint modeling of arm-level existence score and visibility score, and significantly reduce the false alarm rate. The fusion judgment step calculates a comprehensive judgment score based on the existence score, visibility score, and geometric consistency score, and outputs a normal state, a suspected defect state, or a state pending review based on the comprehensive judgment score.

[0009] Furthermore, the quality assessment step includes a quality index determination sub-step, which calculates at least one quality index among global sharpness, local sharpness, reflective coverage ratio, stain coverage ratio, occlusion ratio, and boundary truncation risk of the image to be detected, and determines whether each quality index meets the corresponding preset judgment condition. When any quality index is not met, a pending review status is output and detection is stopped. Thus, through the comprehensive evaluation of multi-dimensional quality indicators, low-quality samples that are not suitable for direct judgment are filtered out at the detection front end, avoiding false alarms caused by low-quality images directly entering the subsequent judgment process.

[0010] Furthermore, it also includes an occlusion compensation step. The occlusion compensation step includes calculating the pixel distance between the left and right arm skeletons on the projection plane when the quality index corresponding to the evaluation output review state is a single occlusion ratio. If the pixel distance is less than a preset overlap distance threshold, it is determined that the left and right arms overlap under the imaging viewpoint. Flexible matching and positioning are performed on the image to be detected based on at least one of the characteristics of color consistency, boundary continuity, or reflectivity consistency of the overlapping area. Thus, when the review state is triggered only by the occlusion ratio, by judging whether there is viewpoint overlap between the two arms and performing flexible matching and positioning, the false defect samples caused by viewpoint overlap are corrected into identifiable samples, reducing the unnecessary review rate.

[0011] Furthermore, the occlusion compensation step also includes determining that the left and right arms do not overlap when the pixel distance between the left and right arm skeletons on the projection plane is greater than or equal to a preset overlap distance threshold, detecting the shadow regions corresponding to the left and right arms respectively, extracting the center line, length, direction, and starting point position of the shadow, calculating the direction consistency score between the shadow direction and the expected projection direction of the corresponding arm when it is assumed to exist, and the distance score between the shadow starting point and the V vertex, calculating the shadow evidence score of the arm based on the weighted fusion result of the direction consistency score and the distance score, if the shadow evidence score of the target arm is higher than the preset shadow threshold, then the arm is determined to exist. The image to be detected is flexibly matched and located based on the center line and direction of the shadow. Thus, when the two arms do not overlap in viewpoint but the occlusion ratio is abnormal, the arm can be proven to exist by detecting the shadow as indirect evidence of the arm's existence. The direction of the shadow is consistent with the expected projection direction of the corresponding arm, and the starting point of the shadow matches the position of the V vertex. This further corrects the false defect samples caused by the shadow into identifiable samples.

[0012] Furthermore, the flexible positioning step includes a template matching sub-step and a local coordinate construction sub-step. The template matching sub-step uses at least one of template features, edge features, local feature point descriptors, or shape context to perform sliding window search or feature point matching on the image to be detected, and outputs the center position, main direction angle, scale factor, reference anchor point, and matching confidence of the target candidate region. The flexible matching positioning allows the target to change within a certain angle, scale, and local deformation range, which is different from the fixed template comparison method. The local coordinate construction sub-step, based on the reference anchor point and its adjacent reference structures, maps the target candidate region to a unified local standard pose through at least one of rotation, translation, scale normalization, or perspective correction, establishing a local coordinate system. This allows for constrained geometric matching that enables the target to change within a certain angle, scale, and local deformation range, significantly improving the positioning robustness under complex working conditions. Simultaneously, local coordinate normalization unifies cotter pin images from different vehicle models, workstations, and shooting angles into a comparable coordinate space, ensuring consistency between subsequent structural measurements and geometric constraints, and guaranteeing the stability and comparability of detection results under different workstations and poses.

[0013] Furthermore, the structure parsing step includes a traditional visual extraction sub-step, a deep learning extraction sub-step, and a structured fusion sub-step. The traditional visual extraction sub-step performs edge detection, binarization segmentation, skeletonization extraction, and connected component analysis on the local standard pose image to obtain visual results. The visual results include edge contours, binary foreground, skeleton centerline, connected components, endpoints, and bifurcation points. The deep learning extraction sub-step infers the local standard pose image through an object detection network, a segmentation network, or a keypoint detection network and outputs the network results. The network results include the object detection results of the cotter pin, the segmentation mask, the keypoint positions of the V vertex and the left and right arm endpoints, the local orientation estimation, and the keypoint visibility confidence. The structured fusion sub-step converts the visual and network results into a unified structured representation oriented towards the cotter pin. This structured representation includes the V vertex position, left arm skeleton, right arm skeleton, left arm endpoint, right arm endpoint, left arm direction vector, right arm direction vector, and relative positional relationships with adjacent components. By parallel extraction and unified fusion of traditional vision and deep learning, the interpretability of traditional vision algorithms and the semantic expression capabilities of deep learning algorithms are balanced, enabling the system to possess both high recall and geometric interpretability under complex working conditions.

[0014] Furthermore, the dual-score calculation step includes an existence score calculation sub-step and a visibility score calculation sub-step. The existence score calculation sub-step samples the local structural response at each point along the left arm skeleton path, performs weighted aggregation on the local structural response, and then maps it to a preset score interval through a monotonically compressed function to obtain the left arm existence score; the same sub-step samples the local structural response at each point along the right arm skeleton path, performs weighted aggregation on the local structural response, and then maps it to a preset score interval through a monotonically compressed function to obtain the right arm existence score. The visibility score calculation sub-step calculates the left arm visibility score using a linear weighted method based on the visibility confidence of the left arm endpoint keypoints, the left arm edge continuity score, the left arm skeleton connectivity score, the left arm region reflectivity, the left arm region stain ratio, and the left arm region boundary truncation risk. Similarly, it calculates the right arm visibility score using a linear weighted method based on the right arm endpoint keypoint visibility confidence, the right arm edge continuity score, the right arm skeleton connectivity score, the right arm region reflectivity, the right arm region stain ratio, and the right arm region boundary truncation risk. By modeling "whether there is a stable structural response" and "whether there are sufficient observation conditions" as existence scores and visibility scores, respectively, the system can distinguish between "true absence" (low existence, high visibility) and "invisible but present" (low existence, low visibility), providing separate inputs of structural and visibility evidence for subsequent fusion judgment.

[0015] Furthermore, the fusion judgment step includes a fusion strategy, which includes calculating a geometric consistency score based on the angle and ratio between the left and right arm skeletons, the stability of the V-vertex, and the connectivity of the left and right arm skeletons; calculating an interference penalty term based on the reflection ratio, stain ratio, occlusion ratio, and boundary truncation risk in the image to be detected; and then weighting and fusing the geometric consistency score, interference penalty term, and the existence and visibility scores of the left and right arms to obtain a comprehensive judgment score. This integrates structural existence evidence, visibility evidence, geometric rationality evidence, and interference factors into the comprehensive judgment model, enabling multi-evidence joint decision-making.

[0016] Furthermore, in the fusion judgment step, when the overall determineability score meets the determineability requirement and the comprehensive judgment score is higher than the qualified threshold, a normal state is output; when the overall determineability score meets the determineability requirement and the comprehensive judgment score is lower than the defect threshold, a suspected defect state is output; when the comprehensive judgment score is between the qualified threshold and the defect threshold, or when the overall determineability score does not meet the determineability requirement, a pending review state is output. Thus, by replacing the traditional binary classification mode with a three-state output mechanism, a definitive conclusion is given when the evidence is sufficient, and a review is actively requested when the evidence is conflicting or insufficient, thereby reducing high-risk misjudgments.

[0017] This invention also provides a rail train cotter pin defect detection system, including a quality assessment module, a flexible positioning module, a structural analysis module, a dual-scoring calculation module, and a fusion judgment module, which are respectively used to perform the steps in the above method. The modules interact with each other through data interface fields with clear definitions, thereby realizing the complete process from image input to detection result output.

[0018] The beneficial effects of this invention are as follows: 1. By using quality gating to filter low-quality samples at the detection front end, low-quality images are prevented from directly entering subsequent judgments, leading to false alarms and improving the reliability of the detection system; 2. By using flexible matching positioning and local coordinate normalization, cotter pin images under different work positions and postures are unified into a comparable coordinate space, giving subsequent structural measurements consistent mathematical meaning; 3. By using V-shaped double-arm topology modeling, cotter pins are explicitly modeled as topological components with double-arm structures, allowing the algorithm to calculate existence evidence around the double arms separately, achieving structural defect localization; 4. By using joint modeling of arm-level existence scoring and visibility scoring, the system can distinguish between real defects and false defects, significantly reducing the false alarm rate; 5. By using three-state output and verification, verification is actively triggered when there is conflict or insufficiency of evidence, rather than forcibly outputting conclusions, meeting the requirements of reliability and traceability in detection. 2. By parallel extraction and unified fusion of traditional vision and deep learning, both interpretability and semantic expression capabilities are taken into account, making it easy to deploy and maintain in actual inspection scenarios. In addition, through the occlusion compensation step, when the quality assessment is triggered to be reviewed due to a single occlusion ratio, the system judges whether there is evidence of viewpoint overlap or shadow on both arms, corrects the false defect samples caused by viewpoint overlap or shadow, and reintroduces special positive samples that would otherwise be rejected into the decision-making link, reducing unnecessary review and manual intervention, and improving the automation completion rate of the detection system. Attached Figure Description

[0019] Figure 1 This is a diagram of the overall method logic framework in this invention; Figure 2 This is a system framework diagram of the present invention; Figure 3 This is a flowchart of the occlusion compensation step in this invention; Figure 4 This is a schematic diagram of the topological modeling of the V-shaped double arm with a cotter pin in this invention; Figure 5 This is a schematic diagram of an actual scenario in Embodiment 2 of the present invention. Detailed Implementation

[0020] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Identical components are denoted by the same reference numerals. It should be noted that the terms "front," "rear," "left," "right," "upper," and "lower" used in the following description refer to directions in the accompanying drawings, and the terms "bottom surface," "top surface," "inner," and "outer" refer to directions toward or away from the geometric center of a specific component, respectively.

[0021] This invention provides a defect detection system for cotter pins on railway trains, such as... Figure 2 As shown, this system can be deployed in train inspection robots, undercarriage inspection robots, or fixed image acquisition equipment to automatically determine whether cotter pins in rail train bogies, braking mechanisms, connecting mechanisms, fork mechanisms, pin limit mechanisms, and various anti-loosening and anti-detachment assembly structures have defects such as broken or missing arms. Cotter pins in common assembly states exhibit a V-shaped double-arm unfolded structure, characterized by a V-shaped apex, two branch arms, and two end points. When one arm of the cotter pin is broken, missing, severely bent, or improperly assembled, the limiting function may be weakened or even completely failed. This invention, based on the above principle, automatically identifies the integrity of the cotter pin's double-arm structure using machine vision, distinguishing between "real defects" and "false defects caused by obstruction, incomplete acquisition, stains, reflections, etc.", thereby outputting three states: normal, suspected defect, or pending verification.

[0022] Example 1 This embodiment provides a specific implementation method for detecting defects in cotter pins on railway trains. The overall process of this method is as follows: Figure 1 As shown, it includes quality assessment steps, flexible positioning steps, structural analysis steps, dual-scoring calculation steps, and fusion judgment steps.

[0023] In the quality assessment step, the first step is to acquire the image of the cotter pin to be inspected, which is captured by the camera on the inspection robot. After the image is acquired, it is preprocessed, including brightness normalization and noise suppression, so that the edge, texture and structural response of the cotter pin area has higher separability. Then, the image quality is assessed, and multiple quality indicators are calculated, including global sharpness, local sharpness, reflective coverage ratio, stain coverage ratio, occlusion ratio and boundary truncation risk.

[0024] In this invention, the calculation of global sharpness is preferably described using Laplace variance, specifically: ,in For the Laplacian response of the image, To improve sensitivity to small targets like cotter pins, the system can also calculate local sharpness indices within the left and right arm local windows for variance calculation. and It is used to identify special conditions where "the entire image is clear but the arm is out of focus in some areas"; the reflective coverage ratio is determined by the brightness of the image. The component is higher than the preset high brightness threshold and the saturation The percentage of pixels with saturation below a preset low saturation threshold is obtained by statistical analysis; the percentage of dirt coverage is obtained by statistical analysis of pixel regions in the image with abnormal texture roughness and brightness significantly lower than the neighborhood average; the percentage of occlusion is obtained by analyzing the contour integrity and boundary continuity within the cotter pin candidate region; the risk of boundary truncation is obtained by calculating the pixel distances from the endpoints of the left and right arms to the image boundary, and when the distance from the endpoint of any arm to the image boundary is less than a preset distance threshold, the arm is determined to have a risk of boundary truncation.

[0025] Determine whether each quality indicator meets the corresponding preset judgment conditions, such as global sharpness. The resolution must be greater than a preset clarity threshold, the reflective coverage ratio must be less than a preset reflective threshold, the occlusion ratio must be less than a preset occlusion threshold, and the risk of boundary truncation must be lower than a preset truncation threshold. When any quality indicator is not met, a pending review status is output and the detection is terminated, thereby filtering out low-quality samples that are not suitable for direct judgment at the detection front end. Through the comprehensive evaluation of multi-dimensional quality indicators, it is possible to avoid low-quality images from directly entering the subsequent judgment process and causing false alarms.

[0026] When the quality assessment result meets the preset judgment conditions, the process enters the flexible positioning step. In the flexible positioning step, flexible matching positioning is performed on the image to be detected, the region where the cotter pin is located is searched, and the positioning parameters of the target candidate region are output. Specifically, at least one of template features, edge features, local feature point descriptors, or shape context is used to perform sliding window search or feature point matching on the image to be detected. The flexible matching of the present invention is not a simple fixed template comparison, but a constrained geometric matching process that allows the target to change within a certain angle, scale, and local deformation range. After the search is completed, the center position, principal orientation angle, scale factor, reference anchor point, and matching confidence of the target candidate region are output.

[0027] Subsequently, based on the reference anchor point and its adjacent reference structures, the target candidate region is mapped to a unified local standard pose through at least one of the following methods: rotation, translation, scale normalization, or perspective correction, thus establishing a local coordinate system. This is done if the physical dimensions of the reference length are known under actual calibration conditions. and corresponding pixel length Then a scaling factor can be introduced. This process unifies all subsequent pixel measurements into physical scale measurements. As a result, cotter pin images from different vehicle models, workstations, and shooting angles are unified into a comparable coordinate space. Subsequent structural measurements and geometric constraints thus have consistent mathematical meaning, ensuring that the detection results remain stable and comparable under different workstations and postures.

[0028] In the structural analysis step, a local coordinate system is established based on the positioning parameters, and the target candidate region is mapped to a unified local standard pose. Then, two types of processing, traditional visual structure extraction and deep learning structure analysis, are performed in parallel in the local standard pose image to extract the structured representation of the cotter pin.

[0029] Traditional visual extraction involves edge detection (e.g., Canny edge detection), binarization segmentation (e.g., OTSU global thresholding or adaptive thresholding), skeletonization extraction (e.g., Zhang-Suen thinning algorithm), and connected component analysis on local standard pose images to obtain edge contours, binary foreground, skeleton centerline, connected components, endpoints, and bifurcation points. For connected components, isolated noise regions are further removed through area filtering.

[0030] The deep learning extraction part uses object detection networks (such as YOLO series), segmentation networks or keypoint detection networks to infer local standard pose images and outputs object detection results for cotter pins, segmentation masks, keypoint positions of V vertices and left and right arm endpoints, local orientation estimates and keypoint visibility confidence. The system does not regard any one result as the only true value, but converts the two types of results into a structured representation oriented towards cotter pins.

[0031] In the structured fusion sub-step, the system first maps the visual results output from the traditional visual extraction sub-step, such as edge contours, skeleton centerlines, endpoints, and bifurcation points, to the network results output from the deep learning extraction sub-step, such as segmentation masks, keypoint locations, local orientation estimations, and keypoint visibility confidence, all to a local standard pose coordinate system. Specifically, the skeleton point sequence output from traditional vision is represented by its pixel coordinates in the image, while the keypoints output from deep learning are represented by their normalized coordinates in the segmentation mask. The system unifies the two to the same coordinate space using a pre-established local standard pose coordinate transformation matrix, such as... Figure 4 As shown; Subsequently, the system performs association matching between traditional visual skeleton endpoints and deep learning keypoints near the same spatial location. A nearest neighbor matching strategy is used to mark point pairs with a distance less than a preset association threshold as different observations of the same structural point. When the same structural point has both traditional visual and deep learning observations, the average of the two coordinates is used as the final coordinates of the point, and the maximum confidence score of the two is taken as the fusion confidence score of the point. When only traditional visual observations or only deep learning observations are used, the observation result is directly used as the final output, but the fusion confidence score of the corresponding structural point is reduced. Finally, based on the V-vertex position, left arm skeleton path, right arm skeleton path, left arm endpoint, right arm endpoint, left arm direction vector, right arm direction vector, and relative positional relationship with adjacent components after association matching, the system constructs a complete structured representation of the cotter pin.

[0032] In the dual-score calculation step, the existence score and visibility score of the left and right arms are calculated respectively based on the structured representation. The existence score is used to reflect whether there is a stable structural response in the corresponding arm, and the visibility score is used to reflect whether the corresponding arm has sufficient observation conditions.

[0033] In the existence score calculation sub-step, the local structural response of each point along the left arm skeleton path is sampled, and the local structural response is weighted and aggregated and then mapped to the preset score interval through a monotonic compression function to obtain the existence score of the left arm; the local structural response of each point along the right arm skeleton path is sampled, and the local structural response is weighted and aggregated and then mapped to the preset score interval through a monotonic compression function to obtain the existence score of the right arm. Specifically, let the first The skeleton path of the strip arm is The sampling points on the path are The local structural response is Then the existence of the arm level is evaluated. Estimate using the formula: ,in This represents the number of path sampling points. The path position weight function assigns higher weights to positions closer to vertex V. For the first The existence threshold of the bar arm For monotonic compression functions, such as the Sigmoid function, The larger the value, the more likely the arm is to actually exist in the current image.

[0034] In the visibility score calculation sub-step, the left arm visibility score is calculated using a linear weighted method based on the visibility confidence of the left arm endpoint keypoints, the left arm edge continuity score, the left arm skeleton connectivity score, the left arm region reflectivity, the left arm region stain ratio, and the left arm region boundary truncation risk. Similarly, the right arm visibility score is calculated using a linear weighted method based on the visibility confidence of the right arm endpoint keypoints, the right arm edge continuity score, the right arm skeleton connectivity score, the right arm region reflectivity, the right arm region stain ratio, and the right arm region boundary truncation risk.

[0035] Left arm visibility score For example, it can be constructed according to the formula: ,in, For the visibility confidence of key points on the left arm, Score for the continuity of the left arm's edge. Score the connectivity of the left arm skeleton. The reflectivity of the left arm area. The percentage of stains in the left arm area. To mitigate the risk of truncation at the left arm boundary, All are weighted coefficients, and the right arm visibility score is also considered. Constructed in the same way.

[0036] By modeling "whether there is a stable structural response" and "whether there are sufficient observation conditions" as existence scores and visibility scores respectively, the system can distinguish between "true absence, i.e., low existence and high visibility" and "invisible but present, i.e., low existence and low visibility", providing separate inputs of structural evidence and visibility evidence for subsequent fusion judgment.

[0037] In the fusion judgment step, the geometric consistency score is calculated based on the angle and ratio between the left and right arm skeletons, the stability of the V vertex, and the connectivity of the left and right arm skeletons. The interference penalty is calculated based on the reflection ratio, stain ratio, occlusion ratio, and boundary truncation risk in the image to be detected. Then, the geometric consistency score, interference penalty, existence score, and visibility score of the left and right arms are weighted and fused to obtain the comprehensive judgment score.

[0038] Specifically, let the existence score of the left arm be... The right arm presence score was The visibility score of the left arm was The visibility score of the right arm was Geometric consistency score: Interference penalty item is Then, the comprehensive judgment score Calculated using the formula ,in The weights for existence evidence, geometric consistency evidence, and interference penalty evidence are respectively, along with the geometric consistency score. The construction method is based on the angle between the left and right arm skeletons. and length ratio Stability score of V vertex and the connectivity score of the left and right arm skeletons and Perform weighted fusion: ,in, , For length ratio consistency function, The closer to 1, the higher the score. For the angle consistency function, The higher the score within the preset normal angle range, the better. All are weighting coefficients; interference penalty term The construction method is as follows: ,in, For reflectivity, For the proportion of stains, To achieve the desired occlusion ratio, To cut off the risk at the boundary, As an indicator of uncertainty, All are weighting coefficients.

[0039] When the overall decidability score The criteria for judgment are met and the overall judgment score is [score]. When the value exceeds the acceptable threshold, the system outputs a normal status and an overall determineability score. The preferred structure is ,when When the value is less than the preset threshold, the system will output a status pending review instead of directly outputting a defect conclusion.

[0040] When the overall decidability score When the overall decidability score When the score is below the defect threshold, the system outputs a suspected defect status; specifically, when the left arm's presence score is... The left arm visibility score was significantly lower than the preset threshold. When the value remains high, it indicates that the left arm region has sufficient observation conditions but lacks real structural evidence, and the system tends to output defects such as missing or broken left arms; the same analysis applies to defects in the right arm.

[0041] When the comprehensive judgment score Between the acceptable threshold and the defective threshold, or the overall determineability score If the criteria for determination are not met, the system outputs a "pending review" status. This status indicates that the current image evidence is insufficient to provide a definitive conclusion, and a re-examination or manual review is required.

[0042] Example 2 This embodiment is based on the case where the left and right arms overlap in the imaging view during the occlusion compensation step; In actual train inspection robot image acquisition scenarios, such as Figure 5 As shown, due to the limitations of the shooting angle, one arm of the cotter pin may overlap with the other arm on the projection plane, resulting in only one complete arm being visible in the image. The other arm is obscured by itself, making it impossible to fully observe its skeleton and outline. Such special positive samples are easily misjudged as "single arm missing" in traditional detection schemes, when in fact the cotter pin is intact. In this embodiment, an occlusion compensation step is used to correct this situation, bringing the special positive samples that would otherwise be rejected back into the scope of judgment.

[0043] When only the occlusion ratio fails to meet the preset judgment criteria during the quality assessment step, while all other quality indicators meet the criteria, the system triggers the occlusion compensation step, such as... Figure 3 As shown, the pixel distance between the left and right arm skeletons on the projection plane is first calculated. That is, for each sampling point on the path of the left arm skeleton and each sampling point on the path of the right arm skeleton, the Euclidean distance between them is calculated. The minimum value among all distances is taken as the minimum pixel distance between the left and right arm skeletons. If the minimum pixel distance is less than the preset overlap distance threshold, it is determined that the left and right arms overlap in the imaging view. After determining that there is an overlap, the system detects at least one of the following features of color consistency, boundary continuity or reflectivity consistency in the overlapping area.

[0044] The existing methods for calculating color consistency features involve extracting the RGB color values ​​of the corresponding positions of the left and right arms within the overlapping area, calculating the color difference between them, and if the color difference is less than a preset color threshold, it indicates that the colors in the overlapping area are consistent, supporting the judgment that both arms truly exist. The existing methods for calculating boundary continuity features involve detecting whether the boundaries of the left and right arms form a continuous transition at the overlapping point within the overlapping area, that is, judging whether the distance between the endpoints of the left and right arm boundaries is less than a preset boundary continuity threshold. If it is less, the boundary at that point has continuity, supporting the judgment that both arms truly exist. The existing methods for calculating reflectivity consistency features involve detecting whether the reflectivity patterns on the surfaces of the left and right arms are consistent within the overlapping area. If the reflectivity patterns are consistent, it indicates that both arms are under similar imaging conditions at that viewpoint, further supporting the judgment that both arms truly exist.

[0045] Based on at least one of the aforementioned features of color consistency, boundary continuity, or reflectivity consistency, the system confirms that the two arms still exist independently even when overlapping. At this point, the system performs flexible matching localization on the image to be detected based on the feature localization results of the overlapping area. This involves re-searching for a template, but limiting the search range to the vicinity of the overlapping area. The search template used is a cotter pin template containing the complete structure of both arms. After flexible matching localization, the structured representation of the cotter pin is re-extracted based on the results, and the dual-score calculation and fusion judgment steps continue. The beneficial effect is that when the state to be verified is triggered only by the occlusion ratio caused by viewpoint overlap, the system can identify that the two arms are actually still intact, thus correcting the special positive sample from "to be verified" to a verifiable sample, ultimately outputting a normal state. This significantly reduces unnecessary verification and manual intervention caused by viewpoint overlap, improving the automation rate of the detection system.

[0046] Example 3 This embodiment is based on the case where the left and right arms do not overlap during the occlusion compensation step; When the pixel distance between the left and right arm skeletons on the projection plane is greater than or equal to the preset overlap distance threshold, the system determines that the left and right arms do not overlap. At this time, the occlusion ratio that triggers the review in the quality assessment may come from other factors, such as stains, shadow interference, or image boundary truncation. This embodiment is for this situation, and uses shadow evidence to determine whether the cotter pin arms actually exist.

[0047] Once it is determined that the left and right arms do not overlap, such as Figure 3 As shown, the system detects the shadow regions corresponding to the left and right arms respectively. The shadow detection method is to search for pixel regions with significantly lower brightness than the neighborhood average within the expected shadow area in the local standard pose image based on prior knowledge of the light source direction (the installation position and illumination direction of the supplementary light of the inspection robot are known). That is, the image is analyzed for brightness distribution, and the brightness difference between each pixel and its surrounding neighboring pixels is calculated. When the brightness difference exceeds the preset shadow difference threshold and the region is located in the expected shadow direction, the region is marked as a candidate shadow region.

[0048] After detecting the shadow area, the system extracts the center line, length, direction, and starting position of the shadow. The center line of the shadow is extracted by performing skeletonization processing on the shadow area; the direction of the shadow is obtained by fitting the main direction angle of the shadow center line with a straight line; the starting position of the shadow is determined by finding the endpoint on the shadow center line that is closest to the vertex or root of the cotter pin V.

[0049] Subsequently, the system calculates the orientation consistency score between the shadow direction and the expected projection direction of the corresponding arm under the assumption of its existence, as well as the distance score between the shadow origin and vertex V. The orientation consistency score is calculated as follows: ,in The main direction angle of the shadow. The expected projection direction angle of the arm, assuming its existence, is determined by the position of the V-vertex and the normal unfolding angle of the cotter pin. A score closer to 1 indicates a closer alignment between the shadow direction and the arm's expected projection direction. The distance score is calculated as follows: ,in The starting position of the shadow. The location of V is either the vertex or the root. The distance attenuation parameter is preset. The closer the score is to 1, the closer the shadow origin is to the root of the arm, supporting the idea that the shadow originates from the projection of that arm.

[0050] The shadow evidence score of the arm is calculated based on the weighted fusion result of the direction consistency score and the distance score. If the shadow evidence score of the target arm is higher than the preset shadow threshold, the system determines that the arm really exists. That is, the direction of the shadow is consistent with the expected projection direction of the corresponding arm, and the shadow start point matches the position of the V vertex. This proves that although the arm is not visible in the image, the existence of its shadow confirms that the arm does exist in the current imaging space. At this time, the system performs flexible matching and positioning on the image to be detected based on the center line and direction of the shadow. That is, the center line of the shadow is used as an auxiliary positioning clue to search for the complete cotter pin structure near the corresponding position of the shadow, re-extract the structured representation of the cotter pin, and continue to execute the dual score calculation step and the fusion judgment step.

[0051] If the shadow evidence score of the target arm is not higher than the preset shadow threshold, the system cannot confirm the existence of the arm, maintains the pending review status and terminates the detection, waiting for subsequent re-shooting or manual review. Through the above processing, when there is no visual overlap between the two arms but the occlusion ratio is abnormal, the system can detect shadows as indirect evidence of the arm's existence. If the direction of the shadow is consistent with the expected projection direction of the corresponding arm and the shadow's starting point matches the position of the V-vertex, it proves that the arm truly exists. This corrects the false defect sample caused by the shadow into a definite sample, further reducing unnecessary review and manual intervention, and improving the automation completion rate of the detection system.

[0052] The above are merely preferred embodiments of the present invention. The scope of protection of the present invention is not limited to the above embodiments. All technical solutions falling within the scope of the present invention's concept are within the scope of protection of the present invention. It should be noted that for those skilled in the art, any improvements and modifications made without departing from the principle of the present invention should also be considered within the scope of protection of the present invention.

Claims

1. A method for detecting defects in cotter pins of railway trains, characterized in that: Includes the following steps: The quality assessment step involves evaluating the image quality of the image to be detected and determining whether to continue the detection based on the quality assessment results. The flexible positioning step involves performing flexible matching positioning on the image to be detected when it is determined that the detection will continue, searching for the area where the cotter pin is located, and outputting the positioning parameters of the target candidate area. The structural analysis step involves establishing a local coordinate system based on the positioning parameters, mapping the target candidate region to a unified local standard pose, and extracting the structured representation of the cotter pin from the local standard pose image. The structured representation includes the V vertex position, left arm skeleton, right arm skeleton, left arm endpoint, and right arm endpoint. The dual-score calculation step calculates the existence score and visibility score of the left and right arms respectively based on the structured representation. The existence score is used to reflect whether the corresponding arm has a stable structural response, and the visibility score is used to reflect whether the corresponding arm has sufficient observation conditions. The fusion judgment step calculates a comprehensive judgment score based on the existence score, visibility score, and geometric consistency score, and outputs a normal state, a suspected defect state, or a state pending review based on the comprehensive judgment score.

2. The method for detecting defects in cotter pins of railway trains according to claim 1, characterized in that: The structural analysis steps include: Traditional visual extraction sub-steps involve edge detection, binarization segmentation, skeleton extraction, and connected component analysis on the local standard pose image to obtain the visual result; The deep learning extraction sub-step involves reasoning about the local standard pose image through object detection networks, segmentation networks, or keypoint detection networks and outputting the network results. The structured fusion sub-step unifies the visual results and network results into a structured representation oriented towards the cotter pin. The structured representation includes the V vertex position, left arm skeleton, right arm skeleton, left arm endpoint, right arm endpoint, left arm direction vector, right arm direction vector, and relative positional relationship with adjacent components.

3. The method for detecting defects in cotter pins of railway trains according to claim 2, characterized in that: The visual results include edge contours, binary foreground, skeleton centerline, connected components, endpoints, and bifurcation points; the network results include target detection results for cotter pins, segmentation masks, key point positions of V vertices and left and right arm endpoints, local orientation estimation, and key point visibility confidence.

4. The method for detecting defects in cotter pins of railway trains according to claim 3, characterized in that: The dual-score calculation step includes an existence score calculation sub-step, which includes sampling the local structural response at each point along the left arm skeleton path, weighting and aggregating the local structural responses, and mapping them to a preset score interval through a monotonically compressed function to obtain the left arm existence score; sampling the local structural response at each point along the right arm skeleton path, weighting and aggregating the local structural responses, and mapping them to a preset score interval through a monotonically compressed function to obtain the right arm existence score.

5. The method for detecting defects in cotter pins of railway trains according to claim 4, characterized in that: The dual-score calculation step also includes a visibility score calculation sub-step. This sub-step calculates the left arm visibility score using a linear weighted method based on the visibility confidence of the left arm endpoint keypoints, the left arm edge continuity score, the left arm skeleton connectivity score, the left arm region reflectivity, the left arm region stain ratio, and the left arm region boundary truncation risk. Similarly, it calculates the right arm visibility score using a linear weighted method based on the right arm endpoint keypoint visibility confidence, the right arm edge continuity score, the right arm skeleton connectivity score, the right arm region reflectivity, the right arm region stain ratio, and the right arm region boundary truncation risk.

6. The method for detecting defects in cotter pins of railway trains according to claim 5, characterized in that: The fusion determination step includes a fusion strategy, which includes calculating a geometric consistency score based on the angle and ratio between the left and right arm skeletons, the stability of the V-vertex, and the connectivity of the left and right arm skeletons; calculating an interference penalty term based on the reflection ratio, stain ratio, occlusion ratio, and boundary truncation risk in the image to be detected; and then weighting and fusing the geometric consistency score, interference penalty term, and the existence and visibility scores of the left and right arms to obtain a comprehensive determination score.

7. A defect detection system for cotter pins on railway trains, characterized in that: include: The quality assessment module performs image quality assessment on the image to be detected and determines whether to continue the detection based on the quality assessment results. The flexible positioning module performs flexible matching positioning on the image to be detected when it is determined that the detection will continue, searches for the area where the cotter pin is located, and outputs the positioning parameters of the target candidate area. The structural analysis module establishes a local coordinate system based on the positioning parameters, maps the target candidate region to a unified local standard pose, and extracts the structured representation of the cotter pin in the local standard pose image. The structured representation includes the V vertex position, left arm skeleton, right arm skeleton, left arm endpoint, and right arm endpoint. The dual-score calculation module calculates the existence score and visibility score of the left and right arms respectively based on the structured representation. The existence score is used to reflect whether the corresponding arm has a stable structural response, and the visibility score is used to reflect whether the corresponding arm has sufficient observation conditions. The fusion judgment module calculates a comprehensive judgment score based on the existence score, visibility score, and geometric consistency score, and outputs a normal status, a suspected defect status, or a pending review status based on the comprehensive judgment score.

Images