A low-light environment-oriented automated bolt set failure detection method

Abstract

The application discloses a kind of low-light environment-oriented automated bolt group fault detection method, including the original color image containing target bolt group is collected and is enhanced processing, then input semantic segmentation neural network, obtain the semantic segmentation mask containing "bolt" area and "anti-loose rope" area, fine anti-loose rope area state is judged, and corresponding to the loose determination of the marker line of bolt area, the rope state of the determination and the bolt loose detection result of the determination are comprehensively generated final fault detection result.The scheme can realize the full-automatic, non-contact detection of train underframe bolt group;Effectively overcome the imaging and identification problem under low light, oil stain, rust and other complex environments;Establish a hierarchical intelligent diagnostic logic deeply fused with mechanical anti-loose principle, can cooperatively and accurately judge the two types of faults of anti-loose rope fracture and bolt loosening, and output the hierarchical early warning information that can directly guide operation and maintenance.

Description

Technical Field

[0001] This invention belongs to the field of computer vision and industrial equipment condition monitoring, and specifically relates to an automated bolt group fault detection method for low-light environments. Background Technology

[0002] Bolted connections, as a fundamental mechanical connection method for critical structures such as the chassis of rail transit vehicles, directly impact operational safety in terms of reliability. To ensure absolute safety, bolt groups in critical areas are often physically connected using anti-loosening ropes, forming a mechanically interlocked "anti-loosening system." This system exhibits two typical and highly correlated failure modes: first, the breakage or detachment of the anti-loosening ropes, a direct manifestation of primary system failure; second, the loosening of one or more bolts, a specific indication of secondary or impending system failure. Currently, the industry still heavily relies on periodic manual visual inspections for monitoring the condition of such components. This method is not only inefficient and costly in terms of manpower, but its inspection quality is also severely limited by personnel experience, work fatigue, and harsh environments such as dim lighting, confined spaces, and oil stains under the vehicle, posing a risk of missed detections and misjudgments. It is insufficient to meet the high-density, high-safety maintenance requirements of modern rail transit.

[0003] With the development of computer vision technology, image-based automated inspection methods have become a research hotspot in industrial inspection, and preliminary explorations have been conducted in the field of bolt inspection. However, existing methods still have significant limitations when dealing with the specific scenario of undercarriage bolt assemblies, mainly in the following three aspects:

[0004] 1. Insufficient environmental adaptability: The space under the vehicle is poorly lit, and there are common problems such as oil stains, rust interference, and vibration blurring caused by vehicle operation. Traditional image enhancement algorithms suffer severe performance degradation in such complex low-light environments, making it difficult to extract target features stably and clearly.

[0005] 2. Isolated Fault Diagnosis Logic: Most existing research treats "bolt inspection" and "rope inspection" as two independent tasks, or focuses only on the "presence" or "absence" of a single target. This isolated diagnostic approach, which "misses the forest for the trees," completely ignores the inherent and mandatory physical logical connection between "anti-loosening ropes" and "connected bolts"—that is, "an intact rope means the system as a whole is under control and the risk is low; a broken rope means the first line of defense has failed, and the tightness of each connected bolt must be precisely checked immediately." Existing technologies have failed to transform this core engineering knowledge into a computable and executable collaborative diagnostic logic, resulting in fragmented outputs that cannot form a reliable assessment of the overall health of the system.

[0006] 3. Lack of engineering practicality: Most research remains at the laboratory level, focusing on improving target recognition accuracy, failing to translate the actual mechanical anti-loosening principles into calculable and executable diagnostic rules. This results in algorithm outputs that are mostly isolated recognition results rather than decision-making information that can directly guide maintenance. The alarm logic is crude, with high false alarm and false negative rates, failing to meet the engineering requirements of efficient and accurate operation and maintenance.

[0007] In particular, in the core step of bolt loosening detection—marker line status analysis—existing methods perform poorly in low-light and multi-interference environments (such as old marker line residues), and lack robust extraction algorithms that can effectively distinguish new painted marker lines from similar colored interference.

[0008] In summary, existing technologies struggle to achieve automated, highly reliable, and engineered diagnostics of the health status of bolt systems under low-light conditions. Therefore, there is an urgent need for an innovative technological solution that can deeply integrate domain knowledge, adapt to harsh imaging conditions, and perform collaborative quantitative analysis and intelligent early warning of the condition of anti-loosening ropes and bolts. This solution would fundamentally replace traditional manual inspections and improve operational safety and intelligence. Summary of the Invention

[0009] The purpose of this invention is to provide an automated bolt group fault detection method for low-light environments, realizing fully automatic and non-contact detection of train undercarriage bolt groups; effectively overcoming the imaging and recognition difficulties in complex environments such as low light, oil stains, and corrosion; and establishing a hierarchical intelligent diagnostic logic that deeply integrates the mechanical anti-loosening principle, which can collaboratively and accurately judge two types of faults: anti-loosening rope breakage and bolt loosening, and output graded early warning information that can directly guide operation and maintenance.

[0010] The specific technical solution for achieving the objective of this invention is as follows:

[0011] An automated bolt group fault detection method for low-light environments includes the following steps:

[0012] Step 1: Acquire the original color image of the train undercarriage containing the target bolt group;

[0013] Step 2: Perform low-light enhancement processing on the original image based on a deep learning model to obtain the enhanced image;

[0014] Step 3: Input the enhanced image into the trained semantic segmentation neural network for processing to obtain a semantic segmentation mask containing the "bolt" region and the "anti-loosening rope" region;

[0015] Step 4: Based on the anti-loosening rope region in the semantic segmentation mask, perform morphological operations, connected component analysis, and rope linear geometry analysis to finely determine the specific state of the anti-loosening rope, including normal and abnormal states; when the rope is determined to be normal, directly output the normal state of the system and end the detection process.

[0016] Step 5: When the rope is determined to be in an abnormal state, for each independent bolt instance, the corresponding image region is extracted from the enhanced image. Then, in the HSV color space, an adaptive clustering algorithm based on the clustering effectiveness index is used, combined with prior knowledge of the interference features to filter out the marker line extraction method. This effectively separates and filters out color-similar interference, thereby accurately extracting the red marker line on the bolt used for loosening determination.

[0017] Step 6: Perform fragment region analysis on the extracted marking lines, and establish a quantitative loosening judgment model based on the number of fragment regions and their geometric distribution characteristics after the marking lines break, to determine whether the corresponding bolts have become loose.

[0018] Step 7: Combine the determined rope condition with the determined bolt loosening detection results to generate the final fault detection result.

[0019] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0020] (1) It has achieved full automation and high efficiency of the testing process, which can replace the high-risk and low-efficiency manual inspection and meet the needs of large-scale and high-frequency operation and maintenance.

[0021] (2) It has excellent low-light and environmental adaptability. Through deep learning enhancement model, the quality of input images under poor lighting conditions is fundamentally improved; the improved segmentation and clustering algorithms have strong robustness against common interferences such as oil stains and rust;

[0022] (3) An innovative hierarchical collaborative diagnostic logic was proposed. A hierarchical fault detection model based on the rope condition was first created, which is in line with the mechanical anti-loosening principle. If the rope is intact, the system is safe, which greatly reduces false alarms and redundant calculations; if the rope is abnormal, bolt detection is initiated. The logic is rigorous and the results are reliable.

[0023] (4) Linear geometric structure analysis is introduced in the rope state determination process. By integrating multi-dimensional geometric features such as straightness deviation, curvature change, and direction consistency, a quantitative description of the "straight" state of the rope is realized. It can finely distinguish the normal, slack, potential breakage, breakage, and detachment states of the rope, overcoming the problem of coarse judgment caused by the traditional method relying only on connected component counting.

[0024] (5) A unified triggering mechanism is adopted: the process is terminated early only when the rope is in normal condition to avoid unnecessary calculations; when any abnormal condition of the rope occurs (including slack, potential breakage, breakage, or detachment), the subsequent bolt precision inspection is triggered. This design ensures the comprehensiveness of the inspection and avoids the logical complexity caused by excessive grading in the rope stage;

[0025] (6) It provides a concise and efficient three-level early warning output. Based on the combination of rope status and bolt status, it distinguishes three different levels of alarms: "Level 1 (highest risk)", "Level 2 (medium-high risk)" and "Level 3 (low risk)". The output information is concise and clear and can be directly used to guide differentiated maintenance decisions, thereby improving the economy of operation and maintenance while ensuring safety.

[0026] The present invention will be further described below with reference to specific embodiments. Attached Figure Description

[0027] Figure 1 The overall flowchart of the automated bolt group fault detection method provided in the embodiment of the present invention is shown.

[0028] Figure 2 This is an image of the target bolt group in an embodiment of the present invention.

[0029] Figure 3 This is a comparison image before and after low-light image enhancement processing in an embodiment of the present invention, where sub-image (a) is the original low-light image and sub-image (b) is the image enhanced by the Kind++ network.

[0030] Figure 4 This is a comparative diagram of a normal anti-loosening rope and a broken anti-loosening rope in an embodiment of the present invention.

[0031] Figure 5 The images show a comparison of the segmentation effect of the improved PIDNet model on the anti-loosening rope in the embodiments of the present invention. Sub-image (a) is the input color image, sub-image (b) is the segmentation result of the original PIDNet model, and sub-image (c) is the segmentation result of the improved PIDNet model.

[0032] Figure 6 The image shown here is a color image of a single bolt containing a mixture of old and new marking lines, illustrating a real-world interference scenario encountered in marking line detection.

[0033] Figure 7 This is a process diagram showing the effect of extracting marker lines using the adaptive clustering segmentation method based on the HSV color space in an embodiment of the present invention. It fully presents the optimized process from initial screening, clustering to post-processing.

[0034] Figure 8This is a schematic diagram of the bolt loosening determination process when the number of marked line fragment regions N=2 in an embodiment of the present invention. Sub-figure (a) is the original image of a single bolt, sub-figure (b) is the extracted binary image of the marked line, and sub-figure (c) is a schematic diagram of calculating the loosening state based on the included angle of the main directions of the two fragment regions.

[0035] Figure 9 This is a schematic diagram of the bolt loosening determination process when the number of marked line fragment regions N≥3 in an embodiment of the present invention. Sub-figure (a) is the original image of a single bolt, sub-figure (b) is the extracted binary image of the marked line (showing multiple fragments), and sub-figure (c) is a schematic diagram of loosening determination based on the linearity of all contour point sets (principal component analysis). Detailed Implementation

[0036] Example

[0037] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. The described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0038] As indicated in this application and claims, unless the context clearly indicates otherwise, the words "a," "an," "an," and / or "the" do not specifically refer to the singular and may also include the plural. Generally speaking, the terms "comprising" and "including" only indicate the inclusion of explicitly identified steps and elements, which do not constitute an exclusive list, and the method or apparatus may also include other steps or elements.

[0039] Unless otherwise specifically stated, the relative arrangement, numerical expressions, and values ​​of the components and steps described in these embodiments do not limit the scope of this application. It should also be understood that, for ease of description, the dimensions of the various parts shown in the drawings are not drawn to actual scale. Techniques, methods, and devices known to those skilled in the art may not be discussed in detail, but where appropriate, such techniques, methods, and devices should be considered part of the specification. In all examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not as limitations. Therefore, other examples of exemplary embodiments may have different values. It should be noted that similar reference numerals and letters in the following drawings denote similar items; therefore, once an item is defined in one drawing, it need not be further discussed in subsequent drawings.

[0040] Combination Figure 1 An automated bolt group fault detection method for low-light environments includes the following steps:

[0041] Step 1: Acquire the original color image of the train undercarriage containing the target bolt group;

[0042] In this embodiment, the image of the target bolt group is as follows: Figure 2 As shown. The assembly consists of three bolts arranged in a triangle, with each pair of bolts connected by a locking rope. The image acquisition device is a smartphone with high-resolution camera capabilities. The photographer uses a phone holder to extend the device under the vehicle to photograph specific sections of the bolt assembly, turning off the flash to avoid localized overexposure and shadow interference.

[0043] Because the ambient light level under a vehicle is typically extremely low (generally below 10 Lux) and the space is cramped, raw color images taken using ambient light tend to be generally dark, have low contrast, blurry details, and significant noise (e.g., Figure 3 (a) is shown.

[0044] Step 2: Perform low-light enhancement processing on the original image based on a deep learning model to obtain the enhanced image:

[0045] Step 2-1: Image decomposition. Input the acquired low-light original image and its corresponding normal exposure reference image into the decomposition network of the pre-trained KinD++ low-light image enhancement network. Decouple each image into the reflection component that represents the inherent properties of the object (such as the object's color, texture, etc.) and the illumination component that controls the brightness distribution of the scene. Through the reflection loss function, constrain the reflection components of the low-light image and the normal light image to tend to be consistent.

[0046] Step 2-2: Reflection component restoration. The reflection component of the decomposed low-light image, together with its illumination component, is input into the reflection restoration network in the KinD++ low-light image enhancement network.

[0047] The reflection restoration network adopts a fully convolutional structure with integrated multi-scale illumination attention (MSIA) modules to suppress noise, correct color distortion, and avoid halo artifacts caused by over-pooling operations, thereby restoring clear and clean reflection information.

[0048] Steps 2-3: Illumination component adjustment: Input the low-light image illumination components obtained from the decomposition into the illumination adjustment network in the KinD++ low-light image enhancement network;

[0049] The illumination adjustment network accepts an adjustable brightness coefficient. (Calculated from paired images during training, and can be manually set according to the target brightness during application) as input to achieve a flexible and controllable nonlinear mapping from low lighting conditions to desired lighting conditions;

[0050] Steps 2-4, Image Fusion and Output: The repaired reflection component and the adjusted illumination component are re-fused to generate an enhanced image with significantly improved visual quality;

[0051] The entire network is trained end-to-end by combining reflection loss, reconstruction loss, and illumination smoothing loss, without requiring real labels for reflection and illumination maps, relying only on low-light / normal-light image pairs.

[0052] Step 3: Input the enhanced image into the trained semantic segmentation neural network for processing to obtain a semantic segmentation mask containing the "bolt" region and the "anti-loosening rope" region;

[0053] The semantic segmentation neural network is an improved model based on the PIDNet architecture. On the basis of the original PIDNet structure, the network embeds the channel attention module ECA-Net at the end of the detail branch to enhance the response to high-frequency detail features such as rope edges.

[0054] In the semantic branches, deformable convolutional modules are used to replace standard convolutions to adapt to the geometric deformation of the target.

[0055] Enhance the image The input is fed into a trained improved PIDNet semantic segmentation neural network, which outputs a category label for each pixel, generating a semantic segmentation mask containing "Bolt", "Wire", and "Background". ;

[0056] The original PIDNet structure has two key improvements: an ECA-Net channel attention module is embedded at the end of the detail branch to enhance the response to high-frequency detail features such as rope edges; and deformable convolution modules are used to replace standard convolutions in the deep convolutions of the semantic branch to adapt to the geometric deformation of the target (such as a bolt hexagonal head or a slender rope).

[0057] Segmentation effect as follows Figure 5 As shown, in the original image (a), the ends of two adjacent ropes are close in space, forming an approximately continuous shape. The segmentation result of the baseline model (b) is limited by its fixed receptive field and context modeling ability, and fails to fully distinguish these two independent steel rope entities, incorrectly identifying and merging them into a continuous connected domain, resulting in misjudgment of the topological structure. This will directly affect the accuracy of subsequent connection-based breakage determination.

[0058] The improved model (c) performs significantly better in distinguishing elongated, close-proximity targets. Thanks to the introduced deformable convolutional module, the model adaptively adjusts the receptive field shape, focusing more accurately on the structural features of each rope. Simultaneously, the ECA attention mechanism enhances the selection of key local features, suppressing interference from background and neighboring information. Therefore, the improved model successfully segments two spatially adjacent but physically independent ropes into two correct independent connected domains, preserving their true topological structure.

[0059] Step 4: Based on the improved PIDNet semantic segmentation neural network, perform morphological operations, connected component analysis, and linear geometric structure analysis of the anti-loosening rope region in the semantic segmentation mask to finely determine the specific state of the anti-loosening rope, including normal and abnormal states; when the rope is determined to be normal, directly output the system's normal state and end the detection process.

[0060] Step 4-1: Semantic segmentation mask for the Wire region of the anti-loosening rope Morphological filtering is performed to remove isolated noise and fine burrs generated during segmentation;

[0061] Step 4-2: Perform connected component analysis on the morphologically processed binary image and count the number of connected regions in the image. ;

[0062] Step 4-3: Count the number of connected components obtained from the statistics. Compared with the pre-stored original number of intact ropes Compare: If (like Figure 4 (c) shows that the rope has come loose; if (like Figure 4 (b) shows that the rope has broken; if (like Figure 4 (a) then proceed to step 4-4 to perform linear geometric structure analysis;

[0063] Step 4-4: Perform linear geometric structure analysis on the rope region with a normal number of connected components, and determine its state:

[0064] Step 4-4-1: Extract the skeleton from the binary image of the rope region to obtain the centerline representation of the rope;

[0065] Step 4-4-2: Perform least-squares line fitting on the rope skeleton point set to obtain the fitted line. Let the point set of the rope skeleton be... ,in For the first The coordinates of the skeleton points The total number of skeleton points; calculate the distance from each skeleton point to the fitted line. vertical distance The formula is as follows:

[0066]

[0067] Calculate the root mean square error (RMSE) from each skeleton point to the fitted line, which serves as an indicator of straightness deviation.

[0068]

[0069] Step 4-4-3: Calculate the curvature change along the rope skeleton and detect if there are any abrupt curvature changes. For skeleton points... Its curvature change The calculation formula is:

[0070]

[0071] in, and The first derivative, and The second derivative is obtained approximately by the difference between adjacent points; if there exists a point with curvature... Exceeding the preset curvature threshold If the curvature changes drastically before and after this point, it is determined that there is a significant curvature abrupt change.

[0072] Step 4-4-4: Divide the rope frame into even segments, let it be divided into... Segment, calculate the principal direction angle of each segment. , ; Calculate the angle difference between adjacent segments If the sum of the angle differences of multiple consecutive adjacent segments exceeds the preset angle threshold If so, then the direction consistency is poor;

[0073] Step 4-4-5: Determine the status based on the comprehensive linear geometric structure analysis results: If the straightness deviation index is less than the preset first threshold, there is no significant curvature change, and the direction consistency is good, then the rope is judged to be normal; if the straightness deviation index is greater than or equal to the preset threshold but less than the preset second threshold, and there is no significant curvature change, then the rope is judged to be slack; if the straightness deviation index is greater than or equal to the preset second threshold, or there is a significant curvature change, or the direction consistency is poor, then the rope is judged to have a potential risk of breakage.

[0074] Steps 4-5: Perform corresponding processing based on the judgment results: When the rope is judged to be normal, directly output the normal status of the system and end the detection process; when the rope is judged to be in any abnormal state (slack / potential break / break / fallout), proceed to step 5.

[0075] Step 5: When the rope is determined to be in an abnormal state, for each individual bolt instance, extract its corresponding image region from the enhanced image. Then, in the HSV color space, use an adaptive clustering algorithm guided by clustering effectiveness indicators, combined with prior knowledge of interference features, to effectively separate and filter out color-similar interference, thereby accurately extracting the red marker line on the bolt used for loosening determination. In a real-world scenario, a single bolt color image containing a mixture of old and new marker lines is shown below. Figure 6 The image shows a real-world interference scenario encountered by marker line detection:

[0076] Step 5-1, instance separation, based on the... The mask area of ​​each bolt From enhanced images Crop the corresponding colored sub-image from the middle. ;

[0077] Step 5-2: Color Space Conversion and Initial Screening. Convert the color sub-image from RGB space to HSV space. Based on the distribution characteristics of the red marker line in the hue channel, set a threshold for initial screening (e.g., This yields a set of candidate pixels containing the red marker line and similarly colored distractors (such as the old marker line). ;

[0078] Step 5-3: Employ an adaptive clustering algorithm guided by a clustering effectiveness metric from candidate pixels. The target marker lines were precisely separated, including:

[0079] Step 5-3-1: Set the number of clusters The search range is set to [specific range] in this embodiment. ;

[0080] Step 5-3-2: For each candidate within the range The K-means clustering was performed, and then the Calinski-Harabasz (CH) index of the clustering results was calculated. The CH index evaluates the effectiveness of clustering by calculating the ratio of between-cluster dispersion to within-cluster dispersion. A higher value usually indicates a clearer cluster structure and better separation.

[0081] Step 5-3-3: Select the option that maximizes the CH index. The value is used as the final cluster number. The corresponding clustering results were then used for subsequent analysis.

[0082] Step 5-3-4: After obtaining the clustering results, each cluster is judged and filtered out based on prior domain knowledge rules to accurately separate the red-marked lines, including:

[0083] Images containing color-similar interfering objects are collected. Within the HSV color space, statistical analysis is performed to determine the core feature range of "low saturation" of the old marker lines due to fading, thus forming an interfering object model.

[0084] For clustering For each cluster, calculate the average saturation of the pixels in the HSV color space. Average tone Statistical characteristics;

[0085] The features of each cluster are compared with the interference model. If the average saturation and average hue of a cluster fall within the core feature range of the interference model, then the cluster is determined to be an "interference cluster".

[0086] All clusters identified as interference are removed from the clustering results. Among the remaining clusters, the cluster with the highest average saturation and linear morphology is identified as the red marker line cluster. All pixels of this cluster are output to generate the initial binary region of the marker line. ;

[0087] Step 5-4, Post-processing optimization: For the binary region of the marker line Morphological filtering is performed to connect possible minor fractures, smooth edges, and eliminate salt-and-pepper noise. Finally, the optimized and accurate binary mask of the red marker line for this bolt instance is output. This is used for subsequent loosening determination analysis, and the process is as follows: Figure 7 As shown.

[0088] Step 6: Perform fragmentation region analysis on the extracted marker lines, and based on the number of fragmentation regions and their geometric distribution characteristics after the marker lines break, establish a quantitative loosening judgment model to determine whether the corresponding bolts have become loose.

[0089] Step 6-1: Create a binary mask for the red marked lines. The fragmentation region N is statistically analyzed within the marked line region;

[0090] Step 6-2: Based on the value of N, use the appropriate strategy to determine the loosening:

[0091] If N=1, it indicates that the bolt has not undergone significant rotational displacement, and the fastener is determined to be not loose.

[0092] If N=2 (e.g.) Figure 8 As shown in the image, this indicates that the bolt has shifted. The degree of shift needs to be determined to determine whether the fastener has become loose.

[0093] First, ellipse fitting is performed on the two marked line areas, and straight lines are fitted with the major axes of the two ellipses as the reference. The angle difference between the two straight lines is calculated. If the angle difference is greater than the threshold, the bolt is determined to be loose. In this embodiment, the angle difference threshold is set to 15°. In actual engineering applications, it can be adjusted within the range of 10° to 30° according to the bolt specifications and detection accuracy requirements.

[0094] If N≥3 (e.g.) Figure 9 As shown in the figure, this indicates that the bolt may have undergone significant displacement. It is necessary to analyze the overall spatial distribution characteristics of all fragment regions and assess their degree of linear preservation to determine the bolt's condition. Specifically:

[0095] In a tightened state, all marked line fragments should maintain a linear arrangement; in a loose state, the fragment distribution will exhibit a scattered characteristic. Therefore, the contour points of all connected components are first extracted to construct a total point set. ,in Total points;

[0096] Compute point set covariance matrix And solve for the eigenvalues:

[0097]

[0098] in, , , For the first The coordinates of the points The mean center of the coordinates of all contour points;

[0099] Solve eigenvalues :

[0100]

[0101] in, yes The identity matrix, ; ;

[0102] Calculate the linearity index:

[0103]

[0104] like If , it means that all points are strictly collinear and perfectly linear; if If , it means that the point set is isotropically distributed and completely scattered; if , then represent different degrees of linear distribution;

[0105] Loosening determination based on linearity index:

[0106]

[0107] in, The threshold value can be determined experimentally; in this embodiment, it is set to 0.8.

[0108] Step 7: Combining the rope condition assessment and the bolt loosening detection results, generate the final fault detection result:

[0109] Level 1 alarm: An emergency alarm is generated when the rope breaks or falls off and there are loose bolts, indicating that there is an immediate risk to the system and that the machine must be stopped immediately for maintenance.

[0110] Level 2 alarm: When the rope is slack or potentially broken and there are loose bolts, a Level 2 alarm is generated, indicating that the system has medium to high risk and that inspection and repair should be arranged as soon as possible.

[0111] Level 3 alarm: When the rope is in any of the abnormal states of slack, potential breakage, breakage or detachment, but all bolts are not loose, a Level 3 alarm is generated, indicating that there is an abnormality in the rope system and that planned maintenance is required.

[0112] No alarm: When the rope is in normal condition and all bolts are not loose, the system will not alarm and will output a normal system status.

[0113] This invention achieves automated, high-precision, and intelligent detection of the health status of vehicle under-body bolt groups by deeply integrating low-light enhancement, improved semantic segmentation, anti-interference marker line extraction, and hierarchical decision logic, effectively overcoming many shortcomings of manual inspection and existing vision methods.

[0114] In addition, this solution also provides an automated bolt group fault detection system for low-light environments, including the following modules:

[0115] Data acquisition module: used to acquire raw color images of the train undercarriage containing the target bolt group;

[0116] Data processing module: used to perform low-light enhancement processing on the original image based on a deep learning model to obtain the enhanced image;

[0117] Segmentation module: This module processes the enhanced image into a trained semantic segmentation neural network to obtain a semantic segmentation mask containing the "bolt" region and the "anti-loosening rope" region.

[0118] Detection module: Based on the anti-loosening rope region in the semantic segmentation mask, morphological operations, connected component analysis, and linear geometric structure analysis of the rope are performed to finely determine the specific state of the anti-loosening rope, including normal and abnormal states; when the rope is determined to be normal, the system normal state is directly output and the detection process ends.

[0119] When the rope is determined to be in an abnormal state, for each individual bolt instance, its corresponding image region is extracted from the enhanced image. Then, in the HSV color space, an adaptive clustering algorithm guided by the clustering effectiveness index is used, combined with prior knowledge of the characteristics of interference objects to filter out the marker line extraction method. This effectively separates and filters out color-similar interference objects, thereby accurately extracting the red marker line on the bolt used for loosening determination.

[0120] The extracted marking lines are analyzed for fragmented regions. Based on the number of fragmented regions and their geometric distribution characteristics after the marking lines break, a quantitative loosening judgment model is established to determine whether the corresponding bolts have become loose.

[0121] The final fault detection result is generated by combining the rope condition determination and the bolt loosening detection results.

[0122] The embodiments described above are merely one implementation method of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A low-light environment-oriented automated bolt set failure detection method, characterized by, Includes the following steps: Step 1: Acquire the original color image of the train undercarriage containing the target bolt group; Step 2: Perform low-light enhancement processing on the original color image based on a deep learning model to obtain the enhanced image; Step 3: Input the enhanced image into the trained semantic segmentation neural network for processing to obtain a semantic segmentation mask containing the "bolt" region and the "anti-loosening rope" region; Step 4: Based on the anti-loosening rope region in the semantic segmentation mask, perform morphological operations, connected component analysis, and rope linear geometry analysis to refine the specific state of the anti-loosening rope, including normal and abnormal states. When the rope is determined to be normal, the system will output a normal status and the testing process will end. Step 4-1: Semantic segmentation mask for the Wire region of the anti-loosening rope Morphological filtering is performed to remove isolated noise and fine burrs generated during segmentation; Step 4-2: Perform connected component analysis on the morphologically processed binary image and count the number of connected regions in the image. ; Step 4-3: Count the number of connected components obtained from the statistics. Compared with the pre-stored original number of intact ropes Compare: If If so, it is determined that the rope has come loose; if If so, the rope is determined to have broken; if Then proceed to step 4-4 to perform linear geometric structure analysis; Step 4-4: Perform linear geometric structure analysis on the rope region with a normal number of connected components, and determine its state: Step 4-4-1: Extract the skeleton from the binary image of the rope region to obtain the centerline representation of the rope; Step 4-4-2: Perform least-squares line fitting on the rope skeleton point set to obtain the fitted line. Let the point set of the rope skeleton be... ,in For the first The coordinates of the skeleton points The total number of skeleton points; calculate the distance from each skeleton point to the fitted line. vertical distance The formula is as follows: ； Calculate the root mean square error (RMSE) from each skeleton point to the fitted line, which serves as an indicator of straightness deviation. ； Step 4-4-3: Calculate the curvature change along the rope skeleton and detect if there are any abrupt curvature changes. For skeleton points... Its curvature change The calculation formula is: ； in, and The first derivative, and The second derivative is obtained approximately by the difference between adjacent points; if there exists a point with curvature... Exceeding the preset curvature threshold If the curvature changes drastically before and after this point, it is determined that there is a significant curvature abrupt change. Step 4-4-4: Divide the rope frame into even segments, let it be divided into... Segment, calculate the principal direction angle of each segment. , ; Calculate the angle difference between adjacent segments If the sum of the angle differences of multiple consecutive adjacent segments exceeds the preset angle threshold If so, then the direction consistency is poor; Step 4-4-5: Determine the status based on the comprehensive linear geometric structure analysis results: If the straightness deviation index is less than the preset first threshold, there is no significant curvature change, and the direction consistency is good, then the rope is judged to be normal; if the straightness deviation index is greater than or equal to the preset threshold but less than the preset second threshold, and there is no significant curvature change, then the rope is judged to be slack; if the straightness deviation index is greater than or equal to the preset second threshold, or there is a significant curvature change, or the direction consistency is poor, then the rope is judged to have a potential risk of breakage. Steps 4-5: Perform corresponding processing based on the judgment results: When the rope is judged to be normal, directly output the normal status of the system and end the detection process; when the rope is judged to be in any abnormal state, proceed to step 5. Step 5: When the rope is determined to be in an abnormal state, for each independent bolt instance, the corresponding image region is extracted from the enhanced image. Then, in the HSV color space, an adaptive clustering algorithm based on the clustering effectiveness index is used, combined with prior knowledge of the interference features to filter out the marker line extraction method. This effectively separates and filters out color-similar interference, thereby accurately extracting the red marker line on the bolt used for loosening determination. Step 6: Perform fragment region analysis on the extracted marking lines, and establish a quantitative loosening judgment model based on the number of fragment regions and their geometric distribution characteristics after the marking lines break, to determine whether the corresponding bolts have become loose. Step 7: Combine the determined rope condition with the determined bolt loosening detection results to generate the final fault detection result.

2. The automated bolt group fault detection method for low-light environments according to claim 1, characterized in that, The low-light enhancement processing of the original color image based on a deep learning model specifically includes: Step 2-1: Image decomposition. The acquired low-light original image and its corresponding normal exposure reference image are paired and decomposed in the KinD++ low-light image enhancement network. Each image is decoupled into the reflection component, which represents the inherent properties of the object, and the illumination component, which controls the brightness distribution of the scene. The reflection loss function is used to constrain the reflection components of the low-light image and the normal light image to tend to be consistent. Step 2-2: Reflection component restoration. The reflection component of the decomposed low-light image, together with its illumination component, is input into the reflection restoration network in the KinD++ low-light image enhancement network. The reflection restoration network adopts a fully convolutional structure that integrates multi-scale illumination attention modules to suppress noise, correct color distortion, and avoid halo artifacts caused by over-pooling operations, thereby restoring clear and clean reflection information. Steps 2-3: Illumination component adjustment: Input the low-light image illumination components obtained from the decomposition into the illumination adjustment network in the KinD++ low-light image enhancement network; The illumination adjustment network accepts an adjustable brightness coefficient as input to achieve a flexible and controllable nonlinear mapping from low illumination conditions to desired illumination conditions. Steps 2-4: Image fusion and output: The repaired reflection component and the adjusted illumination component are fused together again to generate an enhanced image with significantly improved visual quality.

3. The automated bolt group fault detection method for low-light environments according to claim 1, characterized in that, The semantic segmentation neural network is an improved model based on the PIDNet architecture. On the basis of the original PIDNet structure, the network embeds the channel attention module ECA-Net at the end of the detail branch to enhance the response to high-frequency detail features such as rope edges. In the semantic branches, deformable convolutional modules are used to replace standard convolutions to adapt to the geometric deformation of the target. Enhance the image The input is fed into a pre-trained improved PIDNet semantic segmentation neural network, which outputs a category label for each pixel, generating a semantic segmentation mask containing "Bolt", "Wire", and "Background". .

4. The automated bolt group fault detection method for low-light environments according to claim 1, characterized in that, The extraction of the red marking line on the bolt used for loosening determination in step 5 specifically involves: Step 5-1, instance separation, based on the... The mask area of ​​each bolt From enhanced images Crop the corresponding colored sub-image from the middle. ; Step 5-2: Color space conversion and initial screening. Convert the color sub-image from RGB space to HSV space. Based on the distribution characteristics of the red marker line in the hue channel, set a threshold for initial screening to obtain a set of candidate pixels containing the red marker line and similar color interference. ; Step 5-3: Employ an adaptive clustering algorithm guided by a clustering effectiveness metric from candidate pixels. The target marker lines were precisely separated, including: Step 5-3-1: Set the number of clusters The search scope; Step 5-3-2: For each candidate within the range The value is used to perform K-means clustering, and then the CH index of the clustering result is calculated; Step 5-3-3: Select the option that maximizes the CH index. The value is used as the final cluster number. The corresponding clustering results were then used for subsequent analysis. Step 5-3-4: After obtaining the clustering results, each cluster is judged and filtered out based on prior domain knowledge rules to accurately separate the red-marked lines, including: Images containing color-similar interfering objects are collected. Within the HSV color space, statistical analysis is performed to determine the core feature range of "low saturation" of the old marker lines due to fading, thus forming an interfering object model. For clustering For each cluster, calculate the average saturation of the pixels in the HSV color space. Average tone Statistical characteristics; The features of each cluster are compared with the interference model. If the average saturation and average hue of a cluster fall within the core feature range of the interference model, then the cluster is determined to be an "interference cluster". All clusters identified as interference are removed from the clustering results. Among the remaining clusters, the cluster with the highest average saturation and linear morphology is identified as the red marker line cluster. All pixels of this cluster are output to generate the initial binary region of the marker line. ; Step 5-4, Post-processing optimization: For the binary region of the marker line Morphological filtering is performed to connect possible minor fractures, smooth edges, and eliminate salt-and-pepper noise. Finally, the optimized and accurate binary mask of the red marker line for this bolt instance is output. This is used for subsequent loosening determination analysis.

5. The automated bolt group fault detection method for low-light environments according to claim 4, characterized in that, The determination of whether the corresponding bolt is loose in step 6 is as follows: Step 6-1: Create a binary mask for the red marked lines. The fragmentation region N is statistically analyzed within the marked line region; Step 6-2: Based on the value of N, use the appropriate strategy to determine the loosening: If N=1, it indicates that the bolt has not undergone significant rotational displacement, and the fastener is determined to be not loose. If N=2, it indicates that the bolt has shifted. The degree of shift needs to be determined to determine whether the fastener has become loose. First, ellipse fitting is performed on the two marked line areas, and straight lines are fitted with the major axes of the two ellipses as the reference. The angle difference between the two straight lines is calculated. If the angle difference is greater than the threshold, the bolt is determined to be loose. If N≥3, it indicates that the bolt may have undergone significant displacement. It is necessary to analyze the overall spatial distribution characteristics of all fragment regions and assess its degree of linear preservation in order to determine the condition of the bolt.

6. The automated bolt group fault detection method for low-light environments according to claim 5, characterized in that, The analysis of the overall spatial distribution characteristics of all fragment regions and the assessment of their linear preservation degree are used to determine the condition of the bolts. Specifically: In a tightened state, all marked line fragments should maintain a linear arrangement; in a loose state, the fragment distribution will exhibit a scattered characteristic. Therefore, the contour points of all connected components are first extracted to construct a total point set. ,in Total points; Compute point set covariance matrix And solve for the eigenvalues: ； in, , , For the first The coordinates of the points The mean center of the coordinates of all contour points; Solve eigenvalues : ； in, yes The identity matrix, ; ; Calculate the linearity index: ； like If , it means that all points are strictly collinear and perfectly linear; if If , it means that the point set is isotropically distributed and completely scattered; if , then represent different degrees of linear distribution; Loosening determination based on linearity index: ； in, The threshold value is set.

7. The automated bolt group fault detection method for low-light environments according to claim 5, characterized in that, The generation of the final fault detection result in step 7 is specifically as follows: If the rope breaks or falls off, and there are loose bolts, an emergency alarm will be generated to indicate that there is an immediate risk to the system and that the machine must be stopped immediately for maintenance. When the rope is slack or potentially broken and there are loose bolts, a second-level alarm is generated, indicating that the system has a medium to high risk and that inspection and repair should be arranged as soon as possible. When the rope is in any of the abnormal states of slack, potential breakage, breakage, or detachment, but all bolts are not loose, a level 3 alarm is generated, indicating that there is an abnormality in the rope system and that planned maintenance is required. No alarm: When the rope is in normal condition and all bolts are not loose, the system will not alarm and will output a normal system status.

8. An automated bolt group fault detection system for low-light environments, used to perform the method of claim 1, characterized in that, Includes the following modules: Data acquisition module: used to acquire raw color images of the train undercarriage containing the target bolt group; Data processing module: used to perform low-light enhancement processing on the original color image based on a deep learning model to obtain the enhanced image; Segmentation module: This module processes the enhanced image into a trained semantic segmentation neural network to obtain a semantic segmentation mask containing the "bolt" region and the "anti-loosening rope" region. Detection module: Based on the anti-loosening rope region in the semantic segmentation mask, morphological operations, connected component analysis, and linear geometric structure analysis of the rope are performed to finely determine the specific state of the anti-loosening rope, including normal and abnormal states; When the rope is determined to be normal, the system will output a normal status and the testing process will end. When the rope is determined to be in an abnormal state, for each individual bolt instance, its corresponding image region is extracted from the enhanced image. Then, in the HSV color space, an adaptive clustering algorithm guided by the clustering effectiveness index is used, combined with prior knowledge of the characteristics of interference objects to filter out the marker line extraction method. This effectively separates and filters out color-similar interference objects, thereby accurately extracting the red marker line on the bolt used for loosening determination. The extracted marking lines are analyzed for fragmented regions. Based on the number of fragmented regions and their geometric distribution characteristics after the marking lines break, a quantitative loosening judgment model is established to determine whether the corresponding bolts have become loose. The final fault detection result is generated by combining the rope condition determination and the bolt loosening detection results.

Images