Methods, media, and equipment for detecting failure modes of critical components on the surface of railway trains

Abstract

This invention discloses a method, medium, and equipment for detecting failure modes of important components on the surface of railway trains, belonging to the field of intelligent detection engineering technology. The method for detecting failure modes of important components on the surface of railway trains mainly includes: obtaining a training dataset based on images of the outer surface of the railway locomotive; constructing a component segmentation model, and training the component segmentation model using the training dataset to obtain a trained component segmentation model; using the trained component segmentation model to detect the image to be tested to obtain a bolt segmentation image, performing polygon fitting, angle calculation, and detection to obtain the detection result. Implementing the method, medium, and equipment for detecting failure modes of important components on the surface of railway trains provided by this invention can improve the accuracy and reliability of detecting failure modes of important components on the train surface.

Description

Technical Field

[0001] This invention relates to the field of intelligent inspection engineering technology, and more specifically, to a method, medium, and equipment for detecting failure modes of important components on the surface of railway trains. Background Technology

[0002] China's railway network is projected to reach 175,000 kilometers by 2025, making safety maintenance increasingly important. Trains have complex structures, containing numerous small but critical components such as bolts. Currently, methods for bolt failure detection mainly fall into three categories: first, manual visual inspection, which is inefficient and prone to missed or false detections; second, non-destructive testing techniques, such as ultrasonic testing and vibration signal analysis, but these methods are primarily used to identify internal bolt faults and loosening, resulting in low detection efficiency and difficulty in rapidly detecting bolt loosening under actual working conditions; and third, machine vision technology based on deep learning. In recent years, researchers have used object detection networks to coarsely locate bolt positions, semantic segmentation networks to extract anti-loosening marking lines, and Hough transform to obtain the offset angle of the marking lines to determine the bolt condition. However, this method is highly dependent on the clarity and accuracy of the anti-loosening marking lines, which are usually manually painted, resulting in inconsistent shapes, low image contrast, and interference from background noise, leading to errors in offset angle extraction and affecting detection accuracy. Summary of the Invention

[0003] The purpose of this invention is to provide a method, medium, and equipment for detecting the failure modes of important components on the surface of railway trains, which can improve the accuracy and reliability of detecting the failure modes of important components on the train surface.

[0004] This invention provides a method for detecting failure modes of important components on the surface of a train, comprising the following steps: S1: acquiring an image of the outer surface of a railcar, and obtaining a training dataset based on the image; S2: constructing a component segmentation model, the component segmentation model including a combined backbone network and a comprehensive loss function; S3: training the component segmentation model using the training dataset to obtain a trained component segmentation model; S4: performing detection using the trained component segmentation model based on the image to be tested, obtaining a bolt segmentation image; S5: performing polygon fitting, angle calculation, and detection based on the bolt segmentation image to obtain a detection result.

[0005] Further, step S1 specifically includes: S11: acquiring an image of the outer surface of the railcar, preprocessing the image of the outer surface of the railcar to obtain bolt area images, bolt numbers, bolt coordinates, and initial rotation angles; S12: performing annotation and data augmentation based on the bolt area images, bolt numbers, bolt coordinates, and initial rotation angles to obtain a training dataset.

[0006] Furthermore, the aforementioned combined backbone network includes a shared weight encoder, a semantic segmentation decoder, a super-resolution decoder, and a feature guidance module. The shared weight encoder is used to extract image features. The semantic segmentation decoder is used to obtain a first semantic segmentation feature based on the image features. The super-resolution decoder is used to obtain a first super-resolution feature based on the image features. The feature guidance module is used to minimize the distance between the task target and the corresponding task result generated by the feature guidance module based on the first semantic segmentation feature and the first super-resolution feature, guide the super-resolution feature and semantic segmentation feature to focus on the region of interest, and obtain a second semantic segmentation feature and a second super-resolution feature, thereby improving the semantic segmentation effect.

[0007] Furthermore, the aforementioned feature guidance module includes a splicing layer, a first... Convolutional layers, residual channel attention modules, multi-scale convolutional modules, and second convolutional layers. Convolutional layer, third Convolutional layer, first dot product operation, second dot product operation, fourth dot product operation Convolutional layer, fifth Convolutional layer, first addition operation, second addition operation; the concatenation layer is used to concatenate the first semantic segmentation feature and the first super-resolution feature in the channel dimension to obtain the concatenation result; the first Convolutional layers are used to determine the first Feature fusion is performed on the convolutional layer concatenation results; the residual channel attention module uses two 3D arrays. A depthwise convolution of 3 is used to capture local patterns in the integrated features. Two linear layers are then used to directly compress and expand the features to capture channel relationships. A sigmoid function is then used to generate weights along the channel axis based on the aforementioned features. The local patterns are reweighted using a Hadamard product, and finally, the input features are added to the reweighted features. The multi-scale convolution module groups the features processed by the residual channel attention module into groups based on channel size, with the first group using... Convolution maintains the current scale; image pooling is used for the last group to collect global spatial information, while different dilation rates are used for the other groups. Depth convolution is used to capture information at different levels, where the dilation rate is set to the group index minus 1. Features from each group are concatenated along the channel dimension, and then processed using batch normalization and the GELU activation function. The second... The convolutional layer is used to map the output of the residual channel attention module to the space of the first semantic segmentation feature to generate semantic segmentation task attention weights; the third The convolutional layer is used to map the output of the residual channel attention module to the space of the first super-resolution feature to generate super-resolution task attention weights; the first dot product operation is used to multiply the semantic segmentation task attention weights with the first semantic segmentation feature to obtain semantic segmentation weighted features; the second dot product operation is used to multiply the super-resolution task attention weights with the first super-resolution feature to obtain super-resolution weighted features; the fourth The convolutional layer is used to convolve the semantic segmentation weighted features; the fifth... The convolutional layer is used to convolve the weighted features of the super-resolution; the first addition operation layer is used to combine the fourth... The output of the convolutional layer is added to the first semantic segmentation feature to obtain the second semantic segmentation feature; the second addition layer is used to add the fifth semantic segmentation feature. The output of the convolutional layer is added to the first super-resolution feature to obtain the second super-resolution feature.

[0008] Furthermore, the aforementioned combined backbone network also includes a semantic segmentation head and a super-resolution head; the semantic segmentation head is used to obtain semantic segmentation results based on semantic segmentation features; the super-resolution head is used to obtain super-resolution results based on semantic segmentation features.

[0009] Furthermore, the above-mentioned comprehensive loss function is as follows:

[0010] ,

[0011] in, For the comprehensive loss function, For semantic segmentation loss, To enhance semantic segmentation loss, To enhance super-resolution loss, For super-resolution loss, To divide the loss weights for boundary constraints, The loss is segmented for boundary constraints.

[0012] Further, step S4 specifically includes: S41: preprocessing the image to be tested to obtain the image of the bolt region to be tested, the bolt number to be tested, the coordinates of the bolt to be tested, and the initial rotation angle of the bolt to be tested; S42: using the trained component segmentation model to detect the image of the bolt region to be tested to obtain a bolt segmentation image.

[0013] Further, step S5 specifically includes: S51: Based on the bolt segmentation image, perform contour extraction and polygon fitting to obtain a regular hexagon; S52: Based on the regular hexagon, calculate the angle to obtain the bolt rotation angle; S53: Based on the bolt rotation angle, the initial rotation angle of the bolt to be tested, and the preset angle threshold, obtain the detection result.

[0014] The present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the above-described method for detecting failure modes of important components on the surface of rail trains.

[0015] The present invention also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the above-described method for detecting failure modes of important components on the surface of a railcar.

[0016] The method, medium, and equipment for detecting failure modes of important components on the surface of railway trains provided by this invention have the following beneficial effects:

[0017] (1) This invention constructs and trains a dual super-resolution semantic segmentation boundary constraint network DSCBNet, which provides an innovative method for semantic segmentation tasks, significantly improves the segmentation effect of low-resolution images, and pays better attention to the edge details of the target while ensuring accuracy. Traditional segmentation methods often suffer from edge blurring and inaccurate segmentation due to loss of details when processing low-resolution images. The DSCBNet network of this invention effectively solves this problem through the super-resolution and semantic segmentation feature fusion module and boundary constraint loss.

[0018] (2) The present invention determines whether a bolt is loose by the deflection angle of the hexagonal contour of the bolt, providing a more accurate and stable loosening detection method than existing methods. Traditional bolt loosening detection methods usually rely on the offset angle of the marking line for judgment, but the shape of the manually painted marking line is inconsistent and affected by various factors such as contaminants and surface wear, which can easily lead to inaccurate detection results. The present invention utilizes the geometric features of the bolt itself - such as the inherent contour of the hexagon, which avoids the problem of the marking line being difficult to detect accurately in complex environments, has stronger anti-interference ability, and significantly improves the reliability of loosening detection. Attached Figure Description

[0019] The present invention will be further described below with reference to the accompanying drawings and embodiments. In the accompanying drawings:

[0020] Figure 1 This is a flowchart of the method for detecting failure modes of important components on the train surface provided by the present invention;

[0021] Figure 2 This is a schematic diagram of the DSCBNet network backbone structure provided by the present invention;

[0022] Figure 3 This is a schematic diagram of the SFGM structure provided by the present invention;

[0023] Figure 4This is a schematic diagram of the MRCAB structure provided by the present invention;

[0024] Figure 5 This is a schematic diagram of Lcb loss calculation provided by the present invention;

[0025] Figure 6 This is an example diagram of annotation provided by the present invention;

[0026] Figure 7 This is an example diagram of the detection results provided by the present invention;

[0027] Figure 8 This is a schematic diagram for calculating the loosening angle of a hexagonal bolt provided by the present invention;

[0028] Figure 9 This is a structural block diagram of the computer device provided by the present invention. Detailed Implementation

[0029] To provide a clearer understanding of the technical features, objectives, and effects of the present invention, specific embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0030] Figure 1 A schematic diagram of the failure mode detection method for important components on the train surface according to this embodiment is shown. In this embodiment, the failure detection method for important components on the track train surface includes the following steps:

[0031] S1: Obtain an image of the outer surface of the rail locomotive, and obtain a training dataset based on the image of the outer surface of the rail locomotive;

[0032] In one exemplary embodiment, step S1 specifically includes:

[0033] S11: Acquire an image of the outer surface of the railcar, preprocess the image of the outer surface of the railcar to obtain bolt area image, bolt number, bolt coordinates, and initial rotation angle;

[0034] In one exemplary embodiment, in step S11, images of the outer surface of the railcar are acquired using a hardware image acquisition platform consisting of a high-resolution line array camera and a light source fixed on both sides of the track. The host computer software for the camera and light source adjusts appropriate parameters such as exposure time, acquisition frequency, and light intensity to obtain images of the railcar's outer surface suitable for subsequent image processing, which are then stored in a computer. Distortion correction is performed on the railcar's outer surface images. Based on known positional information and prior conditions, images of each bolt area are cropped from the entire train. Each bolt is numbered, and its coordinate information and initial rotation angle are recorded. The cropped bolt area image is preprocessed to improve image quality and enhance the recognizability of bolt features. The image contrast and brightness are adjusted, the image edge is enhanced by high-pass filtering, gamma correction is applied to optimize the uneven brightness distribution of the image, and finally bilateral filtering is used to process the image noise, preserving the edge details of the bolt while reducing noise.

[0035] S12: Based on the bolt area image, bolt number, bolt coordinates, and initial rotation angle, perform annotation and data augmentation to obtain the training dataset;

[0036] In one exemplary embodiment, in step S12, the bolt images after image preprocessing are labeled using labelme to obtain bolt labels; subsequently, data augmentation processing is performed on these bolt image and label data pairs to increase data diversity; the same geometric transformations, such as scaling, displacement, rotation, contrast transformation, etc., are performed on the bolt image and label data pairs; these methods can change the position information, color, size, etc. of features in the image, which can increase the diversity of defect images to a certain extent;

[0037] S2: Construct a component segmentation model, which includes a combined backbone network and a comprehensive loss function;

[0038] In one exemplary embodiment, the combined backbone network includes a shared weight encoder, a semantic segmentation decoder, a super-resolution decoder, and a feature guidance module. The shared weight encoder is used to extract image features. The semantic segmentation decoder is used to obtain a first semantic segmentation feature based on the image features. The super-resolution decoder is used to obtain a first super-resolution feature based on the image features. The feature guidance module is used to minimize the distance between the task target and the corresponding task result generated by the feature guidance module based on the first semantic segmentation feature and the first super-resolution feature, and guide the super-resolution feature and semantic segmentation feature to focus on the region of interest to obtain a second semantic segmentation feature and a second super-resolution feature, thereby improving the semantic segmentation effect.

[0039] In one exemplary embodiment, the feature guidance module includes a splicing layer and a first... Convolutional layers, residual channel attention modules, multi-scale convolutional modules, and second convolutional layers. Convolutional layer, third Convolutional layer, first dot product operation, second dot product operation, fourth dot product operation Convolutional layer, fifth Convolutional layer, first addition operation, second addition operation; the concatenation layer is used to concatenate the first semantic segmentation feature and the first super-resolution feature in the channel dimension to obtain the concatenation result; the first Convolutional layers are used to determine the first Feature fusion is performed on the convolutional layer concatenation results;

[0040] The residual channel attention module uses two 3 A depthwise convolution of 3 is used to capture local patterns in the integrated features. Then, two linear layers are used to directly compress and expand the features to capture channel relationships. The Sigmoid function is then used to generate weights along the channel axis based on the above features. The local patterns are reweighted through the Hadamard product. Finally, the input features are added to the reweighted features.

[0041] The multi-scale convolution module groups the features processed by the residual channel attention module into average groups according to channel size, where the first group uses... Convolution maintains the current scale; image pooling is used for the last group to collect global spatial information, while different dilation rates are used for the other groups. Deep convolution is used to capture information at different levels, where the dilation rate is set to the group index minus 1. The features of each group are concatenated along the channel dimension, and then processed by batch normalization and the GELU activation function.

[0042] The second The convolutional layer is used to map the output of the residual channel attention module to the space of the first semantic segmentation feature to generate semantic segmentation task attention weights; the third The convolutional layer is used to map the output of the residual channel attention module to the space of the first super-resolution feature to generate super-resolution task attention weights; the first dot product operation is used to multiply the semantic segmentation task attention weights with the first semantic segmentation feature to obtain semantic segmentation weighted features; the second dot product operation is used to multiply the super-resolution task attention weights with the first super-resolution feature to obtain super-resolution weighted features; the fourth The convolutional layer is used to convolve the semantic segmentation weighted features; the fifth... The convolutional layer is used to convolve the weighted features of the super-resolution; the first addition operation layer is used to combine the fourth... The output of the convolutional layer is added to the first semantic segmentation feature to obtain the second semantic segmentation feature; the second addition layer is used to add the fifth semantic segmentation feature. The output of the convolutional layer is added to the first super-resolution feature to obtain the second super-resolution feature;

[0043] In one exemplary embodiment, the combined backbone network further includes a semantic segmentation head and a super-resolution head; the semantic segmentation head is used to obtain a semantic segmentation result based on semantic segmentation features; the super-resolution head is used to obtain a super-resolution result based on semantic segmentation features.

[0044] In one exemplary embodiment, a combined backbone network DSCBNet of the above-mentioned component segmentation model is constructed to obtain bolt segmentation images with clear edges. The main structure of the DSCBNet network consists of a shared weight encoder, a semantic segmentation decoder, a super-resolution decoder, and a feature guidance module. The shared encoder, semantic segmentation decoder, and super-resolution decoder can all be flexibly replaced with currently mainstream encoding and decoding structures. The image, labels, and a super-resolution image with a size u times that of the original image are used as inputs to the DSCBNet network. The image is processed by the shared weight encoder to extract features, and the semantic segmentation decoder and super-resolution decoder process to obtain the first semantic segmentation feature Fseg and the first super-resolution feature Fsr. These are then processed by the semantic segmentation head and the super-resolution head to obtain the segmentation result Oseg and the super-resolution result Osr. The semantic segmentation head is a single-layer 1... A convolutional layer of size 1 adjusts the number of channels to n classes to obtain the segmentation result; the super-resolution header consists of 3 convolutional layers and 2 Tanh activation functions, and finally uses pixel shuffle to upsample to the target resolution to obtain the super-resolution result; such as Figure 2 The diagram shows the backbone structure of the DSCBNet network. It should be noted that when high-resolution images are difficult to obtain, the image and label can be downsampled by a factor of u. The downsampled image, along with the label and the original image, is then fed into the network for training. The features Fseg and Fsr obtained after processing by the semantic segmentation decoder and super-resolution decoder are fed into the Shared Feature Guidance Module (SFGM). This minimizes the distance between the task objective and the corresponding task result generated by the SFGM, guiding the super-resolution features and semantic segmentation features to focus on the Region of Interest (ROI), thus improving the semantic segmentation performance. The SFGM module structure is as follows: Figure 3 As shown; Fseg and Fsr are concatenated along the channel dimension, using a 1 1-level convolutions are used to fuse features, and residual channel attention module MRCAB and multi-scale convolution module MS-Conv are used to integrate features from two tasks; two 1-level convolutions are used. The 1-conv mapping of the MS-Conv output to the spaces of segmentation decoding features Fseg and super-resolution decoding features Fsr generates attention weights for the two tasks. These weights are then multiplied by Fseg and Fsr respectively to obtain weighted features, which are then processed by a 1-conv mapping. 1. Convolution and summation of the original features yields a second semantic segmentation feature Ffss and a second super-resolution feature Ffsr that are more focused on the ROI. The semantic segmentation feature Ffss and super-resolution feature Ffsr obtained after processing by the feature guidance module SFGM are then processed by the semantic segmentation head and super-resolution head to obtain the segmentation result Ofseg and the super-resolution result Ofsr. The semantic segmentation head is a single-layer 1-dimensional... One convolutional layer is used, and the number of channels is adjusted to n classes to obtain the segmentation result; the super-resolution head consists of three convolutional layers and two Tanh activation functions, and finally pixel shuffle is used to upsample to the target resolution to obtain the super-resolution result; the MRCAB module structure is as follows. Figure 4 As shown; using two 3 A depthwise convolution of 3 is used to capture local patterns in the integrated features. Two linear layers are then used to directly compress and expand the features to capture channel relationships. A sigmoid function is then used to generate weights along the channel axis based on these features. The local patterns are reweighted using a Hadamard product. Finally, the input features are added to the reweighted features. The multi-scale convolution module MS-Conv divides the features processed by the MRCAB module into k groups according to channel size, and different processing methods are used for different groups. The first group uses a depthwise convolution of 1. 1. Convolution maintains the current scale. For the second to (k-1)th groups, 3×3 depth convolutions with different dilation rates are used to capture information at different levels. The dilation rate is set to the index of the group minus 1. For the kth group, image pooling is used to collect global spatial information. The features of each group are concatenated in the channel dimension and then processed by batch normalization and GELU activation function.

[0045] In one exemplary embodiment, the comprehensive loss function is as follows:

[0046] ,

[0047] in, For the comprehensive loss function, For semantic segmentation loss, To enhance semantic segmentation loss, To enhance super-resolution loss, For super-resolution loss, To divide the loss weights for boundary constraints, The loss is defined by boundary constraints.

[0048] In one exemplary embodiment, to constrain the model to learn more stable and clear boundaries, this embodiment combines deep supervision to construct a comprehensive loss function in the network and optimize model training; through semantic segmentation loss ( Super-resolution loss ), Enhanced semantic segmentation loss ( ), Enhanced super-resolution loss ( ) and boundary constraint segmentation loss ( The weighted mixed loss function (Loss) is used to optimize the parameters of the DSCBNet network and obtain the trained DSCBNet network model; the comprehensive loss function of DSCBNet is shown in equation (1):

[0049] (1)

[0050] In the formula, The cross-entropy loss function is used to measure this. The mean squared error loss function is used to measure this. The Structural Similarity (SSIM) loss function is used to measure this. The loss function is used to measure the regional mutual information (RMI). In this embodiment, the value of the boundary constraint segmentation loss weight γ is chosen to be 2.

[0051] In one exemplary embodiment, the boundary constraint segmentation loss ( )Depend on , , Composition, such as Figure 5 As shown Calculation diagram; First, the semantic segmentation result image Ofseg and label image obtained by the SFGM module are subjected to morphological processing, and erosion operation is performed on both. The structural element of the erosion operation is a circle with a radius of d. After morphological processing, the difference is made with the original image to obtain the semantic segmentation result and the boundary map of the label; The feature map Fseg, semantic segmentation result and label boundary map obtained by the semantic segmentation decoder are processed simultaneously according to the Conditional Correctness Aware Sampling (CCASS) rule to obtain positive and negative samples of the anchor point, and a local class average feature is generated for each anchor point; Conditional Correctness Aware Sampling (CCASS) conforms to the following four rules: (1) Only select K in the neighborhood of the anchor point K samples; (2) the selected samples must be correctly classified by the model; (3) positive sample points are pixels of the same class as the anchor point; (4) negative sample points are pixels of a different class than the anchor point; the CCASS strategy uses a K Hollow convolution of K extracts anchor point K The number of positive samples in the K-domain, for example, a 3 The extractor weights for 3 are initialized as follows:

[0052]

[0053] This hollow quantity extractor convolutional layer performs a convolution operation on each channel of the one-hot label to obtain the positive sample quantity map; positive sample quantity The calculation formula is shown in Equation 1:

[0054] (1)

[0055] In the formula, Indicates the predicted label, One-hot tags representing real tags, This represents element-wise multiplication. Indicates the convolution operation;

[0056] The class average feature refers to the feature for each anchor point Define its local class average The embedding is the average feature of its positive samples, as shown in Equation 2:

[0057] (2)

[0058] In the formula, Represents the set of positive samples The number of samples; similar to the method described above for obtaining the number of positive samples, using K... K's hollow convolution obtains the class average feature map. The calculation formulas are shown in equations 3 and 4:

[0059] (3)

[0060] (4)

[0061] In the formula, Fseg represents the feature map obtained by the semantic segmentation decoder. Used to avoid division by zero;

[0062] Calculate the cross-entropy loss based on the obtained average feature map of the anchor point class and the true label. This is used to ensure that the local category center can correctly reflect the category features, as shown in equation (5):

[0063] (5)

[0064] In the formula, This represents the total number of anchor points. This represents the predicted local class center label. Indicates the true label;

[0065] Based on the pairwise similarity between the supervised boundary pixels and their positive and negative samples, the distance between the boundary pixels and the positive samples is reduced, and the distance between the boundary pixels and the negative samples is reduced, as shown in Equation (6):

[0066] (6)

[0067] In the formula, and These represent the number of positive and negative samples for the boundary pixels, respectively. This is expressed as the cosine similarity between the anchor point features and the features of the positive and negative sample points. Indicates true cosine similarity;

[0068] Based on the pairwise similarity used to supervise each boundary pixel with its local class center feature, i.e., the distance between the class average feature and the boundary pixel point feature, the intra-class consistency of the boundary pixels is increased, as shown in Equation (7):

[0069] (7)

[0070] In the formula, Represented as the first Features of each boundary pixel Represented as the first Local average features of each boundary pixel, This indicates that only if the anchor pixel K This term is calculated only if there is at least one positive sample in the K-domain.

[0071] In summary, the overall boundary constraint segmentation loss function ( As shown in equation (8):

[0072] (8)

[0073] In this embodiment, the values ​​of weights α and β are chosen to be 0.1 and 0.5, respectively.

[0074] S3: Train the component segmentation model using the training dataset to obtain a trained component segmentation model;

[0075] In one exemplary embodiment, the component segmentation model, i.e., the dual hyper-separation boundary constraint network, is trained using the Poly strategy to obtain a trained component segmentation model; specifically, the Adam optimizer is used with a momentum of 0.9 and a weight decay set to 0.0001, and the parameters are... and The values ​​are 0.9 and 0.99 respectively; using the Poly learning scheme, the learning rate is adjusted to:

[0076] ,

[0077] in and Set the values ​​to 0.9 and 40000 respectively, set the initial learning rate to 0.0005, and adjust the batch size according to the available video memory.

[0078] S4: Based on the image to be tested, the trained component segmentation model is used for detection to obtain the bolt segmentation image;

[0079] In one exemplary embodiment, step S4 specifically includes:

[0080] S41: Preprocess the image to be tested to obtain the image of the bolt area to be tested, the bolt number to be tested, the coordinates of the bolt to be tested, and the initial rotation angle of the bolt to be tested;

[0081] In one exemplary embodiment, in step S41, the acquired and distortion-corrected image to be tested is cropped according to prior conditions of known position information, and each bolt is numbered and its coordinates are recorded in a prescribed order. The initial rotation angle corresponding to the bolt is found based on the number and coordinates. ;

[0082] S42: Use the trained component segmentation model to detect the bolt region image to obtain a bolt segmentation image;

[0083] In one exemplary embodiment, in step S42, the cropped bolt image is input into the DSCBNet network of the trained component segmentation model for detection to obtain a hexagonal bolt segmentation image with clear edges.

[0084] In one exemplary embodiment, this embodiment conducted a train exterior bolt loosening detection experiment based on DSCBNet. The experimental platform was: CPU: AMD Ryzen 77735H with Radeon Graphics, GPU: NVIDIA GeForceRTX 4060, 8GB RAM; operating system: Windows 11.

[0085] In this embodiment, to accurately evaluate the model's performance, miou, BFscore, and precision were selected as evaluation metrics. The mean intersection-union ratio (miou) was 96.53%, precision was 95.97%, and recall was 97.24%. The boundary F1 score (BFscore) was 82.47% for a circular structuring element with a radius of 2 pixels and 91.71% for a radius of 4 pixels. An example annotation is shown below. Figure 6 As shown, an example of the detection results is as follows. Figure 7 As shown;

[0086] S5: Based on the bolt segmentation image, perform polygon fitting, angle calculation and detection to obtain the detection result;

[0087] In an exemplary embodiment, step S5 specifically includes: S51: performing contour extraction and polygon fitting based on the bolt segmentation image to obtain a regular hexagon; S52: performing angle calculation based on the regular hexagon to obtain the bolt rotation angle; S53: obtaining the detection result based on the bolt rotation angle, the initial rotation angle of the bolt to be tested, and a preset angle threshold.

[0088] In one exemplary embodiment, bolt loosening is determined in step S5; specifically, polygon fitting is performed on the obtained hexagonal segmentation result of the bolt, and the hexagonal angle is calculated. This mainly includes: first, extracting the contour of the segmentation result, finding the minimum inscribed circle of the hexagon, and drawing a vertically upward ray with the center O as the point where it intersects the contour. The ray rotates 1° clockwise each time, and the relationship between each ray and the segmented contour points is calculated. and Angle between points and O The distance, based on the included angle and O The distance is used to plot an angle-distance curve, which is then smoothed using a Gaussian filter. The peak value of the smoothed curve is calculated, and the peak points are used as key points for contour segmentation. Straight lines are fitted based on the segmented contours, and the intersections of the fitted lines are connected to generate regular hexagons. Then, the calculation is performed. The angle between the side containing a point and the X-axis is defined as follows: when the angle is less than 90°, the side is considered an angle. , Adjacent to the left is , Adjacent to the right is If the included angle is greater than 90°, then the side is defined as... , Adjacent to the left is , Adjacent to the left is ;calculate , , Calculate the absolute angle of rotation, and then calculate the average of the three angles to obtain the bolt rotation angle. ; and with the initial turning angle Doing the difference yields results ,if Greater than the preset angle threshold If it is loose, it is considered loose; otherwise, it is considered not loose. Specific details are as follows: Figure 8 As shown;

[0089] In one exemplary embodiment, the preset angle threshold is 5. .

[0090] This embodiment provides a computer-readable storage medium storing a computer program thereon. When executed by a processor, the computer program implements the steps of the method for detecting failure modes of important components on the surface of a railcar described above. The storage medium may be a magnetic disk, optical disk, read-only memory (ROM), random access memory (RAM), flash memory, hard disk drive (HDD), or solid-state drive (SSD), etc.; the storage medium may also include combinations of the above types of memory.

[0091] This embodiment provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the steps of the above-described method for detecting the failure modes of important components on the surface of a railcar.

[0092] like Figure 9 As shown, the computer device may include: at least one processor 121, such as a central processing unit (CPU), at least one communication interface 123, memory 124, and at least one communication bus 122. The communication bus 122 is used to enable communication between these components. The communication interface 123 may include a display screen and a keyboard; optionally, the communication interface 123 may also include a standard wired interface or a wireless interface. The memory 124 may be high-speed random access memory (RAM) or non-volatile memory, such as at least one disk storage device. Optionally, the memory 124 may also be at least one storage device located remotely from the aforementioned processor 121. The memory 124 stores application programs, and the processor 121 calls the program code stored in the memory 124 to execute any of the aforementioned method steps. The communication bus 122 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The communication bus 122 can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 9The term 124 is represented by a single line, but this does not imply a single bus or a single type of bus. The memory 124 may include volatile memory, such as random-access memory (RAM); it may also include non-volatile memory, such as flash memory, hard disk drive (HDD), or solid-state drive (SSD); or it may include combinations of the above types of memory. The processor 121 may be a central processing unit (CPU), a network processor (NP), or a combination of a CPU and an NP. The processor 121 may further include hardware chips. These hardware chips may be application-specific integrated circuits (ASICs), programmable logic devices (PLDs), or combinations thereof. The PLD may be a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof. Optionally, the memory 124 is also used to store program instructions. The processor 121 can call the program instructions to implement the failure mode detection method for important components on the surface of a rail train, as described in this embodiment.

[0093] The embodiments of the present invention have been described above with reference to the accompanying drawings. However, the present invention is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of the present invention without departing from the spirit and scope of the claims. All of these forms are within the protection scope of the present invention.

Claims

1. A method for detecting the failure modes of important components on the surface of a train, characterized in that, Includes the following steps: S1: Obtain an image of the outer surface of the rail locomotive, and obtain a training dataset based on the image of the outer surface of the rail locomotive; S2: Construct a component segmentation model, which includes a combined backbone network and a comprehensive loss function; the combined backbone network includes a shared weight encoder, a semantic segmentation decoder, a super-resolution decoder, and a feature guidance module; the shared weight encoder is used to extract image features; the semantic segmentation decoder is used to obtain a first semantic segmentation feature based on the image features; the super-resolution decoder is used to obtain a first super-resolution feature based on the image features; the feature guidance module is used to minimize the distance between the task target and the corresponding task result generated by the feature guidance module based on the first semantic segmentation feature and the first super-resolution feature, guide the super-resolution feature and semantic segmentation feature to focus on the region of interest, obtain a second semantic segmentation feature and a second super-resolution feature, and improve the semantic segmentation effect; the feature guidance module includes a stitching layer, a first... Convolutional layers, residual channel attention modules, multi-scale convolutional modules, and second convolutional layers. Convolutional layer, third Convolutional layer, first dot product operation, second dot product operation, fourth dot product operation Convolutional layer, fifth Convolutional layer, first addition operation, second addition operation; the concatenation layer is used to concatenate the first semantic segmentation feature and the first super-resolution feature in the channel dimension to obtain the concatenation result; the first Convolutional layers are used to determine the first Feature fusion is performed on the convolutional layer concatenation results; the residual channel attention module uses two 3D arrays. A depthwise convolution of 3 is used to capture local patterns in the integrated features. Two linear layers are then used to directly compress and expand the features to capture channel relationships. A sigmoid function is then used to generate weights along the channel axis based on the aforementioned features. The local patterns are reweighted using a Hadamard product, and finally, the input features are added to the reweighted features. The multi-scale convolution module groups the features processed by the residual channel attention module into groups based on channel size, with the first group using... Convolution maintains the current scale; image pooling is used for the last group to collect global spatial information, while different dilation rates are used for the other groups. Depth convolution is used to capture information at different levels, where the dilation rate is set to the group index minus 1. Features from each group are concatenated along the channel dimension, and then processed using batch normalization and the GELU activation function. The second... The convolutional layer is used to map the output of the residual channel attention module to the space of the first semantic segmentation feature to generate semantic segmentation task attention weights; the third Convolutional layers are used to map the output of the residual channel attention module to the space of the first super-resolution feature to generate super-resolution task attention weights; The first dot product operation is used to multiply the semantic segmentation task attention weights with the first semantic segmentation features to obtain semantic segmentation weighted features; The second dot product operation is used to multiply the super-resolution task attention weights with the first super-resolution feature to obtain the super-resolution weighted feature; the fourth The convolutional layer is used to convolve the semantic segmentation weighted features; the fifth... Convolutional layers are used to convolve the weighted features at the super-resolution level; The first addition operation layer is used to add the fourth The output of the convolutional layer is added to the first semantic segmentation feature to obtain the second semantic segmentation feature; the second addition layer is used to add the fifth semantic segmentation feature. The output of the convolutional layer is added to the first super-resolution feature to obtain the second super-resolution feature; the combined backbone network also includes a semantic segmentation head and a super-resolution head; the semantic segmentation head is used to obtain a semantic segmentation result based on the semantic segmentation feature; the super-resolution head is used to obtain a super-resolution result based on the semantic segmentation feature; S3: Train the component segmentation model using the training dataset to obtain a trained component segmentation model; S4: Based on the image to be tested, the trained component segmentation model is used for detection to obtain the bolt segmentation image; S5: Based on the bolt segmentation image, perform polygon fitting, angle calculation and detection to obtain the detection result.

2. The method for detecting failure modes of important components on train surfaces according to claim 1, characterized in that, Step S1 specifically includes: S11: Acquire an image of the outer surface of the railcar, preprocess the image of the outer surface of the railcar to obtain bolt area image, bolt number, bolt coordinates, and initial rotation angle; S12: Based on the bolt area image, bolt number, bolt coordinates, and initial rotation angle, perform annotation and data augmentation to obtain the training dataset.

3. The method for detecting failure modes of important components on train surfaces according to claim 1, characterized in that, The comprehensive loss function is as follows: ， in, For the comprehensive loss function, For semantic segmentation loss, To enhance semantic segmentation loss, To enhance super-resolution loss, For super-resolution loss, To divide the loss weights for boundary constraints, The loss is segmented for boundary constraints.

4. The method for detecting failure modes of important components on train surfaces according to claim 1, characterized in that, Step S4 specifically includes: S41: Preprocess the image to be tested to obtain the image of the bolt area to be tested, the bolt number to be tested, the coordinates of the bolt to be tested, and the initial rotation angle of the bolt to be tested; S42: Use the trained component segmentation model to detect the bolt region image to obtain a bolt segmentation image.

5. The method for detecting failure modes of important components on train surfaces according to claim 1, characterized in that, Step S5 specifically includes: S51: Based on the bolt segmentation image, perform contour extraction and polygon fitting to obtain a regular hexagon; S52: Calculate the bolt rotation angle based on the regular hexagon; S53: The detection result is obtained based on the bolt rotation angle, the initial rotation angle of the bolt to be tested, and the preset angle threshold.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the method for detecting failure modes of important components on the train surface as described in any one of claims 1-5.

7. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method for detecting failure modes of important components on the train surface as described in any one of claims 1-5.

Images