Rail train surface important component failure form detection method, medium and equipment

Abstract

The invention discloses a rail train surface important component failure form detection method, medium and equipment, and relates to the technical field of intelligent detection engineering.The rail train surface important component failure form detection method mainly comprises the steps that a training data set is obtained according to an outer surface image of a rail locomotive; constructing a component segmentation model, and training the component segmentation model by using the training data set to obtain a trained component segmentation model; and detecting an image to be detected by using the trained component segmentation model to obtain a bolt segmentation image, and performing polygon fitting, angle calculation and detection to obtain a detection result. By implementing the rail train surface important component failure form detection method, medium and equipment provided by the invention, the accuracy and reliability of train surface important component failure form detection can be improved.

Description

Technical Field

[0001] The present invention relates to the field of intelligent detection engineering technology, and more specifically to a method, medium and equipment for detecting failure forms of important components on the surface of a railway train. Background Art

[0002] China's railway network is expected to reach 175,000 kilometers in 2025, and safety maintenance issues are becoming increasingly important. The structure of trains is complex, with many small but critical components such as bolts. At present, the methods for bolt failure detection are mainly divided into three categories: one is based on manual visual inspection, but the efficiency of manual inspection is low, and it is easy to miss and misdetect; the second is based on non-destructive testing technology, such as ultrasonic testing and vibration signal analysis, but these methods are mainly used to identify internal faults and looseness of bolts, with low detection efficiency, and it is difficult to achieve rapid detection of bolt loosening under actual working conditions; the third is machine vision technology based on deep learning. In recent years, some researchers have used target detection networks to roughly locate bolt positions, used semantic segmentation networks to extract anti-loosening marking lines, and then obtained the offset angle of the marking line through Hough transform to judge the bolt status. However, this method relies heavily on the clarity and accuracy of the anti-loosening marking lines, and these marking lines are usually painted manually, with different shapes, low image contrast, and interference from background noise, resulting in errors in the extraction of offset angles, affecting the detection accuracy. Summary of the invention

[0003] The purpose of the present invention is to provide a method, medium and equipment for detecting failure forms of important components on the surface of railway trains, which can improve the accuracy and reliability of detecting failure forms of important components on the surface of trains.

[0004] The present invention provides a method for detecting failure forms of important components on the surface of a train, comprising the following steps: S1: acquiring an image of the outer surface of a railway locomotive, and obtaining a training data set according to the image of the outer surface of the railway locomotive; S2: constructing a component segmentation model, wherein the component segmentation model comprises a combined backbone network and a comprehensive loss function; S3: training the component segmentation model using the training data set to obtain a trained component segmentation model; S4: performing detection according to an image to be tested using the trained component segmentation model to obtain a bolt segmentation image; S5: performing polygon fitting, angle calculation and detection according to the bolt segmentation image to obtain a detection result.

[0005] Furthermore, the above-mentioned step S1 specifically includes: S11: obtaining an image of the outer surface of a railway locomotive, preprocessing the image of the outer surface of the railway locomotive, and obtaining a bolt area picture, bolt number, bolt coordinates, and an initial rotation angle; S12: labeling and data enhancement are performed according to the bolt area picture, bolt number, bolt coordinates, and initial rotation angle to obtain a training data set.

[0006] Furthermore, the above-mentioned combined backbone network includes a shared weight encoder, a semantic segmentation decoder, a super-resolution decoder and a feature guidance module; the shared weight encoder is used to extract image features; the semantic segmentation decoder is used to obtain a first semantic segmentation feature based on the image features; the super-resolution decoder is used to obtain a first super-resolution feature based on the image features; the feature guidance module is used to minimize the distance between the task objective and the corresponding task result generated by the feature guidance module based on the first semantic segmentation feature and the first super-resolution feature, guide the super-resolution feature and the semantic segmentation feature to focus on the area of ​​interest, obtain the second semantic segmentation feature and the second super-resolution feature, and improve the semantic segmentation effect.

[0007] Furthermore, the feature guidance module includes a splicing layer, a first Convolutional layer, residual channel attention module, multi-scale convolution module, second Convolutional layer, third Convolutional layer, first dot multiplication operation, second dot multiplication operation, fourth Convolutional layer, fifth convolution layer, a first addition operation, a second addition operation; the splicing layer is used to splice the first semantic segmentation feature and the first super-resolution feature in the channel dimension to obtain a splicing result; the first The convolutional layer is used to The convolution layer splicing results are used for feature fusion; the residual channel attention module uses two 3 3 depth convolution is used to capture the local pattern in the integrated features, and then two linear layers are used to directly compress and expand the features to capture the channel relationship. The Sigmoid function is used to generate weights along the channel axis based on the above features, and the local pattern is re-weighted through the Hadamard product. Finally, the input features are added to the re-weighted features; the multi-scale convolution module averages and groups the features processed by the residual channel attention module according to the channel size, where the first group uses The convolution keeps the current scale, uses image pooling to count the global spatial information for the last group, and uses different dilation rates for the other groups. Deep convolution is used to capture information at different levels, where the expansion rate is set to the group index minus 1, and each group of features is concatenated in the channel dimension, and then processed by batch normalization and GELU activation function; the second The convolutional layer is used to map the output of the residual channel attention module to the space of the first semantic segmentation feature to generate the semantic segmentation task attention weight; the third The convolutional layer is used to map the output of the residual channel attention module to the space of the first super-resolution feature to generate a super-resolution task attention weight; the first dot multiplication operation is used to perform a dot multiplication of the semantic segmentation task attention weight and the first semantic segmentation feature to obtain a semantic segmentation weighted feature; the second dot multiplication operation is used to perform a dot multiplication of the super-resolution task attention weight and the first super-resolution feature to obtain a super-resolution weighted feature; the fourth The convolution layer is used to convolve the semantic segmentation weighted features; the fifth The convolution layer is used to convolve the super-resolution weighted features; the first addition operation layer is used to add the fourth The output of the convolution layer is added to the first semantic segmentation feature to obtain a second semantic segmentation feature; the second addition operation layer is used to add the fifth The output of the convolutional layer is added to the first super-resolution feature to obtain a second super-resolution feature.

[0008] Furthermore, the above-mentioned combined backbone network also includes a semantic segmentation head and a super-resolution head; the semantic segmentation head is used to obtain a semantic segmentation result based on the semantic segmentation features; the super-resolution head is used to obtain a super-resolution result based on the semantic segmentation features.

[0009] Furthermore, the above comprehensive loss function is as follows: , in, is the comprehensive loss function, is the semantic segmentation loss, To enhance semantic segmentation loss, To enhance the super-resolution loss, is the super-resolution loss, is the boundary constraint segmentation loss weight, is the boundary constrained segmentation loss.

[0010] Furthermore, the above step S4 specifically includes: S41: preprocessing the image to be tested to obtain a picture of the bolt area to be tested, the number of the bolt to be tested, the coordinates of the bolt to be tested, and the initial rotation angle of the bolt to be tested; S42: using the trained component segmentation model to detect the picture of the bolt area to be tested to obtain a bolt segmentation image.

[0011] Furthermore, the above step S5 specifically includes: S51: based on the bolt segmentation image, contour extraction and polygon fitting are performed to obtain a regular hexagon; S52: angle calculation is performed based on the regular hexagon to obtain the bolt rotation angle; S53: based on the bolt rotation angle, the initial rotation angle of the bolt to be tested and the preset angle threshold, the detection result is obtained.

[0012] The present invention also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned method for detecting failure forms of important components on the surface of a railway train are implemented.

[0013] The present invention also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the steps of the above-mentioned method for detecting failure forms of important components on the surface of a railway train are implemented.

[0014] The implementation of the method, medium and equipment for detecting failure forms of important components on the surface of railway trains provided by the present invention has the following beneficial effects: (1) The present invention constructs and trains a dual super-resolution semantic segmentation boundary constraint network DSCBNet, which provides an innovative method for semantic segmentation tasks, significantly improves the segmentation effect of low-resolution images, and pays more attention to the edge details of the target while ensuring accuracy. When processing low-resolution images, traditional segmentation methods often cause edge blur and inaccurate segmentation due to loss of details. The DSCBNet network of the present invention effectively solves this problem through the super-resolution and semantic segmentation feature fusion module and boundary constraint loss. (2) The present invention determines whether a bolt is loose by the deflection angle of the hexagonal profile of the bolt, providing a more accurate and stable loosening detection method than the existing method. Traditional bolt loosening detection methods usually rely on the offset angle of the marking line for judgment, but the artificially painted marking line has different shapes and is affected by various factors such as pollutants and surface wear, which can easily lead to inaccurate detection results. The present invention utilizes the geometric characteristics of the bolt itself, such as the inherent hexagonal profile, which not only avoids the problem that the marking line is difficult to accurately detect in a complex environment, but also has a stronger anti-interference ability, significantly improving the reliability of loosening detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The present invention will be further described below with reference to the accompanying drawings and embodiments, in which: Figure 1 It is a flow chart of the failure form detection method of important components on the surface of a train provided by the present invention; Figure 2 It is a schematic diagram of the DSCBNet network backbone structure provided by the present invention; Figure 3 It is a schematic diagram of the SFGM structure provided by the present invention; Figure 4 It is a schematic diagram of the MRCAB structure provided by the present invention; Figure 5 It is a schematic diagram of Lcb loss calculation provided by the present invention; Figure 6is a labeled example diagram provided by the present invention; Figure 7 is an example diagram of the detection results provided by the present invention; Figure 8 It is a schematic diagram for calculating the loosening angle of a hexagonal bolt provided by the present invention; Fig. 9 It is a structural block diagram of the computer device provided by the present invention. DETAILED DESCRIPTION

[0016] In order to have a clearer understanding of the technical features, purposes and effects of the present invention, specific embodiments of the present invention are now described in detail with reference to the accompanying drawings.

[0017] Figure 1 A schematic diagram of a method for detecting failure forms of important components on the surface of a train in this embodiment is shown. In this embodiment, the method for detecting failure forms of important components on the surface of a railway train comprises the following steps: S1: Acquire an image of the outer surface of a railway locomotive, and obtain a training data set according to the image of the outer surface of the railway locomotive; In an exemplary embodiment, step S1 specifically includes: S11: Acquire an outer surface image of a railway locomotive, and pre-process the outer surface image of the railway locomotive to obtain a bolt area image, a bolt number, a bolt coordinate, and an initial rotation angle; For an exemplary embodiment, in step S11, the outer surface image of the rail locomotive is collected, and a hardware image acquisition platform consisting of a high-resolution linear array camera and a light source fixed on both sides of the track is used. The upper computer software of the camera and the light source adjusts the appropriate exposure time, acquisition frequency, light intensity and other parameters to obtain the locomotive outer surface image that is convenient for subsequent image processing and stores it in the computer; the locomotive outer surface image is distorted and corrected, and the image of each bolt area is cut out from the whole train through the known position information prior condition, and each bolt is numbered and the coordinate information and initial rotation angle are recorded. ; Perform image preprocessing on the cropped bolt area image to improve image quality and enhance the recognizability of bolt features, adjust the image contrast and brightness, use high-pass filtering to sharpen the image edge, apply gamma correction to optimize the uneven brightness distribution of the image, and finally use bilateral filtering to process the image noise, while retaining the edge detail features of the bolts; S12: labeling and data enhancement are performed according to the bolt area image, bolt number, bolt coordinates, and initial rotation angle to obtain a training data set; For an exemplary embodiment, in step S12, labelme is used to annotate the bolt image after image preprocessing, so as to obtain the bolt label; then, data enhancement processing is performed on these bolt image and label data pairs, so as to increase data diversity; the same geometric transformation is performed on the bolt image and label data pairs, such as scaling, displacement, rotation, contrast transformation and other methods; these methods can change the position information, color, size and other features in the image, and can increase the diversity of defect images to a certain extent; S2: construct a component segmentation model, wherein the component segmentation model includes a combined backbone network and a comprehensive loss function; In an exemplary embodiment, the combined backbone network includes a shared weight encoder, a semantic segmentation decoder, a super-resolution decoder and a feature guidance module; the shared weight encoder is used to extract image features; the semantic segmentation decoder is used to obtain a first semantic segmentation feature according to the image features; the super-resolution decoder is used to obtain a first super-resolution feature according to the image features; the feature guidance module is used to minimize the distance between the task goal and the corresponding task result generated by the feature guidance module according to the first semantic segmentation feature and the first super-resolution feature, guide the super-resolution feature and the semantic segmentation feature to focus on the area of ​​interest, obtain the second semantic segmentation feature and the second super-resolution feature, and improve the semantic segmentation effect; In an exemplary embodiment, the feature guidance module includes a splicing layer, a first Convolutional layer, residual channel attention module, multi-scale convolution module, second Convolutional layer, third Convolutional layer, first dot multiplication operation, second dot multiplication operation, fourth Convolutional layer, fifth convolution layer, a first addition operation, a second addition operation; the splicing layer is used to splice the first semantic segmentation feature and the first super-resolution feature in the channel dimension to obtain a splicing result; the first The convolutional layer is used to The convolutional layer splicing results are used for feature fusion; The residual channel attention module uses two 3 A deep convolution of 3 is used to capture local patterns in the integrated features, and two linear layers are used to directly compress and expand the features to capture channel relationships. The Sigmoid function is used to generate weights along the channel axis based on the above features, and the local patterns are re-weighted through the Hadamard product. Finally, the input features are added to the re-weighted features. The multi-scale convolution module groups the features processed by the residual channel attention module evenly according to the channel size, where the first group uses The convolution keeps the current scale, uses image pooling to count the global spatial information for the last group, and uses different dilation rates for the other groups. Deep convolution is used to capture information at different levels, where the expansion rate is set to the group index minus 1, and each group of features is concatenated in the channel dimension, and then the features are processed by batch normalization and GELU activation function; The second The convolutional layer is used to map the output of the residual channel attention module to the space of the first semantic segmentation feature to generate the semantic segmentation task attention weight; the third The convolutional layer is used to map the output of the residual channel attention module to the space of the first super-resolution feature to generate a super-resolution task attention weight; the first dot multiplication operation is used to perform a dot multiplication of the semantic segmentation task attention weight and the first semantic segmentation feature to obtain a semantic segmentation weighted feature; the second dot multiplication operation is used to perform a dot multiplication of the super-resolution task attention weight and the first super-resolution feature to obtain a super-resolution weighted feature; the fourth The convolution layer is used to convolve the semantic segmentation weighted features; the fifth The convolution layer is used to convolve the super-resolution weighted features; the first addition operation layer is used to add the fourth The output of the convolution layer is added to the first semantic segmentation feature to obtain a second semantic segmentation feature; the second addition operation layer is used to add the fifth The output of the convolutional layer is added to the first super-resolution feature to obtain a second super-resolution feature; In an exemplary embodiment, the combined backbone network further includes a semantic segmentation head and a super-resolution head; the semantic segmentation head is used to obtain a semantic segmentation result according to the semantic segmentation feature; the super-resolution head is used to obtain a super-resolution result according to the semantic segmentation feature; For an exemplary embodiment, a combined backbone network DSCBNet of the above-mentioned component segmentation model is constructed to obtain a bolt segmentation map with clear edges; the main structure of the DSCBNet network consists of a shared weight encoder, a semantic segmentation decoder, a super-resolution decoder and a feature guidance module; wherein the shared encoder, the semantic segmentation decoder and the super-resolution decoder can be flexibly replaced with the current mainstream encoding and decoding structure; the image, the label and the super-resolution image with a size u times that of the original image are used as the input of the DSCBNet network; the image is subjected to feature extraction by the shared weight encoder, and the semantic segmentation decoder and the super-resolution decoder are processed to obtain the first semantic segmentation feature Fseg and the first super-resolution feature Fsr, and the segmentation result Oseg and the super-resolution result Osr are obtained by the semantic segmentation head and the super-resolution head; the semantic segmentation head is a layer 1 1 convolution, adjust the number of channels to nclass, and thus obtain the segmentation result; the super-resolution head is 3 layers of convolution layers and 2 Tanh activation functions, and finally pixel shuffle is used to upsample to the target resolution to obtain the super-resolution result; Figure 2 The figure shows the main structure of the DSCBNet network. It should be noted that when it is difficult to obtain high-resolution images, the image and label can be downsampled u times, and then the downsampled u-fold image and label, and the original image can be passed into the network for training; the features Fseg and Fsr obtained by the semantic segmentation decoder and super-resolution decoder are passed into the feature guidance module (Shared Feature Guidance Module, SFGM) to minimize the distance between the task goal and the corresponding task result generated by the feature guidance module, guide the super-resolution features and semantic segmentation features to focus on the Region of Interest (ROI) area, and improve the semantic segmentation effect. The SFGM module structure is shown in the figure. Figure 3 As shown; Fseg and Fsr are concatenated in the channel dimension, using a 1 1 convolution to fuse features, using the residual channel attention module MRCAB and the multi-scale convolution module MS-Conv to integrate the features from the two tasks; using two 1 1 Conv maps the output of MS-Conv to the space of segmentation decoding feature Fseg and super-resolution decoding feature Fsr to generate the attention weights of the two tasks, and then multiplies the weights with Fseg and Fsr to obtain weighted features, and then passes through a 1 1 convolution and add the original features to obtain the second semantic segmentation feature Ffss and the second super-resolution feature Ffsr that are more focused on ROI; the semantic segmentation feature Ffss and the super-resolution feature Ffsr obtained by the feature guidance module SFGM are processed by the semantic segmentation head and the super-resolution head to obtain the segmentation result Ofseg and the super-resolution result Ofsr; the semantic segmentation head is a layer 1 1 convolution, adjust the number of channels to nclass, and thus obtain the segmentation result; the super-resolution head is a 3-layer convolution layer and a 2-time Tanh activation function, and finally uses pixel shuffle to upsample to the target resolution, thereby obtaining the super-resolution result; Among them, the MRCAB module structure is as follows Figure 4 As shown; using two 3 3 depth convolution is used to capture the local pattern in the integrated features, and then two linear layers are used to directly compress and expand the features to capture the channel relationship. The Sigmoid function is used to generate weights along the channel axis based on the above features. The local pattern is re-weighted through the Hadamard product. Finally, the input features are added to the re-weighted features. Among them, the multi-scale convolution module MS-Conv divides the features processed by the MRCAB module into k groups according to the channel size, and uses different processing methods for different groups of information. The first group uses 1 1. Convolution maintains the current scale. For the second to (k-1)th groups, 3×3 depth convolutions with different dilation rates are used to capture information at different levels. The dilation rate is set to the group index minus 1. For the kth group, image pooling is used to count the global spatial information. The features of each group are concatenated in the channel dimension, and then the features are processed by batch normalization and GELU activation function. In an exemplary embodiment, the comprehensive loss function is as follows: , in, is the comprehensive loss function, is the semantic segmentation loss, To enhance semantic segmentation loss, To enhance the super-resolution loss, is the super-resolution loss, is the boundary constraint segmentation loss weight, is the boundary constraint segmentation loss; For an exemplary embodiment, in order to constrain the model to learn a more stable and clear boundary, this embodiment combines the idea of ​​deep supervision, constructs a comprehensive loss function in the network, and optimizes model training; through the semantic segmentation loss ( ), super-resolution loss ( ), enhanced semantic segmentation loss ( ), enhanced super-resolution loss ( ) and the boundary-constrained segmentation loss ( ) The weighted mixed loss function (Loss) is used to optimize the parameters in the DSCBNet network and obtain the trained DSCBNet network model; the comprehensive loss function of DSCBNet is shown in formula (1): (1) In the formula, The cross entropy loss function is used to measure Use the mean square error loss function to measure, The structural similarity (SSIM) loss function is used to measure The regional mutual information (RMI) loss function is used for measurement. In this embodiment, the value of the boundary constraint segmentation loss weight γ is selected as 2; In an exemplary embodiment, the boundary constraint segmentation loss ( )Depend on , , Composition, such as Figure 5 Shown Calculation diagram; First, the semantic segmentation result map Ofseg and the label map obtained by the SFGM module are morphologically processed and eroded. The structural element of the erosion operation is a circle with a radius of d. After morphological processing, they are subtracted from the original image to obtain the semantic segmentation result and the label boundary map; The feature map Fseg, the semantic segmentation result and the label boundary map obtained by the semantic segmentation decoder are processed according to the conditional correctness aware sampling (CCASS) rule to obtain the positive and negative samples of the anchor point, and generate a local class average feature for each anchor point; Conditional Correctness Aware Sampling (CCASS) complies with the following four rules: (1) Only select K anchor points in the neighborhood of the anchor point. K samples; (2) the selected samples must be correctly classified by the model; (3) positive sample points are pixels of the same category as the anchor point; (4) negative sample points are pixels of a different category than the anchor point; CCASS strategy uses a K K hollow convolution extracts anchor point K The number of positive samples in the K domain, for example, 3 The extractor weights of 3 are initialized as follows:

[0018] Through this hollow number extractor convolution layer, a convolution operation is performed on each channel of the one-hot label to obtain a positive sample number map; the number of positive samples The calculation formula is shown in formula 1: (1) In the formula, represents the predicted label, One-hot labels representing the true labels, represents element-wise multiplication, Represents the convolution operation; The class average feature refers to the , define its local class average The embedding of is the average feature of its positive samples, as shown in Formula 2: (2) In the formula, Represents the positive sample set The number of samples; the same method as above to obtain the number of positive samples, using K The hollow convolution of K obtains the class average feature map, the class average feature map The calculation formula is shown in formula 3 and 4: (3) (4) In the formula, represents the feature map Fseg processed by the semantic segmentation decoder, Used to avoid division by zero; Calculate the cross entropy loss based on the average feature map of the anchor class obtained and the true label , which is used to ensure that the local category center can correctly reflect the category characteristics, as shown in formula (5): (5) In the formula, Expressed as the total number of anchor points, represents the predicted local category center label, represents the true label; According to the pairwise similarity between the supervised boundary pixels and their positive and negative samples, the distance between the boundary pixels and the positive samples is shortened and the distance between the boundary pixels and the negative samples is moved away, as shown in Equation (6): (6) In the formula, and Respectively represent the number of positive and negative samples of boundary pixels, It is expressed as the cosine similarity between the anchor point feature and the positive and negative sample point features. represents the true cosine similarity; According to the pairwise similarity between each boundary pixel and its local category center feature, that is, the distance between the class average feature and the boundary pixel feature, the intra-class consistency of the boundary pixel is increased, as shown in formula (7): (7) In the formula, Expressed as The features of the boundary pixels, Expressed as The local average feature of the boundary pixels, It means that only when the anchor pixel K This item is calculated only when there is at least one positive sample in the K domain; In summary, the total boundary constraint segmentation loss function ( ) as shown in formula (8): (8) In this embodiment, the values ​​of weights α and β are selected as 0.1 and 0.5; S3: training the component segmentation model using the training data set to obtain a trained component segmentation model; In an exemplary embodiment, the Poly strategy is used to train the above-mentioned component segmentation model, i.e., the double super-resolution boundary constraint network, to obtain a trained component segmentation model; specifically, the Adam optimizer is used, the momentum is 0.9, the weight decay is set to 0.0001, and the parameters and They are 0.9 and 0.99 respectively; using the Poly learning scheme, the learning rate is adjusted to: , in and Set them to 0.9 and 40000 respectively, set the initial learning rate to 0.0005, and adjust the batch size appropriately according to the video memory size; S4: performing detection using the trained component segmentation model according to the image to be tested to obtain a bolt segmentation image; In an exemplary embodiment, step S4 specifically includes: S41: preprocessing the image to be tested to obtain a picture of the bolt area to be tested, a number of the bolt to be tested, coordinates of the bolt to be tested, and an initial rotation angle of the bolt to be tested; For an exemplary embodiment, in step S41, the image to be tested that has been collected and corrected for distortion is clipped according to the prior condition of known position information, and each bolt is numbered and its coordinates are recorded in a specified order, and the initial rotation angle corresponding to the bolt is found according to the number and coordinates. ; S42: Using the trained component segmentation model to detect the image of the bolt area to be tested, to obtain a bolt segmentation image; For an exemplary embodiment, in step S42, the cropped bolt image is input into the DSCBNet network of the trained component segmentation model for detection to obtain a hexagonal bolt segmentation image with clear edges; In an exemplary embodiment, this embodiment conducts a train exterior bolt loosening detection experiment based on DSCBNet, and the experimental platform is CPU: AMD Ryzen 77735H with Radeon Graphics, GPU: NVIDIA GeForceRTX 4060, 8g memory; operating system window11; In this embodiment, in order to accurately evaluate the performance of the model, miou, BFscore and Precision are selected as the evaluation indicators of the model; the average intersection-over-union ratio (miou) is 96.53%, the accuracy (Precision) is 95.97%, the recall rate (Recall) is 97.24%, when the radius of the circular structural element is 2 pixels, the boundary F1 score (BFscore) is 82.47%, when the radius of the circular structural element is 4 pixels, the boundary F1 score (BFscore) is 91.71%; Among them, the annotation examples are as follows Figure 6 As shown, the test results are shown in Figure 7 As shown; S5: performing polygon fitting, angle calculation and detection according to the bolt segmentation image to obtain a detection result; In an exemplary embodiment, step S5 specifically includes: S51: performing contour extraction and polygon fitting according to the bolt segmentation image to obtain a regular hexagon; S52: performing angle calculation according to the regular hexagon to obtain a bolt rotation angle; S53: obtaining a detection result according to the bolt rotation angle, the initial rotation angle of the bolt to be tested and a preset angle threshold; For an exemplary embodiment, the bolt loosening judgment is performed in step S5; specifically, the obtained bolt hexagon segmentation result is polygonally fitted and the hexagon angle is calculated, which mainly includes: first, extracting the contour of the segmentation result, finding the minimum inscribed circle of the hexagon, drawing a vertical upward ray with the center O and the point where it intersects with the contour is , the ray rotates 1° clockwise each time, and the contour points between each ray and the segmentation are calculated and Angle between points and O The distance, according to the angle and O The angle-distance curve is drawn based on the distance, and the curve is smoothed using Gaussian filtering. The peak value of the smoothed curve is calculated and the peak point is used as the key point to segment the contour. A straight line is fitted according to the segmented contour, and then the intersection points of the fitted straight line are connected to generate a regular hexagon. Then, the The angle between the edge where the point is located and the X-axis. When the angle is less than 90°, the edge is defined as , The adjacent left is , The adjacent right side is , if the angle is greater than 90°, define the edge as , The adjacent left is , The adjacent left is ;calculate , , The absolute rotation angle of the bolt is obtained by calculating the average value of the three rotation angles. ; and the initial corner Do the difference ,if Greater than the preset angle threshold , it is judged as loose, otherwise it is judged as not loose. The specific situation is as follows Figure 8 As shown; In an exemplary embodiment, the preset angle threshold is 5 .

[0019] This embodiment provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the above-mentioned method for detecting failure forms of important components on the surface of a railway train are implemented. The storage medium may be a magnetic disk, an optical disk, a read-only memory (ROM), a random access memory (RAM), a flash memory, a hard disk (HDD), or a solid-state drive (SSD), etc.; the storage medium may also include a combination of the above-mentioned types of memory.

[0020] This embodiment provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the steps of the above-mentioned method for detecting failure forms of important components on the surface of a railway train are implemented.

[0021] like Fig. 9As shown, the computer device may include: at least one processor 121, such as a central processing unit (CPU), at least one communication interface 123, a memory 124, and at least one communication bus 122. Among them, the communication bus 122 is used to realize the connection and communication between these components. Among them, the communication interface 123 may include a display screen (Display), a keyboard (Keyboard), and the optional communication interface 123 may also include a standard wired interface and a wireless interface. The memory 124 may be a high-speed random access memory (Random Access Memory, RAM), or a non-volatile memory (non-volatile memory), such as at least one disk storage. The memory 124 may also be at least one storage device located away from the aforementioned processor 121. Among them, the memory 124 stores an application program, and the processor 121 calls the program code stored in the memory 124 to perform any of the above method steps. Among them, the communication bus 122 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The communication bus 122 can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Fig. 9Only one line is used to represent it, but it does not mean that there is only one bus or one type of bus. Among them, the memory 124 may include a volatile memory (volatile memory), such as a random-access memory (random-access memory, RAM); the memory may also include a non-volatile memory (non-volatile memory), such as a flash memory (flash memory), a hard disk drive (hard disk drive, HDD) or a solid-state drive (solid-state drive, SSD); the memory 124 may also include a combination of the above-mentioned types of memory. Among them, the processor 121 may be a central processing unit (central processing unit, CPU), a network processor (network processor, NP) or a combination of CPU and NP. Among them, the processor 121 may further include a hardware chip. The above-mentioned hardware chip may be an application-specific integrated circuit (application-specific integrated circuit, ASIC), a programmable logic device (programmable logic device, PLD) or a combination thereof. The above-mentioned PLD may be a complex programmable logic device (complex programmable logic device, CPLD), a field-programmable gate array (field-programmable gate array, FPGA), a generic array logic (generic array logic, GAL) or any combination thereof. Optionally, the memory 124 is also used to store program instructions. The processor 121 can call the program instructions to implement the failure form detection method of important components on the surface of a railway train as in this embodiment.

[0022] The embodiments of the present invention are described above in conjunction with the accompanying drawings, but the present invention is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the enlightenment of the present invention, ordinary technicians in this field can also make many forms without departing from the scope of protection of the purpose of the present invention and the claims, which all fall within the protection of the present invention.

Claims

1. A method for detecting failure forms of important components on the surface of a train, characterized in that: The following steps are involved: S1: Acquire an image of the outer surface of a railway locomotive, and obtain a training data set according to the image of the outer surface of the railway locomotive; S2: construct a component segmentation model, wherein the component segmentation model includes a combined backbone network and a comprehensive loss function; S3: training the component segmentation model using the training data set to obtain a trained component segmentation model; S4: performing detection using the trained component segmentation model according to the image to be tested to obtain a bolt segmentation image; S5: Perform polygon fitting, angle calculation and detection according to the bolt segmentation image to obtain a detection result.

2. The method for detecting failure forms of important components on the surface of a train according to claim 1, characterized in that: Step S1 specifically includes: S11: Acquire an outer surface image of a railway locomotive, and pre-process the outer surface image of the railway locomotive to obtain a bolt area image, a bolt number, a bolt coordinate, and an initial rotation angle; S12: labeling and data enhancement are performed according to the bolt area image, bolt number, bolt coordinates, and initial rotation angle to obtain a training data set.

3. The method for detecting failure forms of important components on the surface of a train according to claim 1, characterized in that: The combined backbone network includes a shared weight encoder, a semantic segmentation decoder, a super-resolution decoder and a feature guidance module; the shared weight encoder is used to extract image features; the semantic segmentation decoder is used to obtain a first semantic segmentation feature based on the image features; the super-resolution decoder is used to obtain a first super-resolution feature based on the image features; the feature guidance module is used to minimize the distance between the task objective and the corresponding task result generated by the feature guidance module based on the first semantic segmentation feature and the first super-resolution feature, guide the super-resolution feature and the semantic segmentation feature to focus on the area of ​​interest, obtain the second semantic segmentation feature and the second super-resolution feature, and improve the semantic segmentation effect.

4. The method for detecting failure forms of important components on the surface of a train according to claim 3, characterized in that: The feature guiding module includes a splicing layer, a first Convolutional layer, residual channel attention module, multi-scale convolution module, second Convolutional layer, third Convolutional layer, first dot multiplication operation, second dot multiplication operation, fourth Convolutional layer, fifth convolution layer, a first addition operation, a second addition operation; the splicing layer is used to splice the first semantic segmentation feature and the first super-resolution feature in the channel dimension to obtain a splicing result; the first The convolutional layer is used to The convolutional layer splicing results are used for feature fusion; The residual channel attention module uses two 3 A deep convolution of 3 is used to capture local patterns in the integrated features, and two linear layers are used to directly compress and expand the features to capture channel relationships. The Sigmoid function is used to generate weights along the channel axis based on the above features, and the local patterns are re-weighted through the Hadamard product. Finally, the input features are added to the re-weighted features. The multi-scale convolution module groups the features processed by the residual channel attention module evenly according to the channel size, where the first group uses The convolution keeps the current scale, uses image pooling to count the global spatial information for the last group, and uses different dilation rates for the other groups. Deep convolution is used to capture information at different levels, where the expansion rate is set to the group index minus 1, and each group of features is concatenated in the channel dimension, and then the features are processed by batch normalization and GELU activation function; The second The convolution layer is used to map the output of the residual channel attention module to the space of the first semantic segmentation feature to generate the semantic segmentation task attention weight; the third The convolutional layer is used to map the output of the residual channel attention module to the space of the first super-resolution feature to generate the super-resolution task attention weight; The first dot product operation is used to perform a dot product of the semantic segmentation task attention weight and the first semantic segmentation feature to obtain a semantic segmentation weighted feature; The second dot product operation is used to perform a dot product of the super-resolution task attention weight and the first super-resolution feature to obtain a super-resolution weighted feature; the fourth The convolution layer is used to convolve the semantic segmentation weighted features; the fifth The convolution layer is used to convolve the super-resolution weighted features; The first adding operation layer is used to add the fourth The output of the convolution layer is added to the first semantic segmentation feature to obtain a second semantic segmentation feature; the second addition operation layer is used to add the fifth The output of the convolutional layer is added to the first super-resolution feature to obtain a second super-resolution feature.

5. The method for detecting failure forms of important components on the surface of a train according to claim 1, characterized in that: The combined backbone network also includes a semantic segmentation head and a super-resolution head; the semantic segmentation head is used to obtain a semantic segmentation result according to the semantic segmentation features; the super-resolution head is used to obtain a super-resolution result according to the semantic segmentation features.

6. The method for detecting failure forms of important components on the surface of a train according to claim 1, characterized in that: The comprehensive loss function is as follows: ， in, is the comprehensive loss function, is the semantic segmentation loss, To enhance semantic segmentation loss, To enhance the super-resolution loss, is the super-resolution loss, is the boundary constraint segmentation loss weight, is the boundary constrained segmentation loss.

7. The method for detecting failure forms of important components on the surface of a train according to claim 1, characterized in that: Step S4 specifically includes: S41: preprocessing the image to be tested to obtain a picture of the bolt area to be tested, a number of the bolt to be tested, coordinates of the bolt to be tested, and an initial rotation angle of the bolt to be tested; S42: Using the trained component segmentation model to detect the image of the bolt area to be tested, to obtain a bolt segmentation image.

8. The method for detecting failure forms of important components on the surface of a train according to claim 1, characterized in that: Step S5 specifically includes: S51: performing contour extraction and polygon fitting according to the bolt segmentation image to obtain a regular hexagon; S52: Calculating the angle according to the regular hexagon to obtain the bolt rotation angle; S53: Obtaining a detection result according to the bolt rotation angle, the initial rotation angle of the bolt to be tested and a preset angle threshold.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method for detecting failure forms of important surface components of a train as described in any one of claims 1-8 are implemented.

10. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method for detecting failure forms of important components on the surface of a train as described in any one of claims 1-8 are implemented.

Images