Intelligent detection system and device for wheel set tread damage of heavy-load train

Abstract

The invention discloses an intelligent detection system and device for wheel set tread damage of a heavy-load train, and relates to the technical field of train detection. The system comprises an image acquisition module, a preprocessing module, a tread defect detection module and an intelligent terminal display interface module. The device comprises an industrial personal computer and a wheel set tread defect detection device. The image acquisition module carries out fixed-point shooting through the symmetrically arranged array vision subsystems, and a tread area is pre-extracted through ROI cutting; and the preprocessing module carries out light reflection removing operation, and image splicing is realized in combination with an SIFT algorithm and weighted fusion. In tread defect detection, a UGate-MSNet model is constructed to segment a defect area, and feature extraction is enhanced through GCM and MSFM modules; defects are accurately positioned through a DV-YOLO model, and the multi-scale feature fusion capability is improved through a double-branch structure and an XX-FPN module. The intelligent terminal realizes detection result visualization and alarm, the device does not need train stopping or deceleration, and the dynamic and high-precision detection requirements of heavy haul train safety operation and maintenance are met.

Description

Technical Field

[0001] This invention relates to the field of train inspection technology, and in particular to an intelligent detection system and device for wheel tread damage of heavy-duty trains. Background Technology

[0002] As a critical component of rail vehicles, the health of wheels directly affects the safety and stability of train operation. Under complex wheel-rail interactions, wheel treads are prone to various types of damage, such as flat spots, peeling, pitting wear, and polygonal wear. These defects can cause severe vibrations and noise, and even threaten train safety. To address this challenge, wheel inspection technology has evolved from early manual static measurement to sensor-based dynamic monitoring methods. This aims to achieve real-time and accurate perception of wheel damage, thereby providing technical support for safe train operation and intelligent maintenance.

[0003] Existing technologies still have problems: they cannot achieve dynamic, online detection; some ultrasonic flaw detection requires the vehicle to be stopped or its speed significantly reduced before the wheels can be removed, which seriously disrupts normal transportation order, resulting in extremely low detection efficiency and making it impossible to achieve routine preventive maintenance; image imaging is difficult, and it is greatly affected by lighting and vibration; the physical characteristics of defects, such as their extremely small size, weak appearance features, and high morphological diversity, make identification difficult and result in low accuracy. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to provide a high-precision and high-efficiency intelligent detection system and device for wheelset tread damage of heavy-duty trains.

[0005] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is as follows:

[0006] A key component of an intelligent detection system for wheelset tread damage in heavy-duty trains is an image acquisition module, an image preprocessing module, a tread defect detection module, and an intelligent terminal display interface.

[0007] The image acquisition module acquires wheelset images and transmits them to the image preprocessing module. The image preprocessing module then inputs the preprocessed images into the tread defect detection module to obtain defect information. The smart terminal display interface acquires the preprocessed tread image and defect information and displays them visually.

[0008] The tread defect detection module constructs the UGate-MSNet model and segments the tread defect region using a tread defect region dataset.

[0009] Furthermore, a DV-YOLO model and a tread defect detection dataset were constructed to accurately detect and locate defects.

[0010] Preferably, the image acquisition module captures the tread image at a fixed point using a camera, obtains the train wheel tread image with the same image position and pixel size as the train, pre-extracts the tread area using the ROI image automatic cropping algorithm, and inputs the tread image into the image preprocessing module.

[0011] The image preprocessing module performs anti-reflection processing and image stitching on the tread image;

[0012] The de-reflective treatment includes the following steps:

[0013] First, the image is converted from RGB color space to HSV color space. Then, Retinex multi-scale Gaussian filtering is used to obtain the illumination components. Next, a two-dimensional Gamma function is used to correct the brightness of the V component in the HSV color space of the original image. Finally, the HSV color space is converted back to RGB color space to obtain the image after de-reflection.

[0014] The image stitching process stitches together the same wheel tread image from different parts captured by multiple cameras in an array vision subsystem to obtain a complete wheel tread image.

[0015] The specific steps are as follows:

[0016] (1) Use the SIFT algorithm to extract image feature points;

[0017] (2) Feature matching and BBF algorithm are used to search for features, and preliminary screening is performed based on the ratio of nearest neighbor to second nearest neighbor distance;

[0018] (3) Use the RANSAC algorithm to estimate the geometric parameter transformation between the images to be stitched and then perform image stitching.

[0019] (4) Image fusion is achieved by using a weighted average method to eliminate stitching marks.

[0020] Preferably, the UGate-MSNet model is based on the U-Net model and consists of two parts: a decoder and an encoder. First, the PatchEmbedding module divides the original image into 4*4 patches. Then, the GCM module initially extracts the feature representation of the tread defect region. Subsequently, MSFM extracts rich tread defect feature representations layer by layer. The features at each scale are downsampled with low information loss by the PatchMerge layer. The decoder part decodes the encoded features layer by layer using the GCM module at each scale. PatchExpand upsampling is used between modules for scale reconstruction, and encoder features are layered for information compensation. Finally, a binary tread defect region segmentation result is obtained through a fully connected segmentation head.

[0021] Then, a loss function is constructed for this model, and the specific steps are as follows:

[0022] Based on cross-entropy loss, the formula is as follows:

[0023]

[0024] In the formula: N represents the number of pixels, and C represents the number of categories. This represents the probability that the i-th pixel is predicted to be of category c; Indicate whether the i-th pixel belongs to category c;

[0025] The Lovász extension based on the submodule function extends the submodule function into a piecewise linear convex function, which serves as a convex surrogate for the Jaccard loss, making the originally non-convex IoU loss convexly optimizeable in the continuous domain.

[0026] The Lovasz extension of set functions is as follows:

[0027]

[0028] The Lovasz loss function, constructed through Lovasz extension for smooth continuation, is as follows:

[0029]

[0030] Where ∆ is a set function, and i is the i-th pixel. The constructed error vector is formulated as follows:

[0031]

[0032] The value of the i-th pixel is used to determine whether it is the true label of class C. Predict the probability that the i-th pixel belongs to the c-th class;

[0033] The formula is as follows:

[0034]

[0035] Where π represents the descending order of the elements of m.

[0036] A hybrid loss function combining joint cross-entropy loss and Lovasz loss was designed for the segmentation head, as shown in the following formula:

[0037]

[0038] Where k is the adjustment factor.

[0039] Preferably, the GCM module first performs layer normalization on the input features, then divides them into two branches. Each branch uses a Linear layer to map the features to a high-dimensional space. The first high-dimensional branch uses parallel deep convolutions to extract multi-scale features, with kernel sizes designed to be 7, 5, and 3 respectively. The feature maps of the parallel branches are added together, then subjected to a SiLU nonlinear activation and layer normalization, while skip connections are added to the features, resulting in the output of this branch. Meanwhile, the other high-dimensional branch is activated using only the SiLU activation function. Element-wise multiplication is performed on the high-dimensional features of the two branches to obtain the implicit high-dimensional features, which are then mapped back to the original dimension using a Linear layer and added to the input features to obtain the final output.

[0040] Preferably, the MSFM module first extracts multi-scale feature information from the input features through three GCM modules, including GCM-S, GCM-M, and GCM-L. The three GCM modules are configured with different convolution kernel sizes: the kernel size of GCM-S is set to [9,7,5], the kernel size of GCM-M is set to [15,13,11], and the kernel size of GCM-L is set to [21,19,17]. Then, the extracted features at the three scales are input into the improved SE attention SE-LIP module for further feature selection.

[0041] The SE-LIP module calculates spatial attention and reverse spatial attention for the input features. Spatial attention calculation first uses global average pooling in the channel dimension to aggregate the input feature map into a single channel, then performs a 15×15 convolution and a sigmoid function to generate a pixel-level attention map Af. Reverse spatial attention is calculated using the same processing as spatial attention, followed by a "1-" operation to generate a reverse attention map Ar. Finally, the forward and reverse attention weights are globally normalized to obtain their global importance weights. and The expression is:

[0042]

[0043] Where Ω represents the global domain taken;

[0044] Furthermore, the input features are weighted by global importance weights and then subjected to weighted average pooling. The resulting forward and reverse pooled features are shown below:

[0045]

[0046]

[0047] in This is a positive pooling feature. Forward pooling features; finally, the output feature is obtained by summing the forward and inverse pooling features, i.e., F. out = Here, it means adding point by point; finally, for the obtained spatial compression feature F out The input is then fed into an MLP with a SiLU activation function to compress and activate channel information. After passing through a Sigmoid function, the final channel attention is obtained. The attention is multiplied by the input features to obtain the output of the final SE-LIP module.

[0048] Preferably, the construction of the DV-YOLO model and the tread defect detection dataset enables accurate defect detection and localization;

[0049] The DV-YOLO model uses YOLOv5 as its basic detection framework, including the backbone, feature extraction branches, and the XX-FPN module.

[0050] The main branch and branch sections encode the input image. The main branch performs feature extraction and dimensionality compression on the image through a downsampling module, and a hybrid attention feature fusion module is added between the downsampling modules. The dual convolution module in the feature extraction branch dynamically generates non-shared attention weights for each receptive field sliding window, emphasizing the importance of key regions in the feature map and extracting feature information. Then, these feature information are fused with the feature information extracted from the main branch through the hybrid attention feature fusion module. Finally, the fused information is reassembled with the branch and the XX-FPN module.

[0051] Preferably, the XX-FPN module includes an upsampling module, a feature fusion node, and a C3 module; the XX-FPN module encodes image features and receives multi-dimensional features output by the hybrid attention feature fusion module in the backbone, including the lowest-dimensional features, intermediate-dimensional features, and highest-dimensional features; by upsampling the high-dimensional features and downsampling the low-dimensional features, the obtained features are then spliced ​​together in multiple paths and at multiple levels to obtain multi-scale features.

[0052] A smart detection device for wheelset tread damage in heavy-duty trains, the device being used in the aforementioned detection system, the key feature of which is...

[0053] The device includes an industrial control computer and a wheelset tread defect detection device;

[0054] The wheelset tread defect detection device acquires wheelset image data and inputs it into the industrial control computer;

[0055] The wheelset tread defect detection device includes four sets of array vision subsystems arranged symmetrically on both sides of the rail. Each array vision subsystem includes an imaging device and a control device.

[0056] The shooting device includes a camera, a compensation light source, and a triggering device;

[0057] Two sets of array vision subsystems located on the same side of the rail, used to photograph the front and rear wheels of the same bogie of the train, are arranged symmetrically. The distance between the first and second array vision subsystems on the same side is determined by the bogie length and pitch angle.

[0058] The array vision subsystem includes multiple sets of imaging devices, which are evenly spaced along the direction of the railway track. In the first set of array vision subsystems, the imaging devices are arranged from the near end to the far end relative to the second set of array vision subsystems, with a triggering device, a camera and a compensation light source in sequence.

[0059] The triggering device, camera, and compensation light source in the shooting device are respectively connected to the control device;

[0060] A set of array vision subsystems includes a control device; the imaging device is connected to an industrial control computer;

[0061] The triggering device uses a PNP three-wire proximity sensor.

[0062] This invention achieves complete acquisition of the tread area image and removal of luminescence by using image preprocessing and a fixed camera in the image acquisition device, resulting in a complete and clear image.

[0063] This invention proposes the UGate-MSNet model, which optimizes the U-shaped structure to allocate new network depth, designs a gated convolutional module (GCM) to enhance feature extraction capabilities, constructs a multi-scale fusion module (MSFM) to capture global contextual information, and adopts the Lovasz loss function to address the problem of extreme sample imbalance. This achieves pixel-level high-precision defect region segmentation and improves the efficiency and accuracy of the system in detecting damage.

[0064] This invention proposes a DV-YOLO model, which provides complementary features through a dual-branch structure, improves the FPN structure, proposes an XX-FPN structure, and achieves adaptive multi-scale fusion. At the same time, it introduces a hybrid attention feature fusion module and a dual convolution module (DualConv-CA) to achieve accurate localization and classification of minute defects, thereby improving the accuracy of damage detection. Attached Figure Description

[0065] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments.

[0066] Figure 1 This is a partial schematic diagram of the tread region segmentation dataset constructed by this invention;

[0067] Figure 2 This is a structural diagram of the UGate-MSNet model proposed in this invention;

[0068] Figure 3 This is a structural optimization diagram of the U-shaped structure according to the present invention;

[0069] Figure 4 This is a structural diagram of the GCM module proposed in this invention;

[0070] Figure 5 This is a structural diagram of the MSFM module proposed in this invention;

[0071] Figure 6 This is a partial schematic diagram of the tread defect detection dataset constructed by this invention;

[0072] Figure 7 This is a structural diagram of the DV-YOLO model proposed in this invention;

[0073] Figure 8 This is a schematic diagram of the structure of the XX-FPN module proposed in this invention;

[0074] Figure 9 This is a structural diagram of the hybrid attention feature fusion module proposed in this invention;

[0075] Figure 10 This is a schematic diagram of the structure of the dual convolution module proposed in this invention;

[0076] Figure 11 This is a schematic diagram of the wheel tread defect detection device in this invention;

[0077] Figure 12 This is a schematic diagram of the display interface of the smart terminal in this invention;

[0078] Figure 13 This is a flowchart illustrating an intelligent detection system for wheelset tread damage of heavy-duty trains proposed in this invention. Detailed Implementation

[0079] To make the above-mentioned objectives, features, and advantages of the present invention more apparent and understandable, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings and specific implementation methods. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0080] A smart detection system for wheelset tread damage in heavy-haul trains includes an image acquisition module, an image preprocessing module, a tread defect detection module, and a smart terminal display interface; such as Figure 13The system obtains the tread area image by segmenting the input image, performs defect detection, and obtains the defect location and defect type. The results are then visualized through a smart terminal display interface.

[0081] After the image acquisition module acquires the wheelset image, it transmits it to the image preprocessing module. The image preprocessing module inputs the preprocessed image into the tread defect detection module to obtain defect information. The smart terminal display interface obtains the preprocessed tread image and defect information and displays it visually.

[0082] In the tread defect detection module, the UGate-MSNet model is constructed and the tread defect region dataset is used to segment the tread defect region.

[0083] Furthermore, a DV-YOLO model and a tread defect detection dataset were constructed to accurately detect and locate defects.

[0084] The image preprocessing module performs anti-reflection processing and image stitching on the tread image;

[0085] As train speeds increase, area scan cameras require stronger light sources to obtain clear images. However, due to the characteristics of the metal tread surface, images may exhibit uneven lighting and reflections. These issues affect image contrast and sharpness, making defects difficult to identify and detect. Therefore, an adaptive image correction algorithm based on a two-dimensional gamma function for uneven lighting is used to de-reflect the images, including the following steps:

[0086] First, the illumination component is extracted from the image using a multi-scale Gaussian function. Then, a brightness correction function based on a two-dimensional gamma function is used to adaptively adjust the function parameters to reduce the brightness value of high-brightness areas and increase the brightness value of dark areas, thereby correcting the uneven illumination of the image.

[0087] Real images captured by vision devices contain both incident and reflected light components. A multi-scale Gaussian function-based illumination component extraction method, based on Retinex theory, can accurately predict the illumination components within a small dynamic range. Therefore, this paper employs this method to extract the illumination components from images, which can correct for uneven illumination while preserving the image's detailed features.

[0088] This invention assigns multiple values ​​to the scale factor, uses a multi-scale Gaussian function to extract different illumination components in the image, and performs a weighted summation to obtain illumination information of global and local features in the image.

[0089] After acquiring the illumination components of an image, the image brightness is corrected based on a two-dimensional gamma function to balance the global and local brightness characteristics. Specifically, this paper adaptively adjusts the parameters of the two-dimensional gamma function based on the distribution characteristics of the extracted illumination components. When the illumination value at a certain point in the image is lower than the global average illumination value, the two-dimensional gamma function exponentially increases the brightness value of that area; conversely, when the illumination value is higher than the global average, the function exponentially decreases the brightness value. This adaptive adjustment process helps ensure that the image maintains appropriate brightness under different illumination conditions, improving the consistency of visual perception. This method not only considers the global illumination distribution but also focuses on changes in local brightness, thus more comprehensively optimizing the brightness effect of the image.

[0090] To obtain more comprehensive vehicle operating status information, image stitching combines images of the same wheel tread captured by multiple cameras in the array vision subsystem from different parts to obtain a complete wheel tread image.

[0091] The specific steps are as follows:

[0092] (1) Use the SIFT algorithm to extract image feature points.

[0093] (2) Feature matching search is performed using K-dtree and BBF algorithm, and preliminary screening is performed based on the ratio of nearest neighbor to second nearest neighbor distance.

[0094] (3) Use the RANSAC algorithm to estimate the geometric parameter transformation between the images to be stitched and then stitch the images together.

[0095] (4) Image fusion is achieved by using a weighted average method to eliminate stitching marks.

[0096] Specifically, the SIFT algorithm consists of the following steps: ① Scale-space extremum detection; ② Keypoint localization; ③ Keypoint orientation assignment; ④ Keypoint feature description. Scale-space extremum detection first constructs the scale space of the image using a Gaussian kernel function, forming a Gaussian pyramid by progressively downsampling the initial image. Then, a Gaussian difference pyramid is obtained by performing a difference operation on adjacent pyramid layers. In each scale space of the difference pyramid, detected local extrema are considered potential keypoints.

[0097] Keypoint localization first compares the detected local keypoints with all neighboring pixels in scale space to achieve initial screening. Considering the deviation of extreme points in discrete and continuous spaces, the DoG (Difference of Gaussian) function is used for curve fitting to calculate the offset, thereby obtaining more accurate keypoint localization. Finally, keypoints with low contrast and high edge responses are eliminated by setting thresholds and calculating edge response values ​​using the H (Hessian) matrix.

[0098] Keypoint orientation assignment involves calculating and comparing the gradient magnitudes of pixels surrounding the keypoint, selecting the orientation of the peak pixel as the keypoint orientation. Keypoint descriptor generation divides the pixels near the keypoint into 16 regions and calculates the gradient magnitude and orientation of each sub-region. The resulting vector is called the keypoint descriptor.

[0099] After finding similar feature points, geometric transformations of the images need to be calculated to align identical positions in the same scene across different images. Common transformations include translation, rotation, and scaling. Feature point matching applies these geometric transformations to align the found similar feature points, ensuring their coordinates are consistent in the stitched image. First, feature point matching is performed on the constructed feature point set using the Kd-tree algorithm, then the RANSAC algorithm is used to remove incorrectly matched feature points. After obtaining the feature matching points, a weighted average fusion stitching method is used to stitch the two tread surface images to be stitched together.

[0100] The tread defect detection module performs defect detection on the input image. Given the severe damage to the treads of heavy-haul freight trains, the significant background interference, complex shapes, and wide coverage of defects, existing target detection algorithms suffer from high false negative rates for large-scale, interconnected defects. Heavy-haul train wheel treads not only have the high reflectivity due to their smooth surface, but their surface defects also exhibit weak defect features and irregular scale variations. Furthermore, for heavy-haul train wheel tread defect detection, it is necessary not only to refine the types of surface defects but also to adapt to different types and materials of wheels. This invention constructs the UGate-MSNet model and a tread defect region dataset in the tread defect detection module to segment the tread defect regions.

[0101] like Figure 1 As shown, a tread region segmentation dataset is constructed, consisting of the original image in the top row and the labeled ground truth values ​​of the tread region in the bottom row.

[0102] Given the high aspect ratio and low contrast of tread defect areas, effective capture of spatial correlation and subtle features is key to achieving complete segmentation of tread defect areas.

[0103] like Figure 2As shown, the UGate-MSNet model proposed in this invention is based on the U-Net model and designs a gated convolutional module as the core encoding unit. It enhances the model's representational ability by introducing a gating mechanism and the idea of ​​reparameterization. Furthermore, it designs gated convolutional modules of different scales based on the gated convolutional module to fuse multi-scale features, and relies on large-size convolutional kernels to significantly expand the effective receptive field of the model, so as to fully model the non-local correlation of the tread region in the entire image. At the same time, it uses components such as layer normalization, PatchEmbedding, PatchMerge, PatchExpand layers and SiLU activation function to improve model performance.

[0104] The UGate-MSNet model mainly consists of two parts: an encoder and a decoder. The encoder abstracts and models the input image. First, the PatchEmbedding module divides the original image into 4x4 patches. Then, the GCM module initially extracts the feature representation of the tread defect region. Next, MSFM extracts richer tread defect feature representations layer by layer. Low-information-loss patch downsampling is performed between features at each scale using the PatchMerge layer. The decoder uses the GCM module to decode the encoded features layer by layer at each scale.

[0105] PatchExpand upsampling is used between modules for scale reconstruction, and encoder features are incorporated layer by layer for information compensation. Finally, a binary tread defect region segmentation result is obtained through a fully connected segmentation head.

[0106] The improvement process is as follows: First, the U-Net model is optimized:

[0107] like Figure 3 As shown, the original U-Net model uses a stack of convolutional layers, batch normalization layers, and ReLU activation functions to extract rich features and improve the model's representation ability. The UGate-MSNet model proposed in this invention uses a fast downsampling method at the beginning of the network. After inputting image data, it uses a PatchEmbedding layer based on stride convolution to extract low-level information simply and efficiently, with a stride set to 4. Furthermore, it changes the original convolution-batch normalization-ReLU design paradigm to a layer normalization-convolution-SiLU design structure, adding residual connections. PatchMerge and PatchExpand layers are used for downsampling and upsampling between feature map size transformations. The model's parameter settings are further optimized, abandoning the U-Net design of repeating the same number of layer stacks in each stage. The number of main layer stacks in stage 2 is changed from [2,2,2,2,2,2,2] in U-Net to [2,9,3,2,2].

[0108] Furthermore, the basic module for feature extraction in the UGate-MSNet model is equipped with a gated convolutional coding module (GCM) based on multi-branch parallelism.

[0109] Tread defect areas are often similar to the background and difficult to distinguish, which requires the model to have strong semantic feature extraction capabilities. Therefore, obtaining high-dimensional implicit representations will help identify tread defect areas. This invention proposes a gating mechanism based on high-dimensional feature nonlinear expression as the basic module for extracting tread defect area features.

[0110] like Figure 4 As shown, the GCM module first performs layer normalization on the input features, then divides them into two branches. Each branch uses a linear layer to map the features to a high-dimensional space. The first high-dimensional branch uses parallel deep convolutions to extract multi-scale features, with kernel sizes of 7, 5, and 3. The feature maps of the parallel branches are summed, followed by a SiLU nonlinear activation and layer normalization. Skip connections are added to the features to improve feature utilization, resulting in the output for this branch. Simultaneously, the other high-dimensional branch is activated using only the SiLU activation function. Element-wise multiplication is performed on the high-dimensional features of both branches to obtain the implicit high-dimensional features. Then, a linear layer is used to map them back to the original dimension, and the output is obtained by adding it to the input features.

[0111] Tread defect regions have a unique shape characterized by extremely long spans but very narrow widths, often covering most or the entire area of ​​an image. Accurate segmentation of these regions necessitates a larger receptive field. Classical convolutional neural networks typically use small 3×3 kernels to extract features, resulting in an extremely limited receptive field and an inability to extract non-local information. Even when the receptive field is expanded by stacking small kernels or increasing the dilation rate, the improvement in the actual effective receptive field remains insufficient. Therefore, this invention proposes the MSFM module, which constructs a global receptive field using large kernels and simultaneously employs kernels of various scales to capture tread defect region information from different receptive fields. Through an efficient fusion structure, features from different receptive fields are efficiently fused, improving the model's ability to identify tread defect regions at different scales.

[0112] Therefore, the base module for extracting features in the two downsampling modules with stack numbers of 9 and 3 is based on the original GCM and adopts the fusion module MSFM.

[0113] like Figure 5As shown, the MSFM module first extracts multi-scale feature information from the input features through three GCM modules. The three GCM modules are set with different convolution kernel sizes: the convolution kernel size of GCM-S is set to [9,7,5], the convolution kernel size of GCM-M is set to [15,13,11], and the convolution kernel size of GCM-L is set to [21,19,17]. Then, the three scale features extracted are input into the improved SE attention SE-LIP module for feature reselection.

[0114] The SE-LIP module computes spatial attention and reverse spatial attention for the input features separately. Spatial attention first uses channel-dimensional global average pooling to aggregate the input feature map into a single channel, then performs a 15×15 convolution and a sigmoid function to generate a pixel-level attention map. The reverse spatial attention is similar, except that it generates a reverse attention map through a "1-" operation. Furthermore, the positive and negative attention weights are globally normalized to obtain their global importance weights. and The expression is:

[0115]

[0116] Where Ω represents the global domain.

[0117] Furthermore, the input features are weighted by global importance weights and then subjected to weighted average pooling. The resulting forward and reverse pooled features are shown below:

[0118]

[0119]

[0120] Finally, the output feature is obtained by summing the features from the forward and reverse pooling, i.e., F. out = Here, it means adding point by point; finally, for the obtained spatial compression feature F out The input is then fed into an MLP with a SiLU activation function to compress and activate channel information. This information is then passed through a Sigmoid function to obtain the final channel attention. The attention is multiplied by the input features to obtain the output of the final SE-LIP module.

[0121] The loss function is constructed for the segmentation head portion, and the specific steps are as follows:

[0122] Based on cross-entropy loss, the formula is as follows:

[0123]

[0124] In the formula: N represents the number of pixels, and C represents the number of categories. This represents the probability that the i-th pixel is predicted to be of category c; Indicate whether the i-th pixel belongs to category c;

[0125] In the field of tread defect detection, the defective area often accounts for less than 10% of the total image pixels. Imbalanced training samples can lead to a decrease in recall, which is often more important than precision because a low recall rate indicates a high false negative rate, implying more operational oversights and greater engineering risks. Therefore, to suppress false negatives in tread defect areas and alleviate the imbalanced training sample problem, this invention proposes a loss penalty term emphasizing IoU. Based on the Lovász extension of the subset function, the subset function is extended into a piecewise linear convex function, serving as a convex surrogate for the Jaccard loss. This allows the originally non-convex IoU loss to be convexly optimized in the continuous domain.

[0126] The Lovasz extension of a set function is as follows:

[0127]

[0128] The Lovasz loss function, constructed through Lovasz extension for smooth continuation, is as follows:

[0129]

[0130] Where ∆ is a set function, and i is the i-th pixel. The constructed error vector is formulated as follows:

[0131]

[0132] The value of the i-th pixel is used to determine whether it is the true label of class C. Predict the probability that the i-th pixel belongs to the c-th class;

[0133] The formula is as follows:

[0134]

[0135] Here, π represents the descending order of the elements of m.

[0136] A hybrid loss function combining joint cross-entropy loss and Lovasz loss was designed for the segmentation head, as shown in the following formula:

[0137]

[0138] Where k is the adjustment factor, which is set to 0.5 in this invention. The Lovasz loss function is used to directly optimize the IoU index, giving higher attention weight to defective pixels. Its advantage lies in effectively mitigating the negative impact of class imbalance, significantly reducing the false negative rate, and avoiding the problem of model prediction bias towards the background.

[0139] Furthermore, a DV-YOLO model and a tread defect detection dataset were constructed to accurately detect and locate defects.

[0140] like Figure 6 As shown, a tread defect detection dataset is constructed, where the top row is the original image and the bottom row is the labeled true value of the tread area.

[0141] Given the small scale and weak grayscale differences of tread defects, effective capture of defect features is crucial for accurate localization and identification of tread defects. Therefore, the overall design process of this model is as follows: ① Using YOLOv5 as the basic detection skeleton; ② Designing a dual-branch structure, enhancing the model's representational capabilities through branch parallelism and dual-branch fusion; ③ Designing a novel XX-FPN structure to fuse multi-scale features; ④ Using dual-convolutional modules and hybrid attention modules to improve model performance.

[0142] like Figure 7 As shown, the DV-YOLO model, based on YOLOv5 as its basic detection framework, includes a backbone, a feature extraction branch, and an XX-FPN module. The backbone primarily utilizes Cross-Stage Local Connections (CSP) and multi-scale feature extraction to efficiently encode hierarchical visual features of the input image, providing rich multi-scale representations for subsequent object detection. The branch integrates with the backbone through a hybrid attention module, achieving dual-path complementarity in spatial details and multi-scale semantics, enhancing the localization accuracy of small targets. The XX-FPN part constructs a multi-scale feature pyramid through a top-down fusion path, enabling the network to simultaneously utilize high-level semantics and low-level details, effectively detecting targets at different scales.

[0143] The main branch and branches encode the input image. The main branch performs feature extraction and dimensionality compression on the image through a downsampling module, and a hybrid attention feature fusion module is added between the downsampling modules. The double convolution module in the feature extraction branch dynamically generates non-shared attention weights for each receptive field sliding window, emphasizing the importance of key regions in the feature map and extracting feature information. Then, these feature information are fused with the feature information extracted by the main branch through the hybrid attention feature fusion module. Finally, the fused information is reassembled with the branches and the XX-FPN module.

[0144] The specific structure is as follows: in the main body, the input information is downsampled, and the downsampling modules are connected through a hybrid attention feature fusion module. In the branches, the features of the input information are downsampled, and the downsampling in the branches uses a double convolution module. Each downsampling operation is connected through a feature fusion operation. Each hybrid attention feature fusion module receives feature maps of the same dimension from the main body and branches. Each feature fusion operation fuses feature maps of the same dimension from the main body and branches.

[0145] The XX-FPN module includes an upsampling module, a feature fusion node, and a C3 module. The XX-FPN module encodes image features and receives multi-dimensional features from the hybrid attention feature fusion module in the backbone, including lowest-dimensional, intermediate-dimensional, and highest-dimensional features. By upsampling the high-dimensional features and downsampling the low-dimensional features, the resulting features are then concatenated through multiple paths and levels to obtain multi-scale features. The structure of XX-FPN is as follows: Figure 8 As shown, light blue represents the lowest-dimensional features, and dark blue represents the highest-dimensional features extracted by the backbone network. Multi-level features are output by cross-splicing multi-level features, upsampling and downsampling after splicing, and then splicing again.

[0146] The core design of XX-FPN lies in its multi-path feature fusion mechanism, which allows features to interact and pass between different levels multiple times. Specifically, shallow features are upsampled to enhance their semantic information, while deep features are downsampled to refine their spatial details. This bidirectional operation ensures that features retain rich semantic information while maintaining accurate spatial localization capabilities during the fusion process.

[0147] Another key feature of XX-FPN is its adaptive feature weighting mechanism. During feature fusion, features at different levels are automatically weighted according to their importance to the current detection task. This adaptive mechanism allows the model to dynamically adjust its attention to features at different levels, making it more flexible and effective in handling defects of different scales.

[0148] Furthermore, XX-FPN gradually constructs a more comprehensive and detailed feature representation through multiple interactions and fusions of multi-scale features. This gradual fusion of multi-scale features not only improves the model's ability to detect small-scale defects but also enhances the semantic understanding of large-scale defects, thereby improving the overall detection performance.

[0149] In summary, XX-FPN effectively addresses the limitations of traditional feature pyramid networks in industrial surface defect detection through multi-path feature fusion, adaptive feature weighting, and stepwise interaction of multi-scale features. Experimental results demonstrate that XX-FPN achieves excellent detection performance on multiple industrial defect datasets, validating its effectiveness and robustness in practical applications.

[0150] like Figure 9 As shown, the hybrid attention feature fusion module obtains information from the feature extraction branch and the backbone information. Input 1 comes from the C3 module of the backbone, and input 2 comes from the dual convolution module of the feature extraction branch. First, the information of input 1 and input 2 are fused by channel division and concatenation. Then, feature extraction is performed by ECA attention and SE attention respectively. The two feature maps are concatenated and then the spatial position information and channel relationship information in the feature map are extracted by the CA attention module. Finally, the feature map is multiplied with the input feature map and output.

[0151] The ECA attention module uses 1D convolution to achieve local cross-channel interaction, capturing the relationship between each channel and its neighboring channels. The kernel size k is adaptively determined to cover an appropriate channel range. The SE module first compresses the H×W×C features into a 1×1×C channel description vector through global average pooling. This is followed by a dimensionality reduction fully connected layer, ReLU, a dimensionality increase fully connected layer, and a sigmoid function to generate channel weights between 0 and 1. Finally, these weights are multiplied with the original features channel by channel to complete recalibration. The CA attention module aggregates features along the horizontal and vertical directions through two 1D global pooling operations, generating two direction-aware feature maps. These two feature maps are then concatenated and transformed by a 1×1 convolution to obtain an intermediate feature map. Next, the intermediate feature map is segmented and independently passed through two 1×1 convolutional layers to generate two attention maps. Finally, these two attention maps are multiplied with the input feature map to enhance the feature representation of the target object.

[0152] The original DualConv extracts local features and compresses computation by using parallel 3×3 and 1×1 convolutions. However, in complex scenes, using only 3×3 convolutions limits the receptive field and makes it difficult to capture long-distance spatial relationships. Therefore, as... Figure 10As shown, this invention proposes a dual-convolution module, DualConv-CA. Based on DualConv, it introduces a CA attention mechanism and a 5×5 convolution. First, the CA attention mechanism uses global information to assign appropriate weights to each channel, suppressing noise and highlighting key features. The additional 5×5 convolution expands the local receptive field from 3×3 to 5×5 without significantly increasing the number of parameters, thus capturing a wider range of spatial relationships. The parallel outputs of different convolution sizes are then summed, which strengthens the representation of important channels and enriches multi-scale details, enabling the module to maintain a lightweight design while possessing higher discriminative power and robustness.

[0153] The intelligent terminal display interface is the core of the system's human-computer interaction, integrating data visualization, alarm prompts, and historical query functions to intuitively present the test results. For example... Figure 12 The interface displays a clear tread image in real time after preprocessing such as de-reflection and stitching, and automatically highlights the location, type, and confidence level of identified defects with a highlighted box. Once a defect is detected, the interface immediately pops up an alarm window to notify maintenance personnel, and supports exporting and archiving the inspection results and reports. Its multi-view, interactive, and responsive design facilitates on-site or remote monitoring and decision-making by staff on industrial control computers or mobile terminals.

[0154] An intelligent detection device for wheelset tread damage of heavy-duty trains includes an industrial control computer and a wheelset tread defect detection device;

[0155] The wheelset tread defect detection device acquires wheelset image data and inputs it into the industrial control computer;

[0156] The wheelset tread defect detection device includes four array vision subsystems arranged symmetrically on both sides of the rail. Each array vision subsystem includes an imaging device and a control device. The imaging device includes a camera 1, a compensation light source 2, and a triggering device 3. Two array vision subsystems located on the same side of the rail are arranged symmetrically to capture images of the front and rear wheels on the same bogie of the train. The distance between the first and second array vision subsystems on the same side is determined by the bogie length and pitch angle.

[0157] The array vision subsystem includes multiple sets of imaging devices, which are evenly spaced along the railway track. In the first set of array vision subsystems, the imaging devices are arranged from near to far relative to the second set of array vision subsystems, with a trigger device 3, a camera 1, and a compensation light source 2. The trigger device 3, camera 1, and compensation light source 2 in the imaging devices are respectively connected to a control device. Each set of array vision subsystems includes one control device. The imaging devices are connected to an industrial control computer. The control device uses a PLC control device.

[0158] This device ensures the integrity of the wheelset tread image by using an array of imaging devices within the array vision subsystem, combined with segmented shooting. Image complementarity between multiple cameras guarantees the complete acquisition of the wheelset tread image. Considering the installation location and operating process of camera 1, an area array camera 1 is selected. The frame rate, pixel count, and number of cameras are determined based on the maximum speed of the freight train, the train wheel diameter, and the maximum field of view of the exposed tread portion. Lens interface and focal length are selected based on the camera 1 model and the working distance during shooting. Camera 1 and lens are a 24FPS, 5-megapixel camera with a 12mm lens. The compensation light source 2 uses a high-brightness LED strobe light. Camera 1 and compensation light source 2 are installed beside the rail at specific horizontal and vertical angles, not higher than the rail surface. Considering the outdoor working environment, the triggering device 3 uses a PNP type magnetic flux sensor. It is installed at a certain distance from camera 1, perpendicular to the rail surface, on the inside of the rail. When a wheel travels directly above it, triggering device 3 is activated, outputting a high-level signal. The spacing between the four array vision subsystems is determined by the wheel diameter and the maximum field of view captured by each camera 1, in order to ensure the continuity and integrity of the images captured by each camera 1.

[0159] In practical use, the cameras in the array vision unit face the unobstructed tread area, which is less than half of the entire tread circumference. Bench tests have verified that the arc shape of the tread results in blurry images at the top and bottom vertices of the wheel, with the clear tread area covering approximately one-third of the circumference. Theoretically, if one camera captures one-third of the circumference, three cameras would be sufficient to obtain a complete tread image. However, since the images captured by each camera have no overlap, ensuring image integrity requires high precision in the array device's installation spacing. Furthermore, device latency and uncontrollable factors on-site can lead to missed images in certain areas. Therefore, redundancy in the array device needs to be considered, ultimately determining that at least four cameras are used in each array vision unit.

[0160] In practical use, such as Figure 11When the train wheels reach a specific area, trigger device 3 is activated. The level signal generated by trigger device 3 is simultaneously sent to camera 1 and the light source via the PLC control device, synchronizing the shutter action of camera 1 with the strobe action of the compensation light source 2. Under the supplementary lighting of the LED strobe light, camera 1 successfully captures a portion of the tread image and transmits the acquired image to the industrial control computer for further processing. As the train continues to travel, when it reaches the sensor position of the second set of equipment, camera 1 in that set of equipment captures another portion of the tread image. This process continues until the train wheels pass through the last set of equipment. Freight trains typically use two-axle bogies, with two sets of array vision systems symmetrically arranged on the same side of the rail, used to photograph the front and rear wheels on the same bogie, respectively. Each array vision system is equipped with a PLC control device. Through PLC logic control, camera 1 and the light source only react to the trigger signal generated by the sensor when the wheelset to be photographed passes by.

[0161] Specifically, when the front wheels of the train bogie first pass through the array vision system, the PLC control device receives the trigger signal from trigger device 3 and ignores it. Camera 1 and compensation light source 2 are in a non-operating state at this time. When the train continues to travel and passes through the second array vision system, the PLC receives the trigger signal from the sensor and outputs it to camera 1 and the light source. Camera 1 and compensation light source 2 are then in an operating state. The triggering sequence of the rear wheels is the reverse of the front wheels; they operate when passing through the first array vision system and do not operate when passing through the second. This device achieves accurate imaging of the front and rear wheel treads, avoiding the need for subsequent screening of unusable images in each system. The PLC control device controls the synchronous triggering of the camera and light source; it controls each array vision unit to only receive the wheel trigger signal of the corresponding wheelset. The PLC sends the sensor trigger signal to both the camera and the light source simultaneously, achieving synchronous triggering and completing image acquisition under the illumination of the light source. Secondly, the PLC can perform initial image screening. If each array vision unit receives the trigger signals of the front and rear wheelsets indiscriminately, it increases the image storage burden and the difficulty of subsequent tread defect identification.

[0162] The above describes the entire process of intelligent detection of wheel tread damage for heavy-haul trains. The camera 1 in the array acquisition subsystem acquires images of the wheel tread surface and performs preprocessing. The images are then input into the image segmentation model proposed in this invention to extract the effective area of ​​the tread from the entire image. The images are then further input into the detection model constructed in the tread defect detection module to locate and detect tread defects in the effective area.

[0163] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. An intelligent detection system for wheelset tread damage in heavy-duty trains, characterized in that, It includes an image acquisition module, an image preprocessing module, a tread defect detection module, and a smart terminal display interface; The image acquisition module acquires wheelset images and transmits them to the image preprocessing module. The image preprocessing module then inputs the preprocessed images into the tread defect detection module to obtain defect information. The smart terminal display interface acquires the preprocessed tread image and defect information and displays them visually. The tread defect detection module constructs the UGate-MSNet model and segments the tread defect region using a tread defect region dataset. Furthermore, a DV-YOLO model and a tread defect detection dataset were constructed to accurately detect and locate defects.

2. The intelligent detection system for wheelset tread damage of heavy-duty trains according to claim 1, characterized in that, The image acquisition module captures tread images at fixed points using a camera, obtains train wheel tread images with the same image position and pixel size as the train, pre-extracts the tread area using an ROI image automatic cropping algorithm, and inputs the tread images into the image preprocessing module. The image preprocessing module performs anti-reflection processing and image stitching on the tread image; The de-reflective treatment includes the following steps: First, the image is converted from RGB color space to HSV color space. Then, Retinex multi-scale Gaussian filtering is used to obtain the illumination components. Next, a two-dimensional Gamma function is used to correct the brightness of the V component in the HSV color space of the original image. Finally, the HSV color space is converted back to RGB color space to obtain the image after de-reflection. The image stitching process stitches together the same wheel tread image from different parts captured by multiple cameras in an array vision subsystem to obtain a complete wheel tread image. The specific steps are as follows: (1) Use the SIFT algorithm to extract image feature points; (2) Feature matching and BBF algorithm are used to search for features, and preliminary screening is performed based on the ratio of nearest neighbor to second nearest neighbor distance; (3) Use the RANSAC algorithm to estimate the geometric parameter transformation between the images to be stitched and then perform image stitching. (4) Image fusion is achieved by using a weighted average method to eliminate stitching marks.

3. The intelligent detection system for wheelset tread damage of heavy-duty trains according to claim 1, characterized in that, The UGate-MSNet model is based on the U-Net model and consists of two parts: a decoder and an encoder. First, the PatchEmbedding module divides the original image into 4*4 patches. Then, the GCM module initially extracts the feature representation of the tread defect area. Subsequently, MSFM extracts rich tread defect feature representations layer by layer. The features at each scale are downsampled with low information loss by the PatchMerge layer. The decoder part uses the GCM module to decode the encoded features layer by layer at each scale, uses PatchExpand upsampling between modules for scale reconstruction, and incorporates encoder features layer by layer for information compensation. Finally, the binary tread defect region segmentation result is obtained through a fully connected segmentation head. Then, a loss function is constructed for this model, and the specific steps are as follows: Based on cross-entropy loss, the formula is as follows: ； In the formula: N represents the number of pixels, and C represents the number of categories. This represents the probability that the i-th pixel is predicted to be of category c; Indicate whether the i-th pixel belongs to category c; The Lovász extension based on the submodule function extends the submodule function into a piecewise linear convex function, which serves as a convex surrogate for the Jaccard loss, making the originally non-convex IoU loss convexly optimizeable in the continuous domain. The Lovasz extension of set functions is as follows: ； The Lovasz loss function, constructed through Lovasz extension for smooth continuation, is as follows: ； Where ∆ is a set function, and i is the i-th pixel. The constructed error vector is formulated as follows: ； The value of the i-th pixel is used to determine whether it is the true label of class C. Predict the probability that the i-th pixel belongs to the c-th class; The formula is as follows: ； Where π represents the elements of m arranged in descending order; A hybrid loss function combining joint cross-entropy loss and Lovasz loss was designed for the segmentation head, as shown in the following formula: ； Where k is the adjustment factor.

4. The intelligent detection system for wheelset tread damage of heavy-duty trains according to claim 3, characterized in that, The GCM module first performs layer normalization on the input features, then divides them into two branches. Each branch uses a Linear layer to map the features to a high-dimensional space. The first high-dimensional branch uses parallel deep convolutions to extract multi-scale features with kernel sizes of 7, 5, and 3. The feature maps of the parallel branches are summed, then subjected to a SiLU nonlinear activation and layer normalization, while skip connections are added to the features to obtain the result of this branch. Meanwhile, the other high-dimensional branch is activated using only the SiLU activation function. Element-wise multiplication is performed on the high-dimensional features of the two branches to obtain the implicit high-dimensional features, which are then mapped back to the original dimension using a Linear layer and added to the input features to obtain the final output.

5. The intelligent detection system for wheelset tread damage of heavy-duty trains according to claim 3, characterized in that, The MSFM module first extracts multi-scale feature information from the input features through three GCM modules: GCM-S, GCM-M, and GCM-L. The three GCM modules are configured with different convolution kernel sizes: GCM-S has a kernel size of [9,7,5], GCM-M has a kernel size of [15,13,11], and GCM-L has a kernel size of [21,19,17]. The extracted features at the three scales are then input into the improved SE attention SE-LIP module for further feature selection. The SE-LIP module calculates spatial attention and reverse spatial attention for the input features. Spatial attention calculation first uses channel-dimensional global average pooling to aggregate the input feature map into a single channel, then performs a 15×15 convolution and a sigmoid function to generate a pixel-level attention map. The reverse spatial attention, after undergoing the same processing as the spatial attention, generates a reverse attention map through a "1-" operation. Furthermore, the positive and negative attention weights are globally normalized to obtain their global importance weights. and The expression is: ； Where Ω represents the global domain taken; Furthermore, the input features are weighted by global importance weights and then subjected to weighted average pooling. The resulting forward and reverse pooled features are shown below: ； ； in This is a positive pooling feature. Forward pooling features; finally, the output feature is obtained by summing the forward and inverse pooling features, i.e., F. out = Here, it means adding point by point; finally, for the obtained spatial compression feature F out The input is then fed into an MLP with a SiLU activation function to compress and activate channel information. After passing through a Sigmoid function, the final channel attention is obtained. The attention is multiplied by the input features to obtain the output of the final SE-LIP module.

6. The intelligent detection system for wheelset tread damage of heavy-duty trains according to claim 1, characterized in that, The construction of the DV-YOLO model and the tread defect detection dataset enables accurate defect detection and localization. The DV-YOLO model uses YOLOv5 as its basic detection framework, including the backbone, feature extraction branches, and the XX-FPN module. The main and branch parts encode the input image. The main part performs feature extraction and dimensionality compression on the image through a downsampling module. A hybrid attention feature fusion module is added between the downsampling modules. The dual convolution module in the feature extraction branch dynamically generates non-shared attention weights for each receptive field sliding window, emphasizing the importance of key regions in the feature map and extracting feature information. Then, these feature information are fused with the feature information extracted from the main branch through the hybrid attention feature fusion module. Finally, the fused information is reassembled with the branch and the XX-FPN module.

7. The intelligent detection system for wheelset tread damage of heavy-duty trains according to claim 6, characterized in that, The XX-FPN module includes an upsampling module, a feature fusion node, and a C3 module; the XX-FPN module encodes image features and receives multi-dimensional features output by the hybrid attention feature fusion module in the backbone, including the lowest-dimensional feature, the intermediate-dimensional feature, and the highest-dimensional feature; By upsampling high-dimensional features and downsampling low-dimensional features, the resulting features are then spliced ​​together through multiple paths and levels to obtain multi-scale features.

8. An intelligent detection device for wheelset tread damage of heavy-duty trains, the device being used to implement the detection system as described in any one of claims 1-7, characterized in that, The device includes an industrial control computer and a wheelset tread defect detection device; The wheelset tread defect detection device acquires wheelset image data and inputs it into the industrial control computer; The wheelset tread defect detection device includes four sets of array vision subsystems arranged symmetrically on both sides of the rail. Each array vision subsystem includes an imaging device and a control device. The shooting device includes a camera (1), a compensation light source (2), and a triggering device (3); Two sets of array vision subsystems located on the same side of the rail, used to photograph the front and rear wheels of the same bogie of the train, are arranged symmetrically. The distance between the first and second array vision subsystems on the same side is determined by the bogie length and pitch angle. The array vision subsystem includes multiple sets of shooting devices, which are evenly spaced along the direction of the railway track. The shooting devices in the first set of array vision subsystem are arranged sequentially from the near end to the far end relative to the second set of array vision subsystem, including a trigger device (3), a camera (1), and a compensation light source (2). The trigger device (3), camera (1) and compensation light source (2) in the shooting device are respectively connected to the control device; A set of array vision subsystems includes a control device; the imaging device is connected to an industrial control computer; the triggering device (3) adopts a PNP three-wire proximity sensor.

Images