Lightweight railway fastener detection method for vehicle-mounted portable device

Abstract

The application discloses a kind of light weight railway fastener detection methods for vehicle-mounted portable device, it is related to computer vision technical field, including, railway fastener image data are acquired and are labeled and preprocessed, form the fastener image dataset for model training;Light weight fastener detection model based on mixed feature enhancement LeYOLO is constructed, and the high-frequency detail features of small target fastener are enhanced by SPPF-SE-C2PSA collaborative enhancement structure and high-frequency enhanced multi-scale prediction front feature detection head;In the training stage, introduce head-neck joint mixed knowledge distillation NH-KD, so that the teacher model guides student model training, obtain model weight file, and the weight file is loaded into detection model to realize railway fastener detection.The application can improve the detection accuracy and recall rate of small target fastener under complex track background under the premise of maintaining low inference overhead, and is suitable for low-power platforms such as vehicle-mounted portable device.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, and in particular to a lightweight railway fastener inspection method for vehicle-mounted portable devices. Background Technology

[0002] Fasteners are a crucial component of railway lines, playing a vital role in ensuring track structure stability and the safe operation of rail transit. Under long-term dynamic loads and complex track environments, fasteners are prone to abnormal conditions such as loosening, missing fasteners, breakage, or displacement. If fastener malfunctions are not detected in a timely manner, they can affect track stability to the point of causing equipment damage and posing safety hazards. In scenarios such as railway freight lines, marshalling yards, port railway sidings, and onboard inspections, the track operating environment is complex, inspection tasks are continuous, and real-time requirements are high, making it difficult to rely on high-performance servers for centralized processing. Therefore, real-time monitoring and accurate detection of railway fastener status is an important aspect of intelligent inspection of rail transit infrastructure.

[0003] Object detection methods based on deep convolutional neural networks have been widely applied to railway fastener inspection tasks, with single-stage detection models, such as YOLO, exhibiting high detection efficiency. To adapt to low-computing platforms such as onboard portable devices, existing methods typically reduce the number of parameters and computational cost by compressing the network structure. However, in real-world railway scenarios, fastener targets are small and subject to complex background interference; excessive model compression can weaken the ability to extract features from small targets, leading to missed detections and false detections. Furthermore, single attention enhancement methods struggle to simultaneously address channel semantic filtering and spatial target focusing, while single knowledge distillation methods also struggle to align intermediate features with the detection head's decision-making results. Summary of the Invention

[0004] In view of the aforementioned existing problems, the present invention is proposed.

[0005] To solve the above-mentioned technical problems, the present invention provides the following technical solution:

[0006] In a first aspect, the present invention provides a lightweight railway fastener detection method for vehicle-mounted portable devices, comprising: acquiring railway fastener image data, and annotating and preprocessing the railway fastener image data to form a fastener image dataset for model training;

[0007] A lightweight fastener detection model based on hybrid feature enhancement LeYOLO is constructed: multi-scale features of railway fastener images are extracted through a backbone network, and these multi-scale features are input into a lightweight neck feature fusion network LeNeck for multi-scale feature fusion; wherein, the backbone network is used to extract multi-scale features from the input railway fastener images and output... , , Feature layers at three scales; , , These represent the feature maps output at different downsampling stages of the backbone network; It is a deep semantic feature layer output at the end of the backbone network, which has strong high-level semantic expression capabilities;

[0008] The SPPF-SE-C2PSA collaborative enhancement structure is used to enhance the multi-scale contextual information, channel features and spatial target regions in the fastener image at the LeNeck end, reducing background noise interference. The high-frequency enhanced multi-scale prediction pre-feature detection head is used to enhance the high-frequency edge and texture information of small targets in the fastener, and the final result is output.

[0009] The lightweight fastener detection model is trained using a training dataset. During the training period, head-neck joint hybrid knowledge distillation is introduced, allowing the teacher model to guide the student model training through neck feature distillation and head cross-decision distillation, resulting in a trained model weight file.

[0010] The image of the railway fastener to be detected is input into a railway fastener detection model loaded with the model weight file, and the fastener detection result is output; the fastener detection result includes the fastener target position, confidence level, and fastener status category;

[0011] Based on the fastener inspection results, determine whether there are any abnormalities in the railway fasteners and output one or more inspection categories.

[0012] As a preferred embodiment of the lightweight railway fastener inspection method for vehicle-mounted portable devices described in this invention, wherein:

[0013] The SPPF-SE-C2PSA collaborative enhancement structure includes an SPPF-SE channel enhancement module and a C2PSA spatial attention module;

[0014] At LeNeck At the core node of the main confluence, a C2PSA spatial attention module is connected in series to achieve geometric target focusing;

[0015] At the end of the backbone network The layer introduces a spatial pyramid pooling SPPF-SE channel enhancement module with channel attention to perform multi-scale context aggregation and channel recalibration on the deep features output by the backbone network, thereby achieving semantic channel filtering;

[0016] Suppose that, given the input feature pooling aggregation result The PPF-SE channel enhancement module obtains channel descriptors through global average pooling. The channel reweighting vector is generated through a two-layer fully connected network. :

[0017]

[0018] in, It is the Sigmoid activation function. The ReLU activation function is used; the output after rescaling the features is... ; and These represent the weight matrices of the dimensionality-reducing and dimensionality-increasing fully connected layers, respectively. C is the channel scaling ratio; C is the number of channels in the feature map. This indicates a global average pooling operation.

[0019] As a preferred embodiment of the lightweight railway fastener inspection method for vehicle-mounted portable devices described in this invention, wherein:

[0020] The high-frequency enhanced multi-scale prediction pre-feature detection head includes a channel-preserving high-frequency sensing plugin (HFP) embedded in the front end of the multi-scale detection head, used to separate low-frequency background and high-frequency details of the input features.

[0021] The high-frequency sensing plugin HFP obtains low-frequency components through local average pooling and obtains high-frequency components through the residual between the input features and the low-frequency components. It combines the high-frequency components to generate channel gate weights and spatial gate weights, and outputs the input features in the form of residuals after adaptive reweighting, thereby enhancing the edge, texture and local defect features of small fastener targets.

[0022] As a preferred embodiment of the lightweight railway fastener inspection method for vehicle-mounted portable devices described in this invention, wherein:

[0023] The head-neck combined mixed knowledge distillation includes neck feature distillation and cross-head distillation;

[0024] The neck feature distillation and crosshead distillation are coordinated in stages using dynamic weights;

[0025] In the early stages of training, distillation constraints are reduced to stabilize the student model. In the later stages of training, feature alignment and decision alignment are enhanced to enable the student model to achieve synergistic optimization between neck semantic representation and head detection decision.

[0026] As a preferred embodiment of the lightweight railway fastener inspection method for vehicle-mounted portable devices described in this invention, wherein:

[0027] The step-by-step coordination of neck feature distillation and crosshead distillation using dynamic weights includes constructing a dynamic weight scheduling function as follows:

[0028]

[0029] in, For the current training round, For the preheating period, Maximum distillation weight;

[0030] The end-to-end training total loss function of the lightweight fastener detection model is defined as:

[0031]

[0032] in, This represents the baseline detection loss, which includes classification and bounding box regression. This represents the head-cross decision distillation loss; This indicates the characteristic distillation loss at the neck; This represents the dynamic weights corresponding to the CrossKD distillation loss in the t-th training round; This represents the dynamic weight corresponding to the HIKD distillation loss in the t-th training round.

[0033] As a preferred embodiment of the lightweight railway fastener inspection method for vehicle-mounted portable devices described in this invention, wherein:

[0034] The fastener status categories include four fastener statuses: normal, missing, broken, and displaced.

[0035] As a preferred embodiment of the lightweight railway fastener inspection method for vehicle-mounted portable devices described in this invention, wherein:

[0036] The detection categories include: if the output category is normal, it is determined that the fastener is not faulty; if the output category is missing, broken, or displaced, it is determined that the fastener is faulty.

[0037] Secondly, the present invention provides a lightweight railway fastener detection system for vehicle-mounted portable devices, comprising a training set construction unit for acquiring railway fastener image data and labeling and preprocessing the railway fastener image data to form a fastener image dataset for model training.

[0038] The model building unit constructs a lightweight fastener detection model based on hybrid feature enhancement LeYOLO;

[0039] The training unit uses the training dataset to train the lightweight fastener detection model. During the training period, head-neck joint hybrid knowledge distillation is introduced, so that the teacher model guides the student model training through neck feature distillation and head cross-decision distillation, and the trained model weight file is obtained.

[0040] The detection unit inputs the image of the railway fastener to be detected into a railway fastener detection model loaded with the model weight file, and outputs the fastener detection result; the fastener detection result includes the fastener target position, confidence level, and fastener status category; and, based on the fastener detection result, determines whether there is an anomaly in the railway fastener, and outputs one or more detection categories.

[0041] Thirdly, the present invention provides a computer device including a memory and a processor, wherein the memory stores a computer program, wherein when the computer program is executed by the processor, it implements any step of the lightweight railway fastener inspection method for vehicle-mounted portable devices as described in the first aspect of the present invention.

[0042] Fourthly, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein: when the computer program is executed by a processor, it implements any step of the lightweight railway fastener inspection method for vehicle-mounted portable devices as described in the first aspect of the present invention.

[0043] The beneficial effects of this invention are: (1) This invention is aimed at engineering application scenarios such as track inspection vehicles and freight line inspection. Based on the LeYOLO-Nano detection model, it can maintain a low number of parameters and computational load, reduce the model's dependence on high-performance GPUs and large-capacity storage devices, and is suitable for deployment on low-computing-power vehicle-mounted portable devices, meeting the requirements of on-site inspection for low computational load, low power consumption and high inference speed.

[0044] (2) The present invention combines multi-scale context aggregation, channel recalibration and spatial attention modeling through SPPF-SE-C2PSA collaborative enhancement structure, which can suppress the interference of factors such as complex track background, track bed debris, lighting changes and track vibration on the foreground features of fasteners and improve the detection stability during vehicle inspection.

[0045] (3) The present invention constructs a high-frequency enhanced multi-scale prediction pre-feature detection head to separate low-frequency background and high-frequency details, and uses channel gating and spatial gating to enhance the edge, texture and local defect information of small fastener targets, thereby reducing the probability of missed detection.

[0046] (4) This invention uses head-neck joint hybrid knowledge distillation NH-KD to transfer the feature relationships and prediction results of the teacher model to the student model during the training phase. The distillation structure is only used during the training phase, and only the weight file of the student model is loaded during the inference phase. There is no need to load the teacher model and distillation branch, so there is no additional inference overhead. It can balance low power consumption deployment, high inference speed and detection accuracy in complex circuit scenarios. Attached Figure Description

[0047] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0048] Figure 1 This is a flowchart of a method for testing lightweight railway fasteners for portable vehicle-mounted devices.

[0049] Figure 2 The image shows the test results of a testing method for lightweight railway fasteners designed for portable vehicle-mounted devices.

[0050] Figure 3 The image shows the EigenCAM feature visualization of a lightweight railway fastener inspection method for portable on-vehicle devices. (a) is the original image of the fastener, and (b) is the heatmap visualization result of the feature detection head before high-frequency enhanced multi-scale prediction. Detailed Implementation

[0051] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0052] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0053] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in different places in this specification does not necessarily refer to the same embodiment, nor is it a single or selective embodiment that is mutually exclusive with other embodiments.

[0054] Example 1, referring to Figure 1 This is one embodiment of the present invention, which provides a method for inspecting lightweight railway fasteners for portable vehicle-mounted devices, including the following steps:

[0055] S1: Acquire railway fastener image data, and annotate and preprocess the railway fastener image data to form a fastener image dataset for model training.

[0056] Furthermore, the railway fastener images are subjected to size standardization, format conversion, and data augmentation processing, and are divided into training set, validation set, and test set.

[0057] S2: Construct a lightweight fastener detection model based on LeYOLO enhanced with hybrid features.

[0058] LeYOLO stands for Lightweight YOLO, a real-time object detection model that stands for You Only Look Once.

[0059] The lightweight fastener detection model includes: extracting multi-scale features from railway fastener images through a backbone network, and inputting the multi-scale features into the lightweight neck feature fusion network LeNeck for multi-scale feature fusion.

[0060] The backbone network is used to extract multi-scale features from the input railway fastener images and output the results. , , Feature layers at three scales; , , These represent the feature maps output at different downsampling stages of the backbone network, where... As a deep semantic feature layer output at the end of the backbone network, it has strong high-level semantic expression capabilities. Compared with introducing shallower or deeper features, this three-scale output method can reduce redundant calculations while ensuring the detection accuracy of small fastener targets, and is suitable for deployment on low-computing-power platforms such as vehicle-mounted portable devices.

[0061] Furthermore, LeNeck is a lightweight neck feature fusion network positioned between the backbone network and the detection head. It receives multi-scale feature layers from the backbone network output and fuses features at different resolutions and semantic levels to enhance the detail and semantic representation of small fastener targets with lower computational overhead. The fusion result serves as input to the subsequent SPPF-SE-C2PSA collaborative enhancement structure and the detection head, thereby improving the detection accuracy of fastener targets in complex orbital backgrounds.

[0062] S3: The SPPF-SE-C2PSA collaborative enhancement structure is used to enhance the multi-scale contextual information, channel features and spatial target regions in the fastener image at the LeNeck end, reducing background noise interference. (The enhanced feature map is output from LeNeck and enters the high-frequency enhanced multi-scale prediction pre-feature detection head.) The high-frequency enhanced multi-scale prediction pre-feature detection head is used to enhance the high-frequency edge and texture information of the small target of the fastener, and the final result is output.

[0063] Furthermore, the SPPF-SE-C2PSA structure includes a channel enhancement module SPPF-SE that integrates Spatial Pyramid Pooling - Fast (SPPF) and Squeeze-and-Excitation (SE) channel enhancement modules, and a cross-stage partial spatial attention (C2PSA) module for spatially focusing the fastener target region in multi-scale fused features, enhancing the target region response, and suppressing background region interference.

[0064] Specifically, in this embodiment, the ultra-lightweight YOLO detection model LeYOLO-Nano is used as the baseline for fastener detection. LeYOLO-Nano is a lightweight YOLO target detection model designed for deployment scenarios with low computing power and portable devices.

[0065] Given a three-channel image with each channel having a two-dimensional feature size of H rows and W columns. backbone network Multi-scale features are extracted based on the inverted bottleneck structure. To minimize computational latency, early redundant pointwise convolutions are removed. The feature extraction steps are as follows:

[0066] 1. Features are fed into the lightweight neck network Le The structure is based on The main integration channel greatly reduces and Calculation frequency:

[0067]

[0068] in, Corresponding to downsampling The feature map.

[0069] At the later main bus node of LeNeck, with The intermediate-scale feature layer is the main fusion channel, which will contain features from... Detailed features and from The deep semantic features are fused, and a C2PSA spatial attention module is connected in series after the fusion node to spatially focus the fastener target area in the fused features, enhance the target response and suppress background interference.

[0070] 2. Enter the SPPF-SE module. Given the input feature pooling aggregation result. This module extracts channel descriptors through Global Average Pooling (GAP). The channel reweighting vector is generated through a two-layer fully connected network. :

[0071]

[0072] in It is the Sigmoid activation function. The ReLU activation function is used. The output after feature rescaling is... and These represent the weight matrices of the dimensionality reduction and dimensionality increase fully connected layers, respectively; r is the channel scaling ratio (here). (Used to characterize the dimensionality reduction ratio). This represents the features after SPPF multi-scale pooling aggregation; This represents the channel descriptor obtained through global average pooling; This represents the channel reweighting vector; C is the number of channels in the feature map.

[0073] 3. Feature map enters At the core node of the main confluence, spatial attention is enhanced by the C2PSA spatial attention module.

[0074] 4. Enter the high-frequency enhanced multi-scale prediction pre-feature detection head.

[0075] High-frequency features are further enhanced using high-frequency feature enhancement before the detection head. The high-frequency feature enhancement steps are as follows:

[0076] Let the input features be First, local average pooling is used to extract low-frequency components. High-frequency characteristics are calculated in the form of residuals.

[0077]

[0078]

[0079] in, This is the pooling kernel size for the low-pass filter; the default value is... . Represents high-frequency components. This represents a local average pooling operation; s represents the pooling step size. Indicates zero padding size. This indicates rounding down to the nearest integer.

[0080] Next, in high-frequency features The channel gating weights are generated respectively. Spatial gating weights .

[0081] For channel gating, firstly, high-frequency components... Perform global average pooling and global max pooling respectively, and then add them together to obtain the high-frequency channel descriptor. :

[0082] Then, Input channel scaling ratio is A two-layer fully connected network is used to generate channel gating weights. :

[0083] in, Represents the high-frequency channel descriptor; This indicates a global average pooling operation; This represents the global max pooling operation; Indicates the channel gating weight; Represents the ReLU activation function; This represents the Sigmoid activation function; This represents the dimension-reduced weight matrix in the channel-gated branch. This represents the weight matrix of the higher dimension in the channel-gated branch; Indicates the number of input feature channels. Indicates the channel scaling ratio.

[0084] For spatial gating, high-frequency components are gated along the channel dimension. Calculate mean pooling and max pooling respectively to generate a two-dimensional saliency map containing high-frequency spatial distribution. Spatial dependencies are extracted through two-dimensional convolution:

[0085] in, Represents a high-frequency spatial saliency map; This indicates that the mean is calculated along the channel dimension; This indicates that the maximum value is calculated along the channel dimension; Indicates feature concatenation operation; Indicates the kernel size as Two-dimensional convolution, Indicates the size of the spatial convolution kernel; This represents the spatial gating weight.

[0086] Finally, the generated channel gating weights are used. and spatial gating weights For the original input features Perform adaptive reweighting and output the results as residuals:

[0087] in, This indicates the enhanced features output by the High Frequency Sensing Plug-in (HFP). This represents element-wise multiplication; because and The dimensions are respectively and In relation to When multiplying, the result is extended to the product through a broadcast mechanism. Same dimensions; + This indicates a residual connection.

[0088] S4: The lightweight fastener detection model is trained using the training dataset. During the training period, head-neck joint hybrid knowledge distillation is introduced, so that the teacher model guides the student model training through neck feature distillation and head cross-decision distillation, and the trained model weight file is obtained.

[0089] During the training period, the improved lightweight fastener detection model was used as the student model, and the improved lightweight fastener detection model was used as the teacher model. The head-neck joint hybrid knowledge distillation (NH-KD) was introduced to guide the training of the student model.

[0090] The NH-KD includes dynamic weight scheduling, neck feature distillation HIKD, and head cross decision distillation CrossKD.

[0091] HIKD is used to align the feature relationships between the teacher model and the student model in the neck multi-scale feature space.

[0092] Let the feature vectors output by the student model and the teacher model at the i-th multi-scale feature layer be mapped as follows: Points in the Poincaré sphere and , where superscript Representing the student model, Representing the teacher model, Indicates the multi-scale feature layer number. Indicates the embedded dimension of hyperbolic space. It represents the spatial curvature of a sphere.

[0093] HIKD is supervised by constructing a distance matrix between features, and its hyperbolic space distance is defined as:

[0094]

[0095] in, and Let represent any two feature points within the Poincaré sphere. This indicates that the two have a curvature of The distance in hyperbolic space, Indicates curvature as Möbius addition, Describes the Euclidean second norm. This represents the inverse hyperbolic tangent function.

[0096] Based on this, a hyperbolic distance relationship matrix for the teacher-student model is constructed. To stabilize the gradient ratio, the mean of the off-diagonal elements is used. The matrix is ​​mean-normalized, and the final relational distillation loss is calculated using Smooth L1 Loss:

[0097]

[0098] in, This represents the hyperbolic distance matrix between feature points in the student model. This represents the hyperbolic distance matrix between feature points in the teacher model. and These represent the mean values ​​of the off-diagonal elements in the corresponding distance matrix, used to normalize the mean of the distance relationship matrix; Represents the smoothing L1 loss function. This indicates the distillation loss characteristic of the neck area.

[0099] CrossKD is used to align the detection decisions of the teacher model and the student model in the detection head prediction space. Specifically, it extracts the intermediate features of the student model at the i-th scale. ,in Represents three prediction scales; the intermediate feature set of the student model Enter frozen teacher detection header Generate cross-prediction results .in, This indicates a teacher's monitoring head where parameters are frozen during training. This represents the cross-prediction result obtained by passing student features through the teacher's detection head. A foreground mask is generated based on the teacher model's prediction confidence. ,in Used to filter foreground target areas and reduce the interference of background areas on distillation losses.

[0100] Foreground mask generated based on teacher confidence. Then, the prediction loss is decoupled into Distribution Focal Loss (DFL-KL) based on KL divergence and Quality Focal Loss (QFL) aligned.

[0101]

[0102] in, This represents the head-cross decision distillation loss. This represents the distillation loss of the bounding box distribution based on KL divergence. Indicates the quality focus classification of distillation loss; These represent the bounding box regression predictions output by the student model and the teacher model, respectively. and These represent the classification predictions output by the student model and the teacher model, respectively. This represents the foreground mask, used to select the foreground region to participate in the distillation calculation; and These represent the balance weights of the bounding box distillation loss and the classification distillation loss, respectively.

[0103] A strategy based on cosine annealing is introduced to dynamically adjust the distillation weight.

[0104] The dynamic weighted scheduling function is defined as follows:

[0105]

[0106] in, For the current training epoch, This is the warm-up epoch. This is the maximum distillation weight. This smooth transition effectively suppresses training oscillations caused by sudden weight changes.

[0107] Combining the above structural enhancement and distillation strategies, the end-to-end training total loss function of the model is defined as:

[0108]

[0109] in, This represents the total loss function for end-to-end training of the model; This represents the loss of the basic detection task, including classification loss and bounding box regression loss. The classification loss uses binary cross-entropy loss (BCE), and the bounding box regression loss includes CIoU loss and distribution focus loss (DFL). This represents the head-cross decision distillation loss; This indicates the characteristic distillation loss at the neck; This represents the dynamic weights corresponding to the CrossKD distillation loss in the t-th training round; This represents the dynamic weights corresponding to the HIKD distillation loss in the t-th training epoch. All distillation components only apply to the training phase and are automatically removed during the inference phase. After training is complete, the model weights file of the student model is saved.

[0110] S5: Input the image of the railway fastener to be detected into the railway fastener detection model loaded with the model weight file, and output the fastener detection result; the fastener detection result includes the fastener target position, confidence level and fastener status category.

[0111] The fastener status categories include four fastener statuses: normal, missing, broken, and displaced.

[0112] Each fastener status category is matched with a corresponding confidence level for output.

[0113] S6: Determine whether there is any abnormality in the railway fasteners based on the fastener inspection results, and output one or more inspection categories.

[0114] Specifically, in the inference detection stage, the student model weight file obtained in the training stage is first loaded into the lightweight railway fastener detection model to form a deployable inference detection model. Then, the image of the railway fastener to be detected is input into the inference detection model, and it goes through backbone network feature extraction, lightweight neck feature fusion network LeNeck multi-scale feature fusion, SPPF-SE-C2PSA collaborative feature enhancement, and high-frequency enhanced multi-scale prediction pre-feature detection head prediction in sequence. Based on the labels of the dataset during training, the fastener target position, confidence level and fastener fault information are obtained for subsequent manual review or alarm by the on-board inspection system.

[0115] like Figure 1 As shown, during the training phase, the image is first input into the backbone network of the student model, and the image is extracted. , , Feature maps at three scales; among them, An SPPF-SE channel enhancement module is then applied after the feature layer to enhance deep semantic features and reduce background noise. A C2PSA spatial focusing module is integrated at the main confluence node to enhance the response of the target region. Subsequently, multi-scale features are fused using the LeNeck lightweight neck feature fusion network to obtain N3, N4, and N5 multi-scale fused features, which are then fed into the HFP high-frequency perception plugin to enhance the high-frequency edge, texture, and local defect information of small fastener targets. Finally, the detection head outputs the fastener target location, confidence level, and fastener state category. During training, the teacher model guides the student model's decisions and feature relationships through NH-KD hybrid distillation. After training, only the student model's weight file is retained for inference detection; the teacher model and distillation branches do not participate in the inference process, thus not adding additional inference overhead.

[0116] Example 2, refer to Figure 2 and Figure 3As an embodiment of the present invention, a lightweight railway fastener testing method for vehicle-mounted portable devices is provided. To verify the beneficial effects of the present invention, scientific demonstration is carried out through economic benefit calculations and simulation experiments.

[0117] This example uses railway fastener image data to verify the lightweight railway fastener inspection method for vehicle-mounted portable devices of the present invention.

[0118] The railway fastener image data includes publicly available fastener image datasets and internal real-world railway line scene fastener image datasets.

[0119] The publicly available fastener image dataset includes fastener states such as normal, broken, and missing, as well as complex background interference categories such as track bed debris; the internal real track scene fastener image dataset is collected by high-definition industrial cameras mounted on track inspection equipment, and includes four fastener states: normal, missing, broken, and displaced.

[0120] Step 1: Construct a dataset of railway fastener images.

[0121] Table 1. Statistical Information of Track Fastener Defect Detection Dataset

[0122] Roboflow Dataset (Dataset A) 2234 Category 6 (Fastener defects and track bed debris) 2061 173 / 640×640 Internal dataset (Dataset B) 2464 Normal, missing, broken, displaced 1724 370 370 512×512

[0123] 1) Roboflow dataset (Dataset A):

[0124] This dataset originates from the Roboflow platform's open-source project. The dataset contains 2234 high-resolution images with automatic orientation correction. image.

[0125] 2) Internal real-world scenario dataset (Dataset B):

[0126] This dataset was collected in the field by high-definition industrial cameras deployed on a subway line track inspection vehicle, with a resolution of [resolution missing]. The dataset contains a total of 2,464 images. Unlike public datasets, Dataset B covers four typical industrial defect states: "ok", "loose", "broke", and "displace".

[0127] Step 2: Train and load the model, perform monitoring, and the detection results are as follows:

[0128] Table 2 Performance comparison of each model on the public dataset (Dataset A)

[0129] LeYOLO-Nano 2.47 1.09 0.950 0.893 0.685 1059.8 LeYOLO-Nano-HFE 2.49 1.09 0.967 0.947 0.714 1115.1 LeYOLO - Large 5.67 2.37 0.955 0.944 0.709 1948.6 LeYOLO-Large-HFE 5.75 2.43 0.957 0.982 0.716 1987.4 YOLOv8n 5.43 2.33 0.979 0.953 0.710 1908.0 YOLOv12n 5.80 2.51 0.954 0.941 0.720 2012.1

[0130] Table 2 presents the quantization comparison results of each model on a public dataset (Dataset A) containing strong interference from track bed debris. The results show that the proposed LeYOLO-Nano-HFE achieves a balance between accuracy and speed.

[0131] Despite the latest lightweight frontier model YOLOv12n achieving a maximum score of 0.720... However, its computational overhead is as high as 5.80 GFLOPs (approximately 2.3 times that of the model in this invention), resulting in an inference time of 2012.1 ms on portable platforms, which is difficult to meet the real-time requirements of portable inspection devices. In contrast, the method of this invention, while maintaining low computational burden and low inference latency, will... The accuracy improved from 0.685 to 0.714, narrowing the gap with the high-performance YOLOv12n model to 0.006. Furthermore, the recall rate was increased to 94.7%, demonstrating its powerful recognition capabilities in complex contexts.

[0132] Table 3 Performance comparison of various models on the internal real-world scenario dataset (Dataset B)

[0133] LeYOLO-Nano 2.49 1.09 0.998 0.989 0.812 1435.8 LeYOLO-Nano-HFE 2.50 1.09 0.998 0.998 0.844 1477.2 LeYOLO - Large 5.67 2.37 0.998 0.993 0.847 2426.5 LeYOLO-Large-HFE 5.75 2.44 1.000 1.000 0.856 2435.1 YOLOv8n 5.43 2.33 0.991 0.993 0.835 2213.7 YOLOv12n 5.75 2.44 0.998 1.000 0.857 2438.1

[0134] For Dataset B collected from industrial sites, the dataset exhibits a typical long-tailed imbalanced distribution and contains various small defects, posing a more stringent challenge to the model's ability to extract small target features.

[0135] As shown in Table 3, under complex backgrounds, this invention achieves higher accuracy and a lower false negative rate while maintaining high inference speed and low computational load. Compared to Compared to the mainstream YOLOv8n model with a score of 0.835, LeYOLO-Nano-HFE achieved a higher score of 0.844, while reducing computational cost by 54%. This result demonstrates that the introduced head-neck hybrid distillation strategy effectively alleviates the problem of target conflict in small-target-dense scenarios.

[0136] This example is validated on both a publicly available fastener image dataset and an internal real-world track scene fastener image dataset. The validation results show that, while maintaining a low number of parameters and low computational cost, this invention improves the recall and average detection accuracy of railway fastener detection, and reduces the probability of missed detection of small target fasteners in complex track backgrounds. Since the knowledge distillation structure is only used during the training phase, and only the student model weight file is loaded during the inference phase, it does not increase additional inference overhead, making it suitable for deployment on low-computing-power platforms such as onboard portable devices.

[0137] Figure 2This is an example image of the detection results. The image contains the location of the fastener target and the fault category. The generated bounding box represents the target location output by the model, and the text above the box indicates the identified target type. These include normal fasteners, missing fasteners, track bed debris, etc. Figure 3 The image shows the feature visualization of the EigenCAM method for lightweight railway fastener detection in portable on-vehicle devices. (a) is the original image of the fastener, and (b) is the heatmap feature visualization result of the high-frequency enhanced multi-scale prediction pre-feature detection head. The model produces strong feature responses at the fastener connection points, bolt holes, sleeper edges, and the area surrounding the fastener. The model configured with the high-frequency enhanced multi-scale prediction pre-feature detection head can pay more attention to the structural information of small fastener targets. Meanwhile, in (b), the overall response in complex background areas such as track bed gravel is relatively weak, indicating that it can suppress low-frequency background interference, enhance high-frequency detail features related to the fastener target, further improve the recall rate, and reduce the risk of missed detection.

[0138] This embodiment also provides a lightweight railway fastener inspection system for vehicle-mounted portable devices, including:

[0139] The training set construction unit acquires railway fastener image data, and annotates and preprocesses the railway fastener image data to form a fastener image dataset for model training.

[0140] The model building unit constructs a lightweight fastener detection model based on LeYOLO with hybrid feature enhancement.

[0141] The training unit uses the training dataset to train the lightweight fastener detection model. During the training period, head-neck joint hybrid knowledge distillation is introduced, so that the teacher model guides the student model training through neck feature distillation and head cross-decision distillation, and the trained model weight file is obtained.

[0142] The detection unit inputs the image of the railway fastener to be detected into a railway fastener detection model loaded with the model weight file, and outputs the fastener detection result; the fastener detection result includes the fastener target position, confidence level, and fastener status category; and, based on the fastener detection result, determines whether there is an anomaly in the railway fastener, and outputs one or more detection categories.

[0143] This embodiment also provides a computer device applicable to the lightweight railway fastener inspection method for vehicle-mounted portable devices, comprising: a memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions to implement the lightweight railway fastener inspection method for vehicle-mounted portable devices as proposed in the above embodiment.

[0144] The computer device can be a terminal, comprising a processor, memory, communication interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, carrier networks, NFC (Near Field Communication), or other technologies. The display screen can be an LCD screen or an e-ink screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad on the computer device's casing, or an external keyboard, touchpad, or mouse.

[0145] This embodiment also provides a storage medium storing a computer program that, when executed by a processor, implements the lightweight railway fastener inspection method for vehicle-mounted portable devices as proposed in the above embodiments. The storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read Only Memory (EPROM), Programmable Red-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0146] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A method for inspecting lightweight railway fasteners for portable vehicle-mounted devices, characterized in that: This includes acquiring railway fastener image data, and labeling and preprocessing the railway fastener image data to form a fastener image dataset for model training; A lightweight fastener detection model based on hybrid feature enhancement LeYOLO is constructed: multi-scale features of railway fastener images are extracted through a backbone network, and these multi-scale features are input into a lightweight neck feature fusion network LeNeck for multi-scale feature fusion; wherein, the backbone network is used to extract multi-scale features from the input railway fastener images and output... , , Feature layers at three scales; , , These represent the feature maps output at different downsampling stages of the backbone network; It is a deep semantic feature layer output at the end of the backbone network, which has strong high-level semantic expression capabilities; The SPPF-SE-C2PSA collaborative enhancement structure is used to enhance the multi-scale contextual information, channel features and spatial target regions in the fastener image at the LeNeck end, reducing background noise interference. The high-frequency enhanced multi-scale prediction pre-feature detection head is used to enhance the high-frequency edge and texture information of small targets in the fastener, and the final result is output. The lightweight fastener detection model is trained using a training dataset. During the training period, head-neck joint hybrid knowledge distillation is introduced, allowing the teacher model to guide the student model training through neck feature distillation and head cross-decision distillation, resulting in a trained model weight file. The image of the railway fastener to be detected is input into a railway fastener detection model loaded with the model weight file, and the fastener detection result is output; the fastener detection result includes the fastener target position, confidence level, and fastener status category; Based on the fastener inspection results, determine whether there are any abnormalities in the railway fasteners and output one or more inspection categories.

2. The method for testing lightweight railway fasteners for portable vehicle-mounted devices as described in claim 1, characterized in that: The SPPF-SE-C2PSA collaborative enhancement structure includes an SPPF-SE channel enhancement module and a C2PSA spatial attention module; At LeNeck At the core node of the main confluence, a C2PSA spatial attention module is connected in series to achieve geometric target focusing; At the end of the backbone network The layer introduces a spatial pyramid pooling SPPF-SE channel enhancement module with channel attention to perform multi-scale context aggregation and channel recalibration on the deep features output by the backbone network, thereby achieving semantic channel filtering; Suppose that, given the input feature pooling aggregation result The SPPF-SE channel enhancement module obtains channel descriptors through global average pooling. The channel reweighting vector is generated through a two-layer fully connected network. : in, It is the Sigmoid activation function. The ReLU activation function is used; the output after rescaling the features is... ; and These represent the weight matrices of the dimensionality-reducing and dimensionality-increasing fully connected layers, respectively. C is the channel scaling ratio; C is the number of channels in the feature map. This indicates a global average pooling operation.

3. The method for testing lightweight railway fasteners for portable vehicle-mounted devices as described in claim 2, characterized in that: The high-frequency enhanced multi-scale prediction pre-feature detection head includes a channel-preserving high-frequency sensing plugin (HFP) embedded in the front end of the multi-scale detection head, used to separate low-frequency background and high-frequency details of the input features. The high-frequency sensing plugin HFP obtains low-frequency components through local average pooling and obtains high-frequency components through the residual between the input features and the low-frequency components. It combines the high-frequency components to generate channel gate weights and spatial gate weights, and outputs the input features in the form of residuals after adaptive reweighting, thereby enhancing the edge, texture and local defect features of small fastener targets.

4. The method for testing lightweight railway fasteners for portable vehicle-mounted devices as described in claim 3, characterized in that: The head-neck combined mixed knowledge distillation includes neck feature distillation and cross-head distillation; The neck feature distillation and crosshead distillation are coordinated in stages using dynamic weights; In the early stages of training, distillation constraints are reduced to stabilize the student model. In the later stages of training, feature alignment and decision alignment are enhanced to enable the student model to achieve synergistic optimization between neck semantic representation and head detection decision.

5. The method for inspecting lightweight railway fasteners for portable vehicle-mounted devices as described in claim 4, characterized in that: The step-by-step coordination of neck feature distillation and crosshead distillation using dynamic weights includes constructing a dynamic weight scheduling function as follows: in, For the current training round, For the preheating period, Maximum distillation weight; The end-to-end training total loss function of the lightweight fastener detection model is defined as: in, This represents the baseline detection loss, which includes classification and bounding box regression. This represents the head-cross decision distillation loss; This indicates the characteristic distillation loss at the neck; This represents the dynamic weights corresponding to the CrossKD distillation loss in the t-th training round; This represents the dynamic weight corresponding to the HIKD distillation loss in the t-th training round.

6. The method for testing lightweight railway fasteners for portable vehicle-mounted devices as described in claim 5, characterized in that: The fastener status categories include four fastener statuses: normal, missing, broken, and displaced.

7. The method for testing lightweight railway fasteners for portable vehicle-mounted devices as described in claim 6, characterized in that: The detection categories include: if the output category is normal, it is determined that the fastener is not faulty; If the output category is missing, broken, or displaced, then the fastener is determined to be faulty.

8. A lightweight railway fastener inspection system for vehicle-mounted portable devices, based on the lightweight railway fastener inspection method for vehicle-mounted portable devices according to any one of claims 1 to 7, characterized in that: include, The training set construction unit acquires railway fastener image data, and annotates and preprocesses the railway fastener image data to form a fastener image dataset for model training. The model building unit constructs a lightweight fastener detection model based on hybrid feature enhancement LeYOLO; The training unit uses the training dataset to train the lightweight fastener detection model. During the training period, head-neck joint hybrid knowledge distillation is introduced, so that the teacher model guides the student model training through neck feature distillation and head cross-decision distillation, and the trained model weight file is obtained. The detection unit inputs the image of the railway fastener to be detected into a railway fastener detection model loaded with the model weight file, and outputs the fastener detection result; the fastener detection result includes the fastener target position, confidence level, and fastener status category; and, based on the fastener detection result, determines whether there is an anomaly in the railway fastener, and outputs one or more detection categories.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, it implements the steps of the lightweight railway fastener inspection method for vehicle-mounted portable devices as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by the processor, it implements the steps of the lightweight railway fastener inspection method for vehicle-mounted portable devices as described in any one of claims 1 to 7.

Images