A method for counting and inspecting railway line by using railway window operation tools

Abstract

This invention provides a method for inventorying tools and inspecting tracks during railway maintenance windows, relating to the field of safety management technology for railway maintenance windows. The method includes: constructing a general detection dataset for handheld railway operations; pre-training a lightweight YOLOv8n improved algorithm based on the dataset; extracting all foreground target features from a reference image or video to construct a reference feature library for the upper track; extracting all foreground target features from a comparison image or video to construct a comparison feature library for the lower track; comparing the lower track comparison feature library with the reference feature library for the upper track feature library feature-by-feature to determine if any tools or materials are missing; and taking photos while patrolling the work area at a constant speed, extracting features from each key frame of the inspection video in real time, identifying foreign objects whose differences from inherent features exceed a preset threshold, and triggering a real-time warning. This method can solve the pain points of manual inventory and inspection, reduce safety hazards, and improve the efficiency and safety of maintenance windows.

Description

Technical Field

[0001] This invention relates to the field of safety management technology for track maintenance windows on railways, and particularly to a method for inventorying tools and equipment and inspecting the track during track maintenance windows on railways. Background Technology

[0002] During railway track maintenance windows, the counting of tools and equipment on and off the tracks and the safety inspection of the track are crucial to ensuring railway traffic safety. If tools and equipment are left on the track, it can directly lead to major safety accidents such as train collisions and derailments, posing a serious threat to traffic safety.

[0003] Currently, manual inventory and inspection are commonly used, but existing technologies have significant shortcomings, as detailed below:

[0004] 1. Manual inventory and inspection have inherent defects: The operation time for railway track maintenance windows is usually about 4 hours. The operators need to carry dozens or even hundreds of tools and various construction materials. Manually counting each item takes 15-30 minutes, which encroaches on the effective working time. Moreover, due to haste and the inventory environment (changes in outdoor weather conditions, on-site lighting, etc.), it is easy to miss or miscount items, especially for small tools and scattered materials, where the probability of omission is extremely high. Manual inspection of foreign objects is inefficient and has a high omission rate, and it is impossible to form a safe closed loop of "inspection-counting". At the same time, manual inventory relies only on paper records or verbal confirmation, without visual evidence, traceability, and difficulty in defining responsibility.

[0005] 2. Existing image recognition technologies lack adaptability: Existing image recognition methods for detecting tools and equipment are mostly designed for fixed scenarios and rely heavily on fixed equipment deployment. They also require the construction of large datasets of tools and equipment, resulting in inflexible deployment, poor adaptability, and an inability to cope with a wide variety of tools, equipment, and materials. Furthermore, they are difficult to adapt to the complex working environment and poor visibility conditions at skylight operation points.

[0006] 3. Other solutions have obvious limitations: RFID or barcode solutions require modification of tools and equipment, resulting in high deployment costs and susceptibility to obstruction and wear, making them unsuitable for outdoor skylight operation scenarios; existing technologies do not combine the counting of tools and equipment on and off the track with on-site foreign object inspection, focusing only on a single counting function, which cannot meet the safety management requirements of the entire skylight operation process.

[0007] In summary, existing technologies cannot simultaneously achieve flexibility, adaptability, and safety, and are unable to solve the pain points of low efficiency, poor accuracy, and lack of traceability in the inventory of tools and equipment and line inspection during track maintenance windows. There is an urgent need for a tool inventory and foreign object detection solution that is adaptable to on-site operating conditions, convenient, efficient, safe, and reliable. Summary of the Invention

[0008] The purpose of this invention is to provide a method for inventorying tools and materials and inspecting the track during railway maintenance windows. It aims to achieve rapid and accurate inventorying of tools and materials by improving image recognition algorithms, solving the pain points of manual inventorying and inspection, reducing safety hazards, and improving the efficiency and safety of maintenance window operations. This method is suitable for complex outdoor railway operation scenarios. The specific technical solution is as follows:

[0009] A method for inventorying tools and equipment and inspecting the track during railway maintenance windows, the method comprising the following steps:

[0010] S100. Dataset Construction and Algorithm Pre-training: Construct a general detection dataset for railway handheld operation scenarios. The dataset covers handheld images of tools and track inspection images, including scene samples with different lighting, shooting angles, stacking occlusion, and target sizes. Based on the dataset, pre-train the lightweight YOLOv8n improved algorithm deployed on the handheld inspection device to obtain the pre-trained lightweight YOLOv8n improved algorithm.

[0011] S200, Pre-track baseline image acquisition and feature extraction: Before track entry at the track window, all tools, equipment and construction materials to be tracked are placed in a unified manner. A handheld acquisition device is used to take multi-angle surround shots to obtain a baseline image or video before track entry. Through the pre-trained lightweight YOLOv8n improved algorithm, all foreground target features in the baseline image or video are extracted to build a track entry baseline feature library without the need to identify the specific category of the target.

[0012] S300. Post-workover image acquisition and feature comparison: After the work is completed and the tools and construction materials are brought back, they are placed in a unified manner. Using the same handheld acquisition equipment and the same shooting method as before the work, post-workover comparison images or videos are obtained. All foreground target features in the comparison images or videos are extracted using a pre-trained lightweight YOLOv8n improved algorithm to construct a post-workover comparison feature library. The cosine similarity comparison algorithm is used to compare the post-workover comparison feature library with the previous benchmark feature library feature by feature to determine whether any tools or materials have been omitted.

[0013] S400, Real-time Inspection of Foreign Objects on Track: Carrying a handheld inspection device, patrols and takes pictures at a constant speed along the work area. The features of each key frame of the inspection video are extracted in real time through the pre-trained lightweight YOLOv8n improved algorithm. The features are compared with the pre-stored inherent feature templates of railway rails, fasteners and track slabs to identify foreign objects whose differences from the inherent features exceed the preset threshold and trigger real-time warnings.

[0014] S500 generates a complete process record.

[0015] Furthermore, in step S100, in the scene samples, small target samples with a resolution of 32×32 pixels or less account for approximately 40%, occluded target samples account for ≥30%, and degraded image samples taken by hand account for ≥30%.

[0016] Furthermore, in step S100, the lightweight YOLOv8n improved algorithm is based on the YOLOv8n infrastructure, including a reconstructed lightweight backbone network, a bidirectional cross-scale feature fusion neck network, a detection head with an embedded lightweight robust attention module, and a small target-specific weighted joint loss function module connected in sequence.

[0017] Furthermore, the reconstructed lightweight backbone network is as follows: All C2f modules of the original YOLOv8n backbone network are replaced with the C2f-Shuffle lightweight module. The C2f-Shuffle lightweight module replaces the standard convolutions of the original Bottleneck branch with sequentially connected depthwise separable convolutions and channel shuffling structures. The depthwise separable convolutions split the standard convolutions into depthwise convolutions and pointwise convolutions. Simultaneously, the two sets of residual connections in the original C2f module are optimized into one set. Parallel shallow small target feature enhancement branches are added to layers 2 and 3 of the backbone network. The input to the shallow small target feature enhancement branch is the shallow features of the main branch of the backbone network. First, channel dimensionality reduction is performed using 1×1 convolutions. Then, detailed features of small targets are extracted using 3×3 depthwise separable convolutions. Next, parallel pooling units of three scales (1×1, 3×3, and 5×5) are used to capture features of small targets of different sizes. Finally, channel dimensions are restored using 1×1 convolutions, and the output features are adaptively concat-fused with the shallow features of the main branch.

[0018] Furthermore, the bidirectional cross-scale feature fusion neck network is as follows: Based on the original YOLOv8n top-down and bottom-up PAN-FPN structures, two sets of cross-scale skip connections are added: the 4× downsampled features of the second layer of the backbone network are directly skipped to the 16× downsampled feature layer in the deep neck layer, and the 8× downsampled features of the third layer of the backbone network are directly skipped to the 32× downsampled feature layer in the deep neck layer; at the same time, an adaptive feature weighting unit is set in each feature fusion node, and the weight of each feature channel is generated by the sigmoid function. High weights are assigned to foreground target features, with the high weight value ranging from 0.5 to 1, and high weights are assigned to rails and tracks. Background features are assigned low weights, with values ​​ranging from 0 to 0.5. An occlusion-aware feature alignment module is embedded in each feature fusion node. This module first performs spatial transformation on the feature map using deformable convolution to spatially align the features of the occluded target. Then, through an occlusion-aware attention mechanism, the foreground confidence of each feature region is calculated to generate a weight mask. Features of occluded regions with a foreground confidence below 0.3 are subject to weight suppression, with a weight suppression range of 0 to 0.3. Unoccluded effective feature regions are assigned high weights, with a high weight range of 0.3 to 1.

[0019] Furthermore, the detection head embedding the lightweight robust attention module is as follows: A lightweight robust attention module with 0.08M parameters is embedded in the front end of the detection head, including parallel channel attention branches and spatial attention branches: The channel attention branch performs global average pooling on the input features, and then performs channel transformation through two sets of 1×1 convolutions to generate a channel weight mask, which is multiplied with the input features to enhance effective feature channels and suppress noisy feature channels; The spatial attention branch first extracts spatial features through 3×3 depthwise separable convolution, and then performs adaptive threshold filtering, dynamically adjusting the filtering threshold according to the brightness and blur of the input image to filter out noise features caused by handheld shooting blur and uneven lighting, generating a spatial weight mask, which is multiplied with the input features to enhance the spatial features of the foreground target; After the output features of the channel attention branch and the spatial attention branch are fused, a residual connection is made with the original input features, and finally the enhanced feature map is output to the detection head.

[0020] Furthermore, the small target-specific weighted joint loss function module is as follows: it adopts a joint loss function of weighted localization loss Loss_W-SIoU and hard example perception classification loss Loss_HA-Focal. The total loss formula is: Loss = λ1×Loss_W-SIoU + λ2×Loss_HA-Focal, where λ1 is the weighted localization loss coefficient and λ2 is the hard example perception classification loss coefficient. The weighted localization loss Loss_W-SIoU is: based on SIoU loss, a target size weighting factor α and an occlusion penalty factor β are introduced, and the formula is: Loss_W-SIoU = (1 - SIoU) ×α×β. The hard example perception classification loss Loss_HA-Focal is: based on Focal Loss, a sample difficulty perception factor γ is introduced. γ is negatively correlated with the sample classification confidence p and IoU score iou, and the formula is: Loss_HA-Focal = -α×(1-p)^γ×log(p), where γ=2.0 / (p×iou+1e-8).

[0021] Furthermore, in step S200, the multi-angle surround shooting method involves taking 3-5 high-definition photos from different angles or recording a 10-20 second surround video.

[0022] Furthermore, in step S300, the similarity threshold of the cosine similarity comparison algorithm is set to 95%, and a similarity of ≥95% is determined to be the same target; in step S400, the similarity threshold for foreign object identification is set to 85%, and if the similarity between the extracted real-time features and the railway inherent feature template is <85%, it is determined to be a foreign object.

[0023] Furthermore, all data in steps S100-S400 is stored locally on the handheld inspection device and automatically synchronized to the railway safety management platform after the network is restored, achieving full-process traceability.

[0024] The present invention provides a method for inventorying tools and equipment and inspecting the track during railway maintenance windows, which has the following beneficial effects:

[0025] This invention breaks through the dependence of existing algorithms on tool and equipment datasets. Based on image feature comparison, it achieves accurate comparison of tools, equipment and materials under different placement positions and shooting angles, avoiding missed judgments and misjudgments, and significantly improving the efficiency and accuracy of counting. By optimizing the YOLOv8n algorithm model, it reduces hardware resource consumption, adapts to the computing power requirements of handheld devices, ensures the real-time performance of counting comparison and foreign object identification, and meets the requirements of daylight window operation conditions. Attached Figure Description

[0026] Figure 1A flowchart illustrating a method for inventorying tools and equipment and inspecting the track during railway maintenance windows, provided by an embodiment of the present invention;

[0027] Figure 2 A flowchart outlining the methods for inventorying tools and equipment and conducting line inspections during railway track maintenance windows.

[0028] Figure 3 Overall architecture diagram for improving the lightweight YOLOv8n algorithm;

[0029] Figure 4 This is a schematic diagram of the C2f-Shuffle lightweight module.

[0030] Figure 5 A schematic diagram of the structure for enhancing the features of shallow small targets;

[0031] Figure 6 This is a schematic diagram of the lightweight, robust attention module. Detailed Implementation

[0032] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. The advantages and features of the present invention will become clearer from the following description. It should be noted that the drawings are all in a very simplified form and use non-precise proportions, and are only used to facilitate and clearly illustrate the purpose of the embodiments of the present invention.

[0033] Example 1

[0034] This embodiment provides a method for inventorying tools and equipment and conducting track inspections during railway maintenance windows. (See reference...) Figure 1 , 2 As shown, the method includes the following steps:

[0035] S100. Dataset Construction and Algorithm Pre-training: Construct a general detection dataset for railway handheld operation scenarios. The dataset covers handheld images of tools and track inspection images, including scene samples with different lighting, shooting angles, stacking occlusion, and target sizes. Based on the dataset, pre-train the lightweight YOLOv8n improved algorithm deployed on the handheld inspection device to obtain the pre-trained lightweight YOLOv8n improved algorithm.

[0036] In one embodiment, in the scene samples, small target samples smaller than 32×32 pixels account for about 40%, occluded target samples account for ≥30%, and degraded image samples taken by hand account for ≥30%.

[0037] In one embodiment, see Figure 3As shown, the lightweight YOLOv8n improved algorithm is based on the YOLOv8n architecture, including a reconstructed lightweight backbone network, a bidirectional cross-scale feature fusion neck network, a detection head with an embedded lightweight robust attention module, and a small target-specific weighted joint loss function module connected in sequence.

[0038] In one embodiment, see Figure 4 , 5 As shown, the reconstructed lightweight backbone network is as follows: All C2f modules of the original YOLOv8n backbone network are replaced with the C2f-Shuffle lightweight module. The C2f-Shuffle lightweight module replaces the standard convolutions of the original Bottleneck branch with sequentially connected depthwise separable convolutions and channel shuffling structures. The depthwise separable convolutions split the standard convolutions into depthwise convolutions and pointwise convolutions, significantly reducing computational cost without sacrificing feature representation capabilities. The channel shuffling operation solves the problem of information blockage between channels caused by depthwise separable convolutions. Simultaneously, the two sets of residual connections in the original C2f module are optimized to one. To further reduce the number of parameters, parallel shallow small target feature enhancement branches are added to the second (4× downsampling) and third (8× downsampling) layers of the backbone network. The input of the shallow small target feature enhancement branch is the shallow features of the main branch of the backbone network. First, channel dimensionality reduction is performed through 1×1 convolution to reduce the amount of computation. Then, the detailed features of small targets are extracted through 3×3 depthwise separable convolution. Then, parallel pooling units of three scales (1×1, 3×3, and 5×5) are used to capture the features of small targets of different sizes. Finally, the channel dimension is restored through 1×1 convolution, and the output features are concat adaptively fused with the shallow features of the main branch to enhance the detailed features of small targets and avoid their loss in subsequent downsampling processes.

[0039] In one embodiment, see Figure 3As shown, the bidirectional cross-scale feature fusion neck network is: the original YOLOv8n top-down and bottom-up PAN-FPN. Based on the existing structure, two new cross-scale skip connections are added: the 4× downsampled features of the second layer of the backbone network are directly skipped to the 16× downsampled feature layer in the deep neck region, and the 8× downsampled features of the third layer of the backbone network are directly skipped to the 32× downsampled feature layer in the deep neck region. This directly fuses the shallow small target detail features with the deep semantic features, avoiding the dilution of small target features during multi-layer propagation. Simultaneously, an adaptive feature weighting unit is set at each feature fusion node, generating weights for each feature channel using the sigmoid function. High weights are assigned to foreground target features, and low weights are assigned to background features such as rails and track slabs, improving the ability to distinguish between foreground and background. An occlusion-aware feature alignment module is embedded in each feature fusion node. This module first performs spatial transformation on the feature map using deformable convolution to spatially align the features of occluded targets, solving the problem of target feature misalignment caused by stacked occlusion. Then, through an occlusion-aware attention mechanism, the foreground confidence of each feature region is calculated to generate a weighted mask. Foreground confidence below 0.3, the weighted mask is applied. We apply weighted suppression to the occluded region features and assign high weights to the effective feature regions that are not occluded, thereby enhancing the extraction of effective features from occluded targets.

[0040] In one embodiment, the detection head with an embedded lightweight robust attention module is: a lightweight robust attention module is embedded at the front end of the detection head, see [reference]. Figure 6 As shown, the lightweight robust attention module has a parameter size of 0.08M, which is only 1 / 10 of that of a conventional CBAM attention module. It includes parallel channel attention branches and spatial attention branches: The channel attention branch performs global average pooling on the input features, and then performs channel transformation through two sets of 1×1 convolutions to generate a channel weight mask, which is multiplied with the input features to enhance effective feature channels and suppress noisy feature channels; The spatial attention branch first extracts spatial features through 3×3 depthwise separable convolution, and then uses adaptive threshold filtering to dynamically adjust the filtering threshold according to the brightness and blur of the input image to filter out noise features caused by handheld shooting blur and uneven lighting, generating a spatial weight mask, which is multiplied with the input features to enhance the spatial features of the foreground target; After the output features of the channel attention branch and the spatial attention branch are fused, a residual connection is made with the original input features to avoid feature loss caused by the attention module, and finally the enhanced feature map is output to the detection head.

[0041] In one embodiment, the small target-specific weighted joint loss function module is as follows: it adopts a joint loss function of weighted localization loss Loss_W-SIoU and hard example perception classification loss Loss_HA-Focal, and the total loss formula is: Loss = λ1×Loss_W-SIoU + λ2×Loss_HA-Focal, where λ1 is the weighted localization loss coefficient, preferably 2.0, and λ2 is the hard example perception classification loss coefficient, preferably 1.0; the weighted localization loss Loss_W-SIoU is: based on SIoU loss, a target size weighting factor α and an occlusion penalty factor β are introduced, and the formula is: Loss_W-SIoU = (1 - SIoU) ×α×β; where α is negatively correlated with the target size, and for small targets smaller than 32×32 pixels, α≥3; β is positively correlated with the target occlusion degree, and the higher the occlusion degree, the larger β is, up to a maximum of 2.5; the difficult example perception classification loss Loss_HA-Focal is: based on Focal Loss, a sample difficulty perception factor γ is introduced. γ is negatively correlated with the sample classification confidence p and IoU score iou, and the formula is: Loss_HA-Focal = -α×(1-p)^γ×log(p), where γ=2.0 / (p×iou+1e-8). The lower the classification confidence and IoU score of the difficult example sample, the larger γ is, and the higher the loss weight is, so that the model prioritizes optimizing high-risk difficult examples that are occluded or blurred.

[0042] Specifically, the lightweight YOLOv8n improvement algorithm of this invention improves YOLOv8n by the following steps:

[0043] Step 1: Lightweight backbone network reconstruction + shallow small target feature enhancement branch

[0044] ① The Bottleneck branch of the C2f module in the YOLOv8n backbone network is replaced with a lightweight structure of depthwise separable convolution + channel shuffle. At the same time, the number of residual connections is optimized from 2 to 1, reducing the number of parameters and computation without losing the feature representation ability. Meanwhile, local feature enhancement units are introduced into the module to avoid the loss of detailed features caused by lightweighting.

[0045] ② In the second and third shallow layers of the backbone network (downsampling factor 4× and 8×, corresponding to small target feature preservation layers), a parallel shallow small target feature enhancement branch (Tiny-Feature Boost Branch, TFBB) is added. The branch uses 1×1 convolution dimensionality reduction + 3×3 depthwise separable convolution to extract detailed features + multi-scale pooling (1×1, 3×3, 5×5, corresponding to different sizes of tools and track foreign objects in the railway scene) to capture the features of small targets of different sizes. The output features are adaptively fused with the shallow features of the main branch to solve the problem of feature loss of small targets after multiple downsampling.

[0046] Step 2: Bidirectional cross-scale fusion of neck network + occlusion-aware feature alignment module

[0047] ① The PAN-FPN is improved into a bidirectional cross-scale feature fusion network Bi-CSFF: Based on the original top-down + bottom-up fusion structure, a shallow-to-deep cross-scale skip connection is added. Specifically, the 4× downsampled features of the second layer and the 8× downsampled features of the third layer of the backbone network are directly skipped to the deep 16× downsampled feature layer and the 32× downsampled feature layer of the neck network, respectively. This directly fuses the shallow small target detail features with the deep semantic feature layer, avoiding the dilution of small target features during the transmission process. At the same time, an adaptive feature weighting module is introduced at each fusion node to assign higher weights to small targets and foreground targets, while weakening the feature weights of complex backgrounds such as rails and track slabs, thereby improving the foreground-background discrimination ability.

[0048] ② Added Occlusion-Aware Feature Alignment Module: An occlusion-aware feature alignment module is embedded in each fusion layer of the neck. It uses deformable convolution to spatially align the features of occluded targets and capture the local effective features of occluded targets. At the same time, an occlusion-aware attention mechanism is introduced. By calculating the foreground confidence of feature regions, a weight mask is generated. Features of occluded regions with foreground confidence below a preset threshold are suppressed by weight. For example, higher weights are assigned to unoccluded feature regions to filter out invalid noise features of occluded objects and solve the problem of missed detection of stacked tools and foreign objects occluded by track components.

[0049] Step 3: Lightweight Robust Attention Module (Embedded in the Front End of the Detection Head)

[0050] We designed a lightweight, robust attention module, Lightweight RobustAttention, with only 0.08M parameters, embedded in the front end of the detection head, and divided into two parallel branches:

[0051] Channel attention branch: Global average pooling + 1×1 convolution is used to capture the dependencies between channels, enhance effective feature channels, and suppress noisy feature channels;

[0052] Spatial attention branch: It adopts depthwise separable convolution + adaptive threshold filtering to filter noise features caused by blur and uneven lighting, and enhance the spatial features of the foreground target; the outputs of the two branches are fused and residually connected with the input features to improve the feature robustness of handheld shooting of blurry, unevenly lit, and motion-degraded images with almost no increase in computing power.

[0053] Step 4: Weighted Joint Loss Function Dedicated to Small Targets

[0054] We designed a joint loss function combining W-SIoU localization loss and HA-Focal classification loss for railway scenarios to replace the original CIoU+Focal Loss:

[0055] Weighted SIoU localization loss W-SIoU: Based on SIoU loss, a target size weighting factor is introduced, which assigns a loss weight of 3 times to small targets smaller than 32×32 pixels, amplifying the localization error of small targets and allowing the model to prioritize optimizing the localization accuracy of small targets; at the same time, an occlusion penalty factor is introduced, which increases the loss weight for occluded targets, strengthening the model's optimization of occluded targets.

[0056] Difficult Example Aware Focal Classification Loss (HA-Focal): Building upon the original Focal Loss, this approach introduces a sample difficulty awareness factor. This factor dynamically adjusts the loss weights based on the sample's classification confidence and IoU score, assigning higher weights to difficult examples such as occluded, blurred, or poorly lit samples. This allows the model to focus on optimizing high-risk, difficult targets in railway scenarios, reducing the false negative rate. The sample difficulty awareness factor is negatively correlated with the sample's classification confidence and IoU score; the lower the classification confidence and IoU score, the higher the loss weight assigned to difficult examples.

[0057] This invention improves the YOLOv8n network by four aspects: backbone network reconstruction and small target feature enhancement, bidirectional cross-scale feature fusion and occlusion perception alignment, lightweight robust attention module, and small target-specific weighted joint loss function. The optimized algorithm model has low resource consumption, can be adapted to existing handheld devices, is lightweight to deploy, has controllable cost, and is highly practical. It is suitable for railway outdoor operation scenarios and is easy to promote and apply on a large scale.

[0058] S200, Pre-track reference image acquisition and feature extraction: Before track entry at the track window, all tools, equipment and construction materials to be put on track are placed in a unified manner. A handheld acquisition device is used to take multi-angle surround shots to obtain a reference image or video before track entry. Through the pre-trained lightweight YOLOv8n improved algorithm, all foreground target features in the reference image or video are extracted to build a track entry reference feature library without the need to identify the specific category of the target.

[0059] In one embodiment, the multi-angle surround shooting method involves taking 3-5 high-definition photos from different angles or recording a 10-20 second surround video.

[0060] S300. Post-workover image acquisition and feature comparison: After the work is completed and the tools and construction materials are brought back, they are placed in a unified manner. Using the same handheld acquisition equipment, the same shooting method as before the work is started is used to obtain post-workover comparison images or videos. All foreground target features in the comparison images or videos are extracted using a pre-trained lightweight YOLOv8n improved algorithm to construct a post-workover comparison feature library. The cosine similarity comparison algorithm is used to compare the post-workover comparison feature library with the previous benchmark feature library feature by feature to determine whether any tools or materials have been omitted.

[0061] In one embodiment, the similarity threshold of the cosine similarity comparison algorithm is set to ≥95%, and a similarity of ≥95% is determined to be the same target.

[0062] S400 Real-time Inspection of Foreign Objects on Track: Carrying a handheld inspection device, the system moves at a constant speed along the work area to take pictures. The system uses a pre-trained lightweight YOLOv8n improved algorithm to extract the features of each key frame of the inspection video in real time. The features are compared with the pre-stored inherent feature templates of railway rails, fasteners, and track slabs to identify foreign objects whose differences from the inherent features exceed a preset threshold, triggering a real-time warning.

[0063] In one embodiment, the similarity threshold for foreign object identification is set to < 85%. If the similarity between the extracted real-time features and the inherent feature template of the railway is < 85%, it is determined to be a foreign object.

[0064] S500 generates a complete process record.

[0065] In one embodiment, steps S100-S400 above can all be run offline. All data in steps S100-S400 is stored locally on the handheld inspection device and automatically synchronized to the railway safety management platform after the network is restored, so as to achieve full-process traceability.

[0066] This invention breaks through the dependence of existing algorithms on tool and equipment datasets. Based on image feature comparison, it achieves accurate comparison of tools and equipment and materials under different placement positions and shooting angles, avoiding omissions and misjudgments. The efficiency and accuracy of counting are significantly improved. The automatic counting and comparison calculation of tools and equipment only takes 2-3 minutes, compared with 15-30 minutes for manual counting, which greatly shortens the overall time and frees up more effective working time for front-line operations.

[0067] Application Examples:

[0068] S1 Dataset Construction and Algorithm Pre-training: A general detection dataset for railway handheld operation scenarios was constructed, consisting of 12,000 images, including 9,600 images for training, 1,200 images for validation, and 1,200 images for testing. The dataset covers images of 15 common railway operation tools and 5 common track foreign objects taken by hand, including scene samples with strong light / backlight / low light at night, different shooting angles, stacked occlusion, and different target sizes. Among them, small target samples smaller than 32×32 pixels account for 45%, occluded target samples account for 30%, and degraded image samples taken by hand account for 35%. Training environment setup: NVIDIA RTX 4090 24G graphics card, Intel i9-13900K processor, 32GB RAM, PyTorch 2.0 framework, input image size 640×640, batch size=16, epoch=300, AdamW optimizer, initial learning rate 0.001, cosine annealing learning rate decay strategy; data augmentation uses Mosaic, random flipping, random brightness, random blur, and random occlusion to simulate real-world railway handheld photography scenarios. After training, pre-trained weights are obtained, which possess the feature extraction capability for general targets in railway scenarios, eliminating the need for secondary training for newly added tools and equipment.

[0069] S2 Pre-Landing Baseline Image Acquisition and Feature Extraction: Before the track opening operation, all tools, equipment, and construction materials to be placed on the leveled area. Workers use industrial-grade handheld terminals to take 3-5 high-definition 1080P photos from different angles around the tools and materials, or record 10-20 seconds of 1080P / 25fps surround video to ensure no tools or materials are missed. After shooting, the images / videos are pre-processed (noise reduction, deblurring, brightness equalization, keyframe extraction) using a pre-trained lightweight YOLOv8n improved algorithm. Then, all foreground target features are extracted from the images / videos, background interference is removed, and the feature information of each target is encoded to construct a baseline feature library for track opening. This eliminates the need to identify the specific category of the target and simultaneously associates information such as the work project, workers, work location, and track opening plan.

[0070] S3 Post-Work Track Comparison Image Acquisition and Feature Comparison: After completing the work, the tools, equipment, and construction materials brought back are placed in the site area. Their placement and order do not need to be identical to those before entering the track. Workers use the same handheld terminal and the same shooting method as before entering the track to take 3-5 high-definition photos from different angles or 10-20 seconds of surround video. The same preprocessing and feature extraction methods as before entering the track are used to obtain the post-work track comparison feature library. A cosine similarity comparison algorithm is used to compare the post-work track comparison feature library with the previous track baseline feature library feature by feature. The similarity threshold is set to 95%. A similarity ≥ 95% indicates the same target. The comparison process is unaffected by the arrangement or position of the features. Result determination: If all features in the next comparison feature library can be matched in the benchmark library, the count is deemed qualified; if the next comparison feature library is missing some features from the benchmark library, it is deemed that there is an omission, and the handheld terminal immediately triggers an audio-visual warning, while marking the image area corresponding to the missing feature on the screen to facilitate the operator's investigation; if there is a newly added feature, it is deemed that there is an over- or mis-added feature, and an early warning is triggered simultaneously.

[0071] S4 Real-time Track Foreign Object Inspection: Operators wearing integrated law enforcement recording equipment patrol the work area at a constant speed of 1-2 m / s. The equipment continuously records 1080P / 30fps inspection video, covering the rails, fasteners, and track slabs in the work area. Using a pre-trained, lightweight YOLOv8n improved algorithm, the equipment extracts image features from each keyframe in real time and compares them with a pre-stored library of inherent feature templates for railway rails, fasteners, and track slabs, with a similarity threshold set to 85%. If the similarity between an object's feature and the inherent feature is less than 85%, the equipment immediately triggers an audible and visual warning, marks the foreign object's location on the screen, and saves the corresponding video clip. Operators promptly remove the foreign object, confirm the removal on the equipment, and record the removal information. If no foreign object is detected throughout the process, an inspection pass report is automatically generated upon completion of the inspection.

[0072] S5 generates a complete process record. All the above steps support offline operation, and all data is stored locally on the device. Once the network is restored, it is automatically synchronized to the railway safety management platform, achieving full process traceability.

[0073] Those skilled in the art should understand that the present invention can be implemented in many other specific forms without departing from the spirit and scope of the invention. Any changes or modifications made by those skilled in the art based on the embodiments of the present invention and the above disclosure shall fall within the protection scope of the claims.

Claims

1. A method for inventorying tools and equipment and inspecting the track during railway maintenance windows, characterized in that, The method includes the following steps: S100. Dataset Construction and Algorithm Pre-training: Construct a general detection dataset for railway handheld operation scenarios. The dataset covers handheld images of tools and track inspection images, including scene samples with different lighting, shooting angles, stacking occlusion, and target sizes. Based on the dataset, pre-train the lightweight YOLOv8n improved algorithm deployed on the handheld inspection device to obtain the pre-trained lightweight YOLOv8n improved algorithm. S200, Pre-track baseline image acquisition and feature extraction: Before track entry at the track window, all tools, equipment and construction materials to be tracked are placed in a unified manner. A handheld acquisition device is used to take multi-angle surround shots to obtain a baseline image or video before track entry. Through the pre-trained lightweight YOLOv8n improved algorithm, all foreground target features in the baseline image or video are extracted to build a track entry baseline feature library without the need to identify the specific category of the target. S300. Post-workover image acquisition and feature comparison: After the work is completed and the tools and construction materials are brought back, they are placed in a unified manner. Using the same handheld acquisition equipment and the same shooting method as before the work, post-workover comparison images or videos are obtained. All foreground target features in the comparison images or videos are extracted using a pre-trained lightweight YOLOv8n improved algorithm to construct a post-workover comparison feature library. The cosine similarity comparison algorithm is used to compare the post-workover comparison feature library with the previous benchmark feature library feature by feature to determine whether any tools or materials have been omitted. S400, Real-time Inspection of Foreign Objects on Track: Carrying a handheld inspection device, patrols and takes pictures at a constant speed along the work area. The features of each key frame of the inspection video are extracted in real time through the pre-trained lightweight YOLOv8n improved algorithm. The features are compared with the pre-stored inherent feature templates of railway rails, fasteners and track slabs to identify foreign objects whose differences from the inherent features exceed the preset threshold and trigger real-time warnings. S500 generates a complete process record.

2. The method for inventorying tools and equipment and inspecting the track during railway maintenance windows as described in claim 1, characterized in that, In step S100, in the scene samples, small target samples with a resolution of less than 32×32 pixels account for about 40%, occluded target samples account for ≥30%, and degraded image samples taken by hand account for ≥30%.

3. The method for inventorying tools and equipment and inspecting the track during railway maintenance windows according to claim 1, characterized in that, In step S100, the lightweight YOLOv8n improved algorithm is based on the YOLOv8n infrastructure, including a reconstructed lightweight backbone network, a bidirectional cross-scale feature fusion neck network, a detection head with an embedded lightweight robust attention module, and a small target-specific weighted joint loss function module connected in sequence.

4. The method for inventorying tools and equipment and inspecting the track during railway maintenance windows according to claim 3, characterized in that, The reconstructed lightweight backbone network consists of the following: The original YOLOv8n backbone network's C2f modules are completely replaced with the C2f-Shuffle lightweight module. The C2f-Shuffle module replaces the standard convolutions of the original Bottleneck branch with sequentially connected depthwise separable convolutions and channel shuffling structures. The depthwise separable convolutions split the standard convolutions into depthwise convolutions and pointwise convolutions. Simultaneously, the two sets of residual connections in the original C2f module are optimized into one set. Parallel shallow small target feature enhancement branches are added to layers 2 and 3 of the backbone network. The input to these branches is the shallow features of the backbone network's main branch. First, channel dimensionality reduction is performed using 1×1 convolutions. Then, detailed features of small targets are extracted using 3×3 depthwise separable convolutions. Finally, parallel pooling units of 1×1, 3×3, and 5×5 scales are used to capture features of small targets of different sizes. Finally, the channel dimension is restored by 1×1 convolution, and the output features are adaptively fused with the shallow features of the main branch by concat.

5. The method for inventorying tools and equipment and inspecting the track during railway maintenance windows according to claim 4, characterized in that, The bidirectional cross-scale feature fusion neck network is as follows: Based on the original YOLOv8n top-down and bottom-up PAN-FPN structures, two new cross-scale skip connections are added: the 4× downsampled features of the second layer of the backbone network are directly skipped to the 16× downsampled feature layer in the deep neck layer, and the 8× downsampled features of the third layer of the backbone network are directly skipped to the 32× downsampled feature layer in the deep neck layer. At the same time, an adaptive feature weighting unit is set in each feature fusion node, which generates the weight of each feature channel through the sigmoid function, assigning high weights to foreground target features and low weights to background features such as rails and track slabs. An occlusion-aware feature alignment module is embedded in each feature fusion node. The occlusion-aware feature alignment module first performs spatial transformation on the feature map through deformable convolution to spatially align the features of the occluded target. Then, through the occlusion perception attention mechanism, the foreground confidence of each feature region is calculated to generate a weight mask. The occluded region features with a foreground confidence of less than 0.3 are subject to weight suppression, and the effective feature regions that are not occluded are assigned high weights.

6. The method for inventorying tools and equipment and inspecting the track during railway maintenance windows according to claim 5, characterized in that, The detection head with embedded lightweight robust attention module is as follows: A lightweight robust attention module is embedded in the front end of the detection head. The lightweight robust attention module has 0.08M parameters and includes parallel channel attention branch and spatial attention branch: The channel attention branch performs global average pooling on the input features, and then performs channel transformation through two sets of 1×1 convolutions to generate channel weight masks, which are multiplied with the input features to enhance effective feature channels and suppress noisy feature channels; The spatial attention branch first extracts spatial features through a 3×3 depthwise separable convolution, and then uses adaptive threshold filtering to dynamically adjust the filtering threshold according to the brightness and blur of the input image to filter out noise features caused by handheld shooting blur and uneven lighting, generating a spatial weight mask, which is multiplied with the input features to enhance the spatial features of the foreground target. After the output features of the channel attention branch and the spatial attention branch are fused, a residual connection is made with the original input features, and the final enhanced feature map is output to the detection head.

7. The method for inventorying tools and equipment and inspecting the track during railway maintenance windows according to claim 6, characterized in that, The weighted joint loss function module for small targets is as follows: it adopts a joint loss function of weighted localization loss Loss_W-SIoU and hard example perception classification loss Loss_HA-Focal. The total loss formula is: Loss = λ1×Loss_W-SIoU + λ2×Loss_HA-Focal, where λ1 is the weighted localization loss coefficient and λ2 is the hard example perception classification loss coefficient. The weighted localization loss Loss_W-SIoU is: based on SIoU loss, a target size weighting factor α and an occlusion penalty factor β are introduced, and the formula is: Loss_W-SIoU = (1 - SIoU) ×α×β. The hard example perception classification loss Loss_HA-Focal is: based on Focal Loss, a sample difficulty perception factor γ is introduced. γ is negatively correlated with the sample classification confidence p and IoU score iou, and the formula is: Loss_HA-Focal = -α×(1-p)^γ×log(p), where γ=2.0 / (p×iou+1e-8).

8. The method for inventorying tools and equipment and inspecting the track during railway maintenance windows according to claim 1, characterized in that, In step S200, the multi-angle surround shooting method involves taking 3-5 high-definition photos from different angles or recording a 10-20 second surround video.

9. The method for inventorying tools and equipment and inspecting the track during railway maintenance windows according to claim 1, characterized in that, In step S300, the similarity threshold of the cosine similarity comparison algorithm is set to 95%, and a similarity of ≥95% is determined to be the same target; in step S400, the similarity threshold of foreign object identification is set to 85%, and if the similarity between the extracted real-time features and the railway inherent feature template is <85%, it is determined to be a foreign object.

10. The method for inventorying tools and equipment and inspecting the track during railway maintenance windows according to claim 1, characterized in that, All data in steps S100-S400 is stored locally on the handheld inspection device and automatically synchronized to the railway safety management platform after the network is restored, achieving full-process traceability.

Images