Training method, detection method and system of multi-scale shielding tool detection model

Abstract

The invention provides a training method, a detection method and a system of a multi-scale shielding tool detection model. The method comprises the following steps: acquiring a training sample set containing multi-scale shielding railway tool images and labels; and constructing an initial target tool detection model comprising a backbone network, a feature fusion network and a detection head. A multi-branch self-adaptive receptive field module of the backbone network dynamically fuses different receptive field characteristics through a data-dependent soft switch to realize multi-scale target perception; the feature pre-correction structure of the feature fusion network performs frequency domain decomposition and recombination on the features before cross-scale fusion, and improves the quality of the features under shielding through a low-frequency smooth semantic region and a high-frequency enhanced edge; and the detection head outputs the prediction box, the confidence and the category. And training the initial detection model by minimizing joint loss including positioning, confidence and classification to obtain a final tool detection model. According to the invention, the detection precision and robustness of the multi-scale shielding tool in the railway construction scene are effectively improved.

Description

Technical Field

[0001] This invention relates to the fields of railway operation and maintenance safety and computer vision technology, and in particular to a training method, detection method and system for a multi-scale occlusion tool detection model. Background Technology

[0002] Completing high-intensity operations and accurately inventorying all tools and equipment within the limited "maintenance window" is crucial for ensuring driving safety. Currently, this mainly relies on manual inventory, which is inefficient and prone to errors in complex environments such as nighttime, rain, and snow. If a tool is left on the track, it can easily lead to a major safety accident.

[0003] In the field of computer vision technology, general-purpose object detection models such as YOLO have been widely used in common scenarios, but they are difficult to directly adapt to the special requirements of railway tool inspection. Specifically, the tools vary greatly in size, making it difficult for the model to capture features of both large and small targets simultaneously; the cluttered and heavily occluded layout on-site leads to a high rate of missed detections; in addition, the limited computing power of handheld devices on-site makes it impossible for complex models to meet real-time requirements.

[0004] Existing improvement studies mostly optimize through attention mechanisms or feature pyramids, but they often lack dedicated datasets for railway scenarios, resulting in insufficient model generalization ability. Overall, current technologies are still limited by the "multi-scale, high occlusion, and low computing power" field conditions, making it impossible to achieve reliable and efficient automatic inventory of tools and equipment.

[0005] Therefore, there is an urgent need in this field for a railway tool detection and identification technology solution that can specifically address the above-mentioned problems. Summary of the Invention

[0006] In view of this, embodiments of the present invention provide a training method, detection method and system for a multi-scale occlusion tool detection model, in order to solve the problem of low detection accuracy caused by large differences in tool scale, severe mutual occlusion and limited computing power of edge devices in the prior art.

[0007] One aspect of the present invention provides a training method for a multi-scale occlusion tool detection model, the method comprising the following steps: A training method for a multi-scale occlusion detection model, comprising the following steps: A training sample set containing multiple samples is obtained. Each sample contains an image of railway construction tools and corresponding annotation information. The annotation information includes the target existence label, category label, and bounding box coordinate label for each tool in the image. The railway construction tool image contains a scene where multi-scale tools with significant size differences occlude with each other. An initial target tool detection model is constructed, comprising a backbone network, a feature fusion network, and a detection head. The backbone network includes a multi-branch adaptive receptive field module for processing the input tool image. The multi-branch adaptive receptive field module, for the multi-scale target tool, analyzes the local contextual information of the input features and dynamically fuses features extracted by convolutional kernels with different dilation rates based on a data-dependent switching strategy, outputting a feature map containing the multi-scale target tool. The feature fusion network includes a feature pre-correction and fusion structure. For mutually occluded scenes, before cross-scale feature fusion, the feature pre-correction and fusion structure performs frequency domain decomposition and recombination on the feature map to achieve feature pre-correction, including: smoothing the semantic consistency region of the feature map through low-frequency filtering and enhancing the edge features of the target in the feature map through high-frequency filtering, outputting an enhanced fused feature map. The detection head processes the enhanced fused feature map and outputs the predicted bounding box, prediction confidence, and prediction category for the multi-scale target tool. A bounding box regression loss is constructed based on the deviation between the predicted bounding box and the bounding box coordinate label; a confidence loss is constructed based on the deviation between the predicted confidence score and the target existence label; and a classification loss is constructed based on the deviation between the predicted category and the category label. A joint loss is constructed based on the bounding box regression loss, the confidence loss, and the classification loss. The initial target tool detection model is trained using the training sample set, and the parameters of the initial target tool detection model are iteratively updated by minimizing the joint loss to obtain the multi-scale occlusion tool detection model.

[0008] In some embodiments of the present invention, the multi-branch adaptive receptive field module implements the data-dependent switching strategy through a lightweight network consisting of a filling layer, a pooling layer, a convolutional layer, and an activation function connected in sequence, to generate soft switching weights for fusing features extracted by convolutional kernels with at least two different dilation rates.

[0009] In some embodiments of the present invention, the backbone network further includes a residual connection efficient operator; the residual connection efficient operator processes the input features through window multi-head self-attention operation, depthwise separable convolution operation and cross-spatial attention operation executed in parallel or sequentially, and performs residual connection on the output features obtained by the group of processing operations to enhance the feature extraction capability of the multi-scale target tool after occlusion.

[0010] In some embodiments of the present invention, the feature fusion network is a pre-corrected bidirectional feature pyramid network, which is composed of a front-end frequency-aware feature fusion module and a subsequent bidirectional feature pyramid network connected in series, so as to first perform frequency domain pre-correction on the input feature map and then perform cross-scale feature fusion.

[0011] In some embodiments of the present invention, the bounding box regression loss is EIoU loss, the confidence loss is BCEWithLogitsLoss loss, and the classification loss is Focal Loss loss.

[0012] In some embodiments of the present invention, prior to training using the training sample set, data augmentation processing is further performed on the tool image; the data augmentation processing includes at least one of the following operations: Mosaic stitching is performed on the tool images to generate a synthetic training image containing denser multi-scale tool targets; The tool image is subjected to copy-paste occlusion simulation to generate the synthetic training image with more complex mutual occlusion relationships.

[0013] In some embodiments of the present invention, after iteratively updating the parameters of the initial target tool detection model by minimizing the joint loss, a lightweight processing step is further included: The updated initial target tool detection model is subjected to knowledge distillation and model quantization processes in sequence to obtain the multi-scale occlusion tool detection model.

[0014] A detection method for a multi-scale occlusion tool detection model, the method comprising: Acquire images of the railway construction tools and equipment to be inspected; The multi-scale occlusion tool detection model is obtained by inputting the image of the tool to be detected into the training method of the multi-scale occlusion tool detection model described in any of the above-mentioned methods, and outputs the multi-scale target tool location, confidence level and category information in the tool image.

[0015] A multi-scale occlusion detection system includes a processor, a memory, and a computer program or instructions stored in the memory. The processor is used to execute the computer program or instructions, and when the computer program or instructions are executed, the system performs the steps of the method described above.

[0016] A computer-readable storage medium having a computer program or instructions stored thereon, which, when executed by a processor, implement the steps of the method as described in any of the preceding claims.

[0017] This invention provides a training method, detection method, and system for a multi-scale occluded tool detection model. The method includes: acquiring a training sample set containing images and annotation information of railway tools with multi-scale occlusion, effectively constructing a data foundation covering real and complex scenarios; constructing an initial detection model including a backbone network, a feature fusion network, and a detection head, wherein the backbone network, by embedding a multi-branch adaptive receptive field module, dynamically fuses features of different receptive fields based on data-dependent soft switching, realizing adaptive feature extraction for targets of different scales, and solving the problem of insufficient multi-scale perception capability of traditional models; the feature fusion network, by introducing a feature pre-correction structure before cross-scale fusion, performs frequency domain decomposition and recombination of the feature map, uses low-frequency filtering to smooth semantically inconsistent regions within a class, and simultaneously restores and enhances the edge details of the target through high-frequency filtering, significantly improving the model's feature discrimination and boundary preservation capabilities under severe occlusion conditions; the detection head outputs predicted bounding boxes, confidence scores, and categories, and performs end-to-end optimization of the model by constructing and minimizing a joint loss function containing localization, confidence scores, and classification, to obtain the final high-performance detection model.

[0018] Furthermore, by combining the soft-switching strategy in the multi-branch adaptive receptive field module with the frequency domain processing in the feature pre-correction structure, the model simultaneously achieves excellent scale adaptability and occlusion robustness during training. Ultimately, it realizes accurate and stable detection of multi-scale, mutually occluded tools in railway construction scenarios, effectively improving the efficiency and safety of work inventory.

[0019] Additional advantages, objects, and features of the invention will be set forth in part in the description which follows, and will also become apparent in part to those skilled in the art upon studying the description, or may be learned by practice of the invention. The objects and other advantages of the invention can be realized and obtained by means of the structures specifically pointed out in the description and drawings.

[0020] Those skilled in the art will understand that the objectives and advantages achievable with the present invention are not limited to those specifically described above, and that the above and other objectives achievable with the present invention will become clearer from the following detailed description. Attached Figure Description

[0021] The accompanying drawings, which are provided to further illustrate the invention and form part of this application, are not intended to limit the scope of the invention.

[0022] Figure 1 This is a flowchart illustrating the training method for a multi-scale occlusion tool detection model according to an embodiment of the present invention.

[0023] Figure 2 This is a schematic diagram of the algorithm architecture of the multi-scale occlusion tool detection model according to another embodiment of the present invention.

[0024] Figure 3 This is a schematic diagram of the principle of the multi-branch adaptive receptive field module (SAConv) used in the backbone network of the training method of the multi-scale occlusion tool detection model described in another embodiment of the present invention.

[0025] Figure 4 This is a schematic diagram illustrating the principle of the residual connection high-efficiency operator used in the backbone network of the training method for the multi-scale occlusion tool detection model according to another embodiment of the present invention.

[0026] Figure 5 This is a schematic diagram of the structure of the frequency-aware feature fusion module (FreqFusion) used in the feature fusion network of the training method of the multi-scale occlusion tool detection model in another embodiment of the present invention. Detailed Implementation

[0027] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the embodiments and accompanying drawings. Here, the illustrative embodiments and descriptions of this invention are used to explain the invention, but are not intended to limit the invention.

[0028] It should also be noted that, in order to avoid obscuring the invention with unnecessary details, only the structures and / or processing steps closely related to the solution according to the invention are shown in the accompanying drawings, while other details that are not closely related to the invention are omitted.

[0029] In the fields of railway operation and maintenance safety and computer vision technology, existing tool and equipment inventory technologies mainly rely on manual visual inspection and the direct application of general target detection models. Faced with key challenges such as significant differences in tool and equipment size at construction sites, cluttered placement and severe occlusion, and limited computing power of edge devices, existing technical solutions have three main shortcomings in practical applications: First, the lack of dedicated datasets for complex railway scenarios results in weak model generalization ability; second, general detection models struggle to adapt to the multi-scale features of both large and small tools, and exhibit low sensitivity to occluded targets; third, the high computational complexity of the models fails to meet the real-time requirements of edge devices.

[0030] In view of this, one aspect of the present invention provides a training method for a multi-scale occlusion tool detection model, such as... Figure 1 As shown, the method includes the following steps S101~S103: S101: Obtain a training sample set containing multiple samples. Each sample contains an image of railway construction tools and corresponding annotation information. The annotation information includes the target existence label, category label, and bounding box coordinate label of each tool in the image. The railway construction tool images contain scenes where multi-scale tools with significant size differences occlude with each other.

[0031] S102: Construct an initial target tool detection model comprising a backbone network, a feature fusion network, and a detection head. The backbone network includes a multi-branch adaptive receptive field module for processing input tool images. For multi-scale target tools, the multi-branch adaptive receptive field module analyzes the local contextual information of the input features and dynamically fuses features extracted by convolutional kernels with different dilation rates based on a data-dependent switching strategy, outputting a feature map containing multi-scale target tools. The feature fusion network includes a feature pre-correction and fusion structure. For mutually occluded scenes, before cross-scale feature fusion, the feature pre-correction and fusion structure performs frequency domain decomposition and recombination on the feature map to achieve feature pre-correction, including: smoothing semantically consistent regions of the feature map through low-frequency filtering and enhancing the edge features of the target in the feature map through high-frequency filtering, outputting an enhanced fused feature map. The detection head processes the enhanced fused feature map and outputs the predicted bounding box, prediction confidence, and prediction category for multi-scale target tools.

[0032] S103: Construct a bounding box regression loss based on the deviation between the predicted bounding box and the bounding box coordinate label; construct a confidence loss based on the deviation between the predicted confidence score and the target existence label; construct a classification loss based on the deviation between the predicted category and the category label; construct a joint loss based on the bounding box regression loss, confidence loss, and classification loss; train the initial target tool detection model using the training sample set; and iteratively update the parameters of the initial target tool detection model by minimizing the joint loss to obtain a multi-scale occlusion tool detection model.

[0033] In step S101, a large number of images of tools and equipment collected from the railway construction site are acquired. The scenes cover various lighting and environmental conditions, including daytime, nighttime, rain, and snow. The images ensure that the placement of the tools and equipment includes typical scenarios with significant size differences and overlapping / occlusion. A labeling tool is used to finely annotate the tools and equipment targets in each image. The annotation information includes: target presence label, category label, and bounding box coordinate label. The target presence label is a binary label indicating whether a target exists at that location. The category label indicates the specific category of the tool or equipment, such as a pickaxe, wrench, or electric pick. The bounding box coordinate label uses a format such as (x_min, y_min, x_max, y_max) to represent the target's location.

[0034] In some embodiments, publicly available cross-domain datasets, such as the CLCXray dataset which contains samples of various occlusion scenarios, may be introduced to initially verify the generality of the model architecture.

[0035] In some embodiments, in order to ensure the accurate identification of railway special tools by the detection model, after acquiring the original images of the actual tools used in railway maintenance windows through on-site photography, these original images can be processed. Specifically, the tool targets are separated from the complex background using image processing software, and an automated script program is used to randomly scale, rotate, and transform the separated targets, and then re-synthesize them with different background images, thereby generating training samples in batches that simulate multi-scale distribution and high occlusion conditions on-site.

[0036] In some embodiments, a training sample set can be constructed by combining the public dataset CLCXray with the private dataset RWTools to balance the model's generalization ability and the recognition accuracy of railway-specific tools. The dataset is divided into a training set, a validation set, and a test set in an approximately 8:1:1 ratio.

[0037] In step S102, a neural network model based on a single-stage target detection framework is constructed. This model mainly consists of three parts: a backbone network, a feature fusion network, and a detection head.

[0038] The backbone network is responsible for extracting multi-level features from the input image, and its core is an embedded multi-branch adaptive receptive field module. The multi-branch adaptive receptive field module dynamically adjusts the receptive field size through a data-dependent switching strategy to capture features of targets at different scales.

[0039] In some embodiments, the multi-branch adaptive receptive field module can employ the C3k2_SAConv module, and the aforementioned switching strategy is implemented by a lightweight network. This lightweight network consists of a padding layer, a pooling layer, a convolutional layer, and an activation function connected sequentially, and its output generates a soft switching weight for each spatial location. The method uses at least two convolutional kernels with different dilation rates in parallel, such as 3x3 convolutional kernels with dilation rates of 1 and 3, to extract features emphasizing local details and global context, respectively. Finally, the module dynamically weights and fuses the features extracted by convolutional kernels with different dilation rates based on the generated soft switching weights, outputting a feature map containing multi-scale target information.

[0040] In other embodiments, the backbone network also includes a residual connection efficient operator to enhance the feature extraction capability of the detection model in occluded scenarios. This operator processes the input features through parallel or sequential window attention (EW-MHSA), depthwise separable convolution (DW-Conv), and cross-spatial attention (EMAttention) operations. Window attention strengthens the correlation between similar pixels, depthwise separable convolution maintains feature diversity while reducing computational overhead, and cross-spatial attention helps capture clearer semantic boundaries. The output features obtained from this set of processing operations are then residually connected and fused. This design effectively mitigates feature loss or confusion caused by stacked tools occluding objects, improving the detection model's ability to preserve and recover features of occluded targets.

[0041] The feature fusion network receives multi-scale feature maps output by the backbone network and is responsible for fusing these features to improve detection performance. This network employs a pre-corrected bidirectional feature pyramid structure, and its core innovation lies in adding a frequency domain pre-correction step before feature fusion.

[0042] Specifically, the feature fusion network consists of a front-end frequency-aware feature fusion module and a subsequent bidirectional feature pyramid network connected in series, forming a "correction first, then fusion" processing flow.

[0043] In some embodiments, the pre-processed frequency-aware feature fusion module can be a FreqFusion module, and the bidirectional feature pyramid network can be a BiFPN network. The FreqFusion frequency-aware feature fusion module achieves feature pre-correction by performing frequency domain decomposition and reconstruction operations. This operation mainly includes two parallel processing branches: one branch performs low-frequency filtering on the input feature map to smooth its features, aiming to fuse regions with high semantic consistency in the feature map and alleviate intra-class feature differences; the other branch performs high-frequency filtering on the input feature map to enhance the high-frequency components of its features, aiming to sharpen the edge contours of the target in the feature map and alleviate boundary blurring caused by downsampling or occlusion.

[0044] The features, after low-frequency smoothing and high-frequency enhancement, are weighted and fused to output an enhanced fused feature map. This feature map is then input into a bidirectional feature pyramid network, where cross-scale, bidirectional feature transfer and weighted fusion are performed to ultimately output a fused feature map with rich semantic and detailed information suitable for use by the detection head.

[0045] In other embodiments, the aforementioned low-frequency and high-frequency filtering are implemented using adaptively generated filter kernels. Specifically, an adaptive low-pass filter generator, consisting of convolutional and normalization layers, processes the input features to dynamically generate a low-pass filter kernel for smoothing. Simultaneously, the low-pass filter kernel is subtracted from a unit impulse kernel to obtain the corresponding high-pass filter kernel. Then, the generated adaptive low-pass and high-pass filter kernels are used to perform convolution operations on the input features, thereby achieving adaptive low-frequency smoothing and high-frequency enhancement. This adaptive strategy allows the feature pre-correction process to be dynamically adjusted according to the content of the input features, improving the accuracy and flexibility of pre-correction.

[0046] By introducing this pre-correction strategy, the feature fusion network can effectively improve the quality of feature maps before multi-scale feature fusion. Low-frequency smoothing operations reduce the feature variance within similar targets, making the responses of similar features more consistent during subsequent fusion; high-frequency enhancement operations strengthen the details such as the edges of the targets, which is beneficial for the localization and recognition of small targets or targets with blurred boundaries. This structure improves the overall detection accuracy and robustness for tool targets with mutual occlusion and varying scales.

[0047] The detection head is used to process the enhanced fused feature map and output the predicted bounding box, prediction confidence and prediction category of the multi-scale target tool.

[0048] In step S103, a bounding box regression loss is constructed based on the deviation between the predicted bounding box and the bounding box coordinate label; a confidence loss is constructed based on the deviation between the predicted confidence and the target existence label; and a classification loss is constructed based on the deviation between the predicted category and the category label. A joint loss is constructed based on the above three losses. The initial target tool detection model is trained using the training sample set, and the parameters of the model are iteratively updated by minimizing the joint loss to obtain a multi-scale occlusion tool detection model.

[0049] Before training begins, data augmentation is performed on the tool images to improve model robustness.

[0050] In some embodiments, the data augmentation process includes at least one of the following operations: performing Mosaic stitching on tool images to stitch multiple images and their annotation information into a single training image, thereby generating a synthetic training image containing denser and multi-scale tool targets; performing Copy-Paste occlusion simulation on tool images to randomly copy and paste tool targets to other locations in the training image, thereby generating a synthetic training image with more complex mutual occlusion relationships.

[0051] During the training process, three losses are first calculated based on the prediction information output by the detection head and the labeled information in the training samples.

[0052] In some embodiments, to achieve better training results, the three loss functions described above are implemented in a specific high-level form. The bounding box regression loss is calculated using the difference between the predicted bounding box and the labeled bounding box coordinates, for example, using the EIoU loss. This loss function, in addition to the traditional IoU loss, considers the difference in center point distance and aspect ratio, thereby guiding the bounding box regression more accurately.

[0053] The confidence loss is calculated by utilizing the difference between the predicted confidence level and the target existence label, for example, by using the BCEWithLogitsLoss loss, which is more stable in numerical calculation.

[0054] The classification loss is calculated by utilizing the difference between the predicted class and the class label, such as using Focal Loss, which effectively alleviates the problem of class imbalance by adjusting the weights.

[0055] Subsequently, the three losses are weighted and summed according to preset weights to form a joint loss function.

[0056] In some embodiments, model training employs a stochastic gradient descent optimizer combined with a cosine annealing strategy to dynamically schedule the learning rate. The training process is accomplished through iterative operations: inputting a batch of training images into the initial target tool detection model, performing forward propagation to obtain prediction results; calculating the joint loss; performing backpropagation to calculate the gradient; and finally, updating the model parameters according to the optimizer rules.

[0057] In some embodiments, after iteratively updating the model parameters by minimizing the joint loss, a lightweight processing step is also included. Specifically, the updated, convergent initial target tool detection model undergoes knowledge distillation and model quantization sequentially. Knowledge distillation utilizes a more powerful teacher model to guide the current student model; model quantization converts the model's weights and activation values ​​from high-precision floating-point numbers to low-bit integer representations. After these processes, a multi-scale occlusion tool detection model with significantly reduced parameter and computational costs is obtained, suitable for deployment on edge devices.

[0058] On the other hand, the present invention also provides a detection method for a multi-scale occlusion tool detection model, the method comprising: Acquire images of the railway construction tools and equipment to be inspected; The multi-scale occlusion tool detection model is obtained by inputting the tool image to be detected into the training method of the multi-scale occlusion tool detection model mentioned above, and outputting the multi-scale target tool location, confidence level and category information in the tool image.

[0059] On the other hand, the present invention also provides a multi-scale occlusion tool detection system, including a processor, a memory, and a computer program or instructions stored in the memory. The processor is used to execute the computer program or instructions, and when the computer program or instructions are executed, the system implements the steps of any of the above methods.

[0060] On the other hand, the present invention also provides a computer-readable storage medium having a computer program or instructions stored thereon, which, when executed by a processor, implement the steps of any of the methods described above.

[0061] The present invention will now be described with reference to a specific embodiment: This embodiment is based on the YOLOv11 model architecture and features targeted improvements to construct a multi-scale occlusion detection model for railway construction sites. The following sections will elaborate on the complete process, including the construction of the training dataset, the detailed design of the model's core modules, the training strategy and optimization methods, and the lightweight processing and deployment of the model. The specific technical solutions adopted are as follows: 1. Algorithm Architecture for Detection and Recognition of Railway Construction Tools and Equipment The method for detecting and identifying construction tools and equipment in multi-scale occlusion scenarios consists of three parts: data acquisition and annotation, model training and optimization, and edge inference and application. It covers the entire process from "data construction to model training to quantitative deployment to field application," with each module working together to achieve intelligent detection and inventory of tools and equipment. The specific algorithm architecture is as follows: Figure 2 As shown: (1) Data collection and annotation: Take pictures of the tools and equipment in the railway construction operation scene, and then use annotation software to manually annotate and data augment the tools and equipment in the pictures. Responsible for on-site tool and equipment image collection, dataset construction, annotation and cross-domain verification data adaptation.

[0062] (2) Model training and optimization: Based on the YOLOv11 model, improvements were made to the backbone network module, neck module and convolutional layer respectively to construct the YOLO-EBEM architecture, and model training, ablation optimization, loss function tuning and quantization distillation were completed. (3) Edge reasoning and application: The model is lightweighted to reduce the number of parameters to adapt to mobile devices. The quantized model is deployed to edge devices to perform image preprocessing, forward reasoning, and result postprocessing, and is linked with the tool and equipment ledger to achieve inventory. 2. Dataset Setup In the process of building the dataset for the detection and identification algorithm of railway construction tools, in order to ensure the generalization effect of the model and its adaptability to railway business applications, a combination of public and private datasets was used to fully verify the algorithm model. The specific dataset details are as follows: (1) The public dataset CLCXray (9565 images, 12 categories, including occluded scene samples such as knives and liquid containers) was divided into a training set (7652 images), a validation set (956 images), and a test set (956 images) in an 8:1:1 ratio to verify the generalization ability of the model. (2) Private dataset RWTools: 862 images of tools and equipment at the skylight operation site were taken with a GoPro camera. The tools and equipment were separated from the background using Photoshop. Multi-scale, high-occlusion scene samples were generated through automated code. After screening, 2000 labeled images were obtained, covering 7 categories including pickaxe, gauge, electric pickaxe, and wrench. They were divided into training set (1429 images), validation set (286 images), and test set (285 images) in an 8:1:1 ratio. 3. Improvements to the convolution mechanism This solution is based on the YOLOv11 model, replacing the original C3k2 module with the C3k2_SAConv module. It also introduces multi-branch dilated convolution and shared weight strategies to effectively expand the receptive field while maintaining the same number of parameters. This allows it to capture both global features of large-sized equipment and local details of small tools. The principle of the SAConv module is as follows: Figure 3 As shown: The C3k2_SAConv module innovatively inserts an attention mechanism into the C3k2 layer instead of a typical Conv convolutional layer, which better enables accurate recognition of tools of different sizes in complex scenes. The improvement method is as follows.

[0063] For the input C3k2_SAConv learns to output its mapping Y For each position The size of the receptive field depends on the data-dependent soft switch. Decision made. The calculation process for this module is as follows: 1) Calculate the location-related fusion weights based on the input features. The calculation formula is as follows: ; in, This represents the input feature map; C represents the number of channels; Indicates altitude; Indicates width; This indicates data dependence on soft-switching weights; This indicates that a 2-pixel edge padding operation is performed on the input feature map X; This indicates that a 5×5 global average pooling operation is performed on the filled features; express Convolution operation; This represents the Sigmoid activation function; This represents the soft-switch weights that depend on the data.

[0064] 2) Using the generated weights, the convolutional features from two different receptive fields are mixed. The calculation formula is as follows: ; in, This represents the local neighborhood of the input feature map at position (i,j); Indicates ordinary Convolution operation, dilation 1; Indicates a cavity Convolution operation, dilation 3; This represents the value of the output feature map at position (i,j).

[0065] This approach integrates C3k2_SAConv into the overall network structure. By embedding the soft-switching strategy and multi-scale feature selection into the CSP gradient flow, and embedding the SAConv mechanism into each C3k module, a global lightweight embedding is achieved while preserving the CSP multi-gradient flow, enabling the model to better identify multi-scale targets.

[0066] 4. High-efficiency operators for residual connection In the backbone network, this scheme proposes an efficient residual connection operator, the principle of which is as follows: Figure 4 As shown: Following the MMB (Expand-Compute-Compress) architecture, it integrates EW-MHSA (Window Attention), DW-Conv (Depthly Separable Convolution), and EMAttention (Cross-Spatial Attention). Through residual fusion, it ensures that intra-class features are consistent and boundaries are clear in occluded scenarios. In severe occlusion scenarios, the model can significantly improve its ability to perceive occluded targets and enhance feature focusing in occluded regions.

[0067] First, EW-DW performs two steps: window attention mechanism and channel-wise convolution. Then, EW-MHSA is used to enhance pixels of the same type, thereby achieving the fusion of information between channels and making the internal features of the object more consistent.

[0068] Secondly, using DW-Conv to process each channel independently maintains diversity, significantly improving the model's ability to perceive occluded targets while retaining low computational overhead. Subsequently, the EMAttention mechanism groups and rearranges input features and learns across spatial boundaries, achieving clearer edges and more consistent semantics with almost no increase in the number of parameters. This avoids information loss caused by channel compression in traditional attention and strengthens feature focusing in occluded regions.

[0069] Finally, by using residual connections to connect EMAttention and EW-DW processed features, key information can be preserved and feature representation can be mutually enhanced, significantly improving the effectiveness of the model.

[0070] 5. Pre-correction bidirectional feature pyramid This scheme pre-lays a FreqFusion frequency-aware feature fusion module on top of the BiFPN feature fusion network, forming a "correction-fusion" feature processing mode. Furthermore, it embeds a C3k2_SAConv module at key nodes to mitigate cross-scale semantic bias. The FreqFusion principle diagram is shown below. Figure 5 As shown, the specific principle is as follows: The compressed features after fusion The ALPF generator will perform this. Convolution and Softmax normalization operations produce a low-pass kernel for each pixel location. Upsampling yields smooth features When the ALPF generator cannot correct inconsistencies in large or thin boundary regions, the current feature is replaced with a neighboring feature of high similarity. This process involves two steps: first, [the following steps are performed on the ALPF generator]... The cosine similarity between each pixel (i,j) and its neighbors p,q is then calculated, followed by resampling to obtain the predicted offset. ∈ , where G is the number of groups divided by channel.

[0071] The formula for calculating cosine similarity is: ; in, Represents the center point of layer l With neighboring points Cosine similarity between them; Representation of feature map The value at position (i,j) in channel c.

[0072] The formula for calculating the predicted offset is: ; in, Indicates the predicted offset; This represents the compressed feature map of the l-th layer; This represents a cosine similarity graph.

[0073] AHPF is used to recover low-level high-frequency boundaries. First, a low-pass kernel is obtained using a 3x3 convolution kernel (Softmax). The calculation formula is as follows: ; in, Represents the input feature map; This indicates the adaptive low-pass filter kernel for the output.

[0074] Then use unit impact nucleus Subtract the low-throughput core to get the high-throughput core Finally, a high-pass kernel is convolved into the low-level original features. Then, add the resulting residuals to the original feature calculation formula: ; in, This represents the original input feature map of the lower layer; This represents the enhanced feature map after high-frequency recovery processing.

[0075] 6. Training strategy optimization (1) Optimizer and parameters: The stochastic gradient descent (SGD) optimizer was used with an initial learning rate of 0.01, a learning rate momentum of 0.937, a weight decay coefficient of 0.0005, a batch size of 16, and 150 training rounds. (2) Positive sample allocation: The Task-Aligned Assigner strategy is adopted to accurately match positive samples in multi-scale and occluded scenarios. (3) Learning rate scheduling: adopt cosine annealing strategy to dynamically adjust the learning rate to avoid local optima. (4) Loss Function: The loss function based on the YOLOv11 model includes localization loss, confidence loss, and classification loss. The localization loss uses EIoU (Enhanced IoU) loss, which measures the deviation between the predicted and ground truth bounding boxes in terms of position and size, ensuring the predicted boxes are as consistent as possible with the ground truth boxes. The confidence loss uses BCEWithLogitsLoss, which measures the deviation between the model's predicted anchor box containing the target and the actual situation. The classification loss uses Focal Loss multi-label detection, which measures the deviation between the model's predicted target class probability and the actual class label, enabling the model to accurately identify the class of the target within the anchor box.

[0076] (5) Data augmentation: Combine strategies such as Mosaic stitching, Copy-Paste occlusion enhancement, color jitter, and random cropping to enhance the model's adaptability to complex scenes. 7. Beneficial effects In the model architecture design, the backbone network incorporates the C3k2_SAConv module, utilizing a data-dependent soft switching mechanism to dynamically fuse convolutional features with different dilation rates. This enables the model to adaptively capture multi-scale targets, ranging from large machinery to small tools. Testing on the private dataset RWTools shows that this design achieves a multi-scale target mean accuracy (mAP50-95) of 81.6%, a 4.2% improvement over the baseline model YOLOv11n. Simultaneously, the efficient residual connection operator in the backbone network integrates window attention and cross-spatial learning mechanisms, significantly enhancing the model's ability to preserve occluded features. On the same dataset, this improves recall to 73.5%, a 6.2% improvement over the baseline model.

[0077] The feature fusion network employs a pre-corrected bidirectional feature pyramid structure. Before standard bidirectional feature pyramid fusion, this structure performs low-frequency smoothing and high-frequency enhancement on the feature maps through a frequency-aware feature fusion module, effectively mitigating intra-class feature inconsistencies and boundary blurring issues. This improvement enhances the model's detection performance for small targets, increasing the F1 score by 3.8%, while also improving the accuracy and efficiency of multi-scale feature fusion.

[0078] During the model training and optimization phases, a joint optimization objective including advanced loss functions such as EIoU and Focal Loss was adopted, along with targeted data augmentation strategies such as Mosaic and Copy-Paste, to enhance the model's robustness to complex occlusion and lighting changes. Validation on the cross-domain dataset CLCXray showed that the model achieved an mAP50-95 of 77.4%, a 0.8% improvement over the baseline, demonstrating its excellent generalization ability.

[0079] To meet on-site deployment requirements, the trained model underwent knowledge distillation and quantization, resulting in a final model parameter size of only 2.981M and computational cost controlled within 7.7 GFLOPs. The quantized model can be deployed on inference engines such as TensorRT or ncnn, with a single-frame inference latency of no more than 300ms on edge devices, meeting real-time inventory requirements. This model is compatible with various hardware devices, including handheld terminals and edge computing boxes, and can interface with existing railway operation and maintenance ledger systems, enabling rapid deployment and application without large-scale hardware modifications.

[0080] In summary, this invention provides a training method, detection method, and system for a multi-scale occluded tool detection model. The method includes: acquiring a training sample set containing images and annotation information of railway tools with multi-scale occlusion, effectively constructing a data foundation covering real and complex scenarios; constructing an initial detection model comprising a backbone network, a feature fusion network, and a detection head, wherein the backbone network, by embedding a multi-branch adaptive receptive field module, dynamically fuses features from different receptive fields based on data-dependent soft switching, achieving adaptive feature extraction for targets of varying scales and solving the problem of insufficient multi-scale perception capability in traditional models; the feature fusion network, by introducing a feature pre-correction structure before cross-scale fusion, performs frequency domain decomposition and reorganization of the feature map, uses low-frequency filtering to smooth semantically inconsistent regions within a class, and simultaneously restores and enhances the edge details of the target through high-frequency filtering, significantly improving the model's feature discrimination and boundary preservation capabilities under severe occlusion conditions; the detection head outputs predicted bounding boxes, confidence scores, and categories, and performs end-to-end optimization of the model by constructing and minimizing a joint loss function containing localization, confidence scores, and classification, obtaining the final high-performance detection model.

[0081] Furthermore, by combining the soft-switching strategy in the multi-branch adaptive receptive field module with the frequency domain processing in the feature pre-correction structure, the model simultaneously achieves excellent scale adaptability and occlusion robustness during training. Ultimately, it realizes accurate and stable detection of multi-scale, mutually occluded tools in railway construction scenarios, effectively improving the efficiency and safety of work inventory.

[0082] Those skilled in the art will understand that the exemplary components, systems, and methods described in conjunction with the embodiments disclosed herein can be implemented in hardware, software, or a combination of both. Whether implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this invention. When implemented in hardware, it can be, for example, electronic circuits, application-specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, etc. When implemented in software, the elements of this invention are programs or code segments used to perform the desired tasks. The programs or code segments can be stored in a machine-readable medium or transmitted over a transmission medium or communication link via data signals carried in a carrier wave.

[0083] It should be clarified that the present invention is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present invention is not limited to the specific steps described and shown. Those skilled in the art can make various changes, modifications, and additions, or change the order of steps, after understanding the spirit of the present invention.

[0084] In this invention, features described and / or illustrated for one embodiment may be used in the same or similar manner in one or more other embodiments, and / or combined with or in place of features of other embodiments.

[0085] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, various modifications and variations of the embodiments of the present invention are possible. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A training method for a multi-scale occlusion tool detection model, characterized in that, The method includes the following steps: A training sample set containing multiple samples is obtained. Each sample contains an image of railway construction tools and corresponding annotation information. The annotation information includes the target existence label, category label, and bounding box coordinate label for each tool in the image. The railway construction tool image contains a scene where multi-scale tools with significant size differences occlude with each other. An initial target tool detection model is constructed, comprising a backbone network, a feature fusion network, and a detection head. The backbone network includes a multi-branch adaptive receptive field module for processing the input tool image. The multi-branch adaptive receptive field module, for the multi-scale target tool, analyzes the local contextual information of the input features and dynamically fuses features extracted by convolutional kernels with different dilation rates based on a data-dependent switching strategy, outputting a feature map containing the multi-scale target tool. The feature fusion network includes a feature pre-correction and fusion structure. For mutually occluded scenes, before cross-scale feature fusion, the feature pre-correction and fusion structure performs frequency domain decomposition and recombination on the feature map to achieve feature pre-correction, including: smoothing the semantic consistency region of the feature map through low-frequency filtering and enhancing the edge features of the target in the feature map through high-frequency filtering, outputting an enhanced fused feature map. The detection head processes the enhanced fused feature map and outputs the predicted bounding box, prediction confidence, and prediction category for the multi-scale target tool. A bounding box regression loss is constructed based on the deviation between the predicted bounding box and the bounding box coordinate label; a confidence loss is constructed based on the deviation between the predicted confidence score and the target existence label; and a classification loss is constructed based on the deviation between the predicted category and the category label. A joint loss is constructed based on the bounding box regression loss, the confidence loss, and the classification loss. The initial target tool detection model is trained using the training sample set, and the parameters of the initial target tool detection model are iteratively updated by minimizing the joint loss to obtain the multi-scale occlusion tool detection model.

2. The training method for the multi-scale occlusion tool detection model according to claim 1, characterized in that, The multi-branch adaptive receptive field module implements the data-dependent switching strategy through a lightweight network consisting of a filling layer, a pooling layer, a convolutional layer, and an activation function connected in sequence, to generate soft switching weights for fusing features extracted by convolutional kernels with at least two different dilation rates.

3. The training method for the multi-scale occlusion tool detection model according to claim 1, characterized in that, The backbone network also includes a residual connection efficient operator; the residual connection efficient operator processes the input features through window multi-head self-attention operation, depthwise separable convolution operation and cross-spatial attention operation executed in parallel or sequentially, and performs residual connection on the output features obtained by the group of processing operations to enhance the feature extraction capability of the multi-scale target tool after occlusion.

4. The training method for the multi-scale occlusion tool detection model according to claim 1, characterized in that, The feature fusion network is a pre-corrected bidirectional feature pyramid network, which is composed of a front-end frequency-aware feature fusion module and a subsequent bidirectional feature pyramid network connected in series. It first performs frequency domain pre-correction on the input feature map and then performs cross-scale feature fusion.

5. The training method for the multi-scale occlusion tool detection model according to claim 1, characterized in that, The bounding box regression loss uses EIoU loss, the confidence loss uses BCEWithLogitsLoss loss, and the classification loss uses Focal Loss loss.

6. The training method for the multi-scale occlusion tool detection model according to claim 1, characterized in that, Before training using the training sample set, the method further includes data augmentation processing on the tool images; the data augmentation processing includes at least one of the following operations: Mosaic stitching is performed on the tool images to generate a synthetic training image containing denser multi-scale tool targets; The tool image is subjected to copy-paste occlusion simulation to generate the synthetic training image with more complex mutual occlusion relationships.

7. The training method for the multi-scale occlusion tool detection model according to claim 1, characterized in that, After iteratively updating the parameters of the initial target tool detection model by minimizing the joint loss, a lightweight processing step is also included: The updated initial target tool detection model is subjected to knowledge distillation and model quantization processes in sequence to obtain the multi-scale occlusion tool detection model.

8. A detection method for a multi-scale occlusion tool detection model, characterized in that, The method includes: Acquire images of the railway construction tools and equipment to be inspected; The image of the tool to be detected is input into the multi-scale occlusion tool detection model obtained by the training method of the multi-scale occlusion tool detection model according to any one of claims 1 to 7, and the multi-scale target tool location, confidence level and category information in the tool image are output.

9. A multi-scale occlusion tool detection system, comprising a processor, a memory, and a computer program or instructions stored in the memory, characterized in that, The processor is configured to execute the computer program or instructions, and when the computer program or instructions are executed, the system implements the steps of the method as described in any one of claims 1 to 8.

10. A computer-readable storage medium having a computer program or instructions stored thereon, characterized in that, When the computer program or instructions are executed by a processor, they implement the steps of the method as described in any one of claims 1 to 8.

Images