A chain hugging identification method, system, device and storage medium

Abstract

The present invention discloses a chain-lock recognition method, system, device, and storage medium, relating to the technical field of railway brake detection. The method comprises: inputting an image to be detected into a trained target detection model based on a first improved YOLOv8m network for detection to determine whether a chain exists in the image to be detected; if a chain exists, inputting an image of the chain detection frame area into a trained instance segmentation model based on a second improved YOLOv8m‑Seg network for chain segmentation; for the chain mask image output by the instance segmentation model, a parabola fitting algorithm is used to extract the centerline of the chain, and the degree of curvature of the chain is evaluated based on the ratio of the chain arc depth to the chain chord length; if the degree of curvature is greater than a preset curvature threshold, a brake risk warning is issued. The present invention achieves efficient detection and positioning of chains and precise segmentation of chains. Experiments have shown that the chain-lock recognition accuracy of the present invention exceeds 95%.

Description

Technical Field

[0001] The present invention relates to the technical field of railway brake detection, and in particular to a chain brake identification method, system, equipment and storage medium. Background Art

[0002] During the railway hump shunting operation, if the brake shoes hold the wheels and brake, the vehicle will operate with the brakes on. The shunting process of the vehicle with brakes on may cause it to stop, or even collide or derail, which will seriously affect the railway operation efficiency and may even cause safety accidents.

[0003] Intelligent identification of railway brake engagement is a key technology for railway transportation safety. Brake engagement occurs when all brake shoes remain pressed against the wheel treads during hump dismantling and shunting operations due to issues such as failure to properly ventilate the vehicle being shunted or failure to release the manual brake mechanism as required. This can cause the vehicle to stop mid-flight, leading to frequent collisions, derailments, and other accidents. Railway freight cars use two types of brakes to stop and control speed during operation: air brakes and manual brakes. Air brakes involve injecting air into a cylinder, pushing a piston and extending a bellows rod, which drives the brake shoes to engage the wheels, creating friction between the wheels. Manual brakes involve manually rotating the manual brake mechanism, which pulls the brake shoes against the wheels, creating friction between the wheels. Chain engagement occurs when the manual brake mechanism fails to release the brake mechanism as required during shunting operations, causing the brake shoes to engage the wheels, resulting in the vehicle operating with the brakes engaged.

[0004] In recent years, with the continuous emergence of new models in the field of deep learning, vehicle intelligent detection technology based on computer vision has also made great progress. In the field of target detection, mainstream algorithms are mainly divided into two categories: (1) Single-stage model: This type of model simplifies the target detection task into a regression and classification problem, directly predicting the target category and bounding box by inputting the image. Due to its simple structure, the single-stage model has a high detection speed and is suitable for real-time detection scenarios and devices with limited computing power. (2) Two-stage model: Unlike the single-stage model, the two-stage model divides the detection task into two stages. The first stage generates candidate regions, and the second stage further classifies and regresses these candidate regions. By gradually refining the target features, the model performs better in small target detection and complex scenes. However, the multi-stage computational characteristics make the model structure more complex, which correspondingly increases the computational overhead.

[0005] Therefore, it is worth studying how to use deep learning to dynamically detect and warn of railway brake risks in real time. Summary of the Invention

[0006] In view of the above problems, the present invention proposes a chain embrace identification method, system, device and storage medium to try to solve or alleviate one or more of the above problems.

[0007] According to one aspect of the present invention, a chain hugging identification method is proposed, the method comprising:

[0008] Inputting the image to be detected into the trained object detection model based on the first improved YOLOv8m network for detection to determine whether there is a chain in the image to be detected; wherein the improvements of the first improved YOLOv8m network include: adding a spatial pyramid pooling module and a partial self-attention module to the original YOLOv8m backbone network; and adding a coordinate attention mechanism after the neck network and before the head network;

[0009] If there is a chain in the image to be detected, a chain detection frame area image is extracted, and the chain detection frame area image is input into a trained instance segmentation model based on the second improved YOLOv8m-Seg network for chain segmentation to obtain a chain mask image; wherein, the improvements of the second improved YOLOv8m-Seg network include: adding a spatial pyramid pooling module and a partial self-attention module to the original YOLOv8m-Seg backbone network; adding a multi-scale sequence fusion module between the backbone network and the neck network; adding a feature fusion module between the neck network and the head network; for the chain mask image, a parabola fitting algorithm is used to extract the center line of the chain, and the curvature of the chain is evaluated according to the ratio of the chain arc depth to the chain chord length;

[0010] If the bending degree is greater than the preset bending threshold, a brake risk warning is issued.

[0011] Furthermore, inputting the image to be detected into the trained target detection model based on the first improved YOLOv8m network for detection to determine whether there is a chain in the image to be detected includes:

[0012] The image to be detected is input into the backbone network for feature extraction, including: using multiple C2f feature extraction modules in the backbone network to extract local features of the image and perform feature fusion; then, the features of different scales are aggregated through the spatial pyramid pooling module, including: processing the features of different scales through convolutional layers, batch normalization and SiLU activation functions to obtain feature maps with half the number of channels; passing through multiple maximum pooling layers in sequence to gradually reduce the size of the feature maps; the feature maps after pooling at different scales are spliced ​​in the channel dimension; then, through the partial self-attention module, the feature maps aggregated by the spatial pyramid pooling module are evenly divided into two parts, one part of the feature map enters the self-attention module to perform global information modeling through matrix operations between the query vector, key vector and value vector; the other part of the feature map is fused with the output of the self-attention module through jump connections;

[0013] The neck network is used to fuse, enhance and process the features extracted by the backbone network;

[0014] The coordinate attention mechanism is used to generate feature maps with enhanced position information. The method includes the following steps: first, global average pooling is performed on the features output by the neck network in the horizontal and vertical directions to extract compressed features in different directions; then, these features are fused through a set of convolutions with shared weights; the fused features are re-divided into horizontal and vertical parts, and the attention matrix is ​​generated by 1×1 convolution and sigmoid activation function respectively; finally, the features output by the neck network are weighted by the attention matrix through element-wise multiplication to obtain a feature map with enhanced position information;

[0015] The head network is used to transform the feature map enhanced with position information into the final object detection result.

[0016] Furthermore, the dimension of the query vector and the key vector in the partial self-attention module is half of the value vector; batch normalization is used instead of layer normalization for normalization.

[0017] Furthermore, inputting the chain detection frame area image into a trained instance segmentation model based on the second improved YOLOv8m-Seg network to perform chain segmentation includes:

[0018] The chain detection box area is input into the backbone network for feature extraction, including: using multiple C2f feature extraction modules in the backbone network to extract local image features and perform feature fusion; then, the features of different scales are aggregated through the spatial pyramid pooling module, including: processing the features of different scales through convolutional layers, batch normalization and SiLU activation functions to obtain feature maps with half the number of channels; passing through multiple maximum pooling layers in sequence to gradually reduce the size of the feature maps; the feature maps after pooling at different scales are spliced ​​in the channel dimension; then, through the partial self-attention module, the aggregated feature maps are evenly divided into two parts, one part of the feature map enters the self-attention module and performs global information modeling through matrix operations between the query vector, key vector and value vector; the other part of the feature map is fused with the output of the self-attention module through jump connections;

[0019] The features extracted by the backbone network are fused using a multi-scale sequence fusion module. This involves extracting feature maps at levels P2 to P5 from the backbone network, resizing them, and initially fusing low-level local detail information with high-level global semantic information through element-wise splicing to generate a comprehensive description of the target features. Subsequently, 1×1 convolution is used to reduce the channel dimension and map the features to a higher-level feature space. Finally, the RepVGG module with convolution kernels of 3 and 5 is used to extract edge details and contextual information of the target under different receptive fields.

[0020] The neck network is used to fuse, enhance and process the features extracted by the backbone network;

[0021] The fusion features extracted by the multi-scale sequence fusion module are fused with the feature map of the neck network P3 level using a feature fusion module, including: splicing the two feature maps in the channel dimension; generating a feature weight set through 3×3 convolution and Sigmoid activation function; adjusting the features to be fused according to the feature weight set; and fusing the adjusted two features through element-wise addition.

[0022] The head network is used to convert the feature map fused by the feature fusion module into the final instance segmentation result; the head network includes a segmentation head and a prediction head. The segmentation head is used to generate a high-resolution native mask of the target, and the prediction head is used to predict the target attributes, including the target box position, category information, and mask coefficients related to segmentation.

[0023] Furthermore, the formula for evaluating the degree of chain bending based on the ratio of the chain arc depth to the chain chord length is:

[0024]

[0025] in, is the curvature of the chain centerline, represents the chord length of the chain's centerline, Indicates the maximum depth of the chain's centerline relative to the chord length.

[0026] According to another aspect of the present invention, a chain hugging identification system is provided, the system comprising:

[0027] A chain detection module is configured to input an image to be detected into a trained object detection model based on a first improved YOLOv8m network to detect whether a chain exists in the image to be detected; wherein the improvements of the first improved YOLOv8m network include: adding a spatial pyramid pooling module and a partial self-attention module to the original YOLOv8m backbone network; and adding a coordinate attention mechanism after the neck network and before the head network;

[0028] A chain segmentation module is configured to extract a chain detection frame area image if a chain exists in the image to be detected, input the chain detection frame area image into a trained instance segmentation model based on the second improved YOLOv8m-Seg network to perform chain segmentation, and obtain a chain mask image; the improvements of the second improved YOLOv8m-Seg network include: adding a spatial pyramid pooling module and a partial self-attention module to the original YOLOv8m-Seg backbone network; adding a multi-scale sequence fusion module between the backbone network and the neck network; and adding a feature fusion module between the neck network and the head network;

[0029] The chain lock identification module is configured to use a parabola fitting algorithm to calculate the chain centerline for the chain mask image, and evaluate the degree of chain bending based on the ratio of the chain arc depth to the chain chord length; if the degree of bending is greater than a preset bending threshold, a brake risk warning is issued.

[0030] According to another aspect of the present invention, an electronic device is also proposed, which includes a memory, a processor and a computer program; wherein the computer program is stored in the memory and is configured to be executed by the processor to implement the above-mentioned link embrace identification method.

[0031] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the storage medium stores a computer program; the computer program is executed by a processor to implement the above-mentioned chain embrace identification method.

[0032] The beneficial technical effects of the present invention are:

[0033] The present invention proposes a chain brake recognition method, system, device and storage medium, which input the image to be detected into a trained target detection model based on a first improved YOLOv8m network for detection to determine whether there is a chain in the image to be detected; if there is a chain in the image to be detected, the chain detection frame area image is extracted, and the chain detection frame area image is input into a trained instance segmentation model based on a second improved YOLOv8m-Seg network for chain segmentation to obtain a chain mask image; for the chain mask image, a parabola fitting algorithm is used to calculate the chain centerline, and the degree of bending of the chain is evaluated according to the ratio of the chain arc depth to the chain chord length; if the bending degree is greater than a preset bending threshold, a brake risk warning is issued. Among them, the spatial pyramid pooling module and the partial self-attention module are introduced in the fourth stage of the original backbone network of the first improved YOLOv8m network; the coordinate attention mechanism is introduced in the third stage of the neck network; by adaptively adjusting the feature weights, the model can focus on the target boundary more accurately, thereby achieving efficient detection and positioning of the chain; the spatial pyramid pooling module and the partial self-attention module are added to the original backbone network of the second improved YOLOv8m-Seg network; the multi-scale sequence fusion module is added between the backbone network and the neck network; the feature fusion module is added between the neck network and the head network. By effectively fusing global context information and position information, the model's ability to analyze complex forms at different scales is enhanced, thereby achieving accurate segmentation of the target. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] The above and other objects, features and advantages of the exemplary embodiments of the present invention will become readily apparent by reading the following detailed description with reference to the accompanying drawings, in which several embodiments of the present invention are shown by way of example and not limitation, in which:

[0035] Figure 1 This is a flow chart of a link recognition method according to an embodiment of the present invention;

[0036] Figure 2 is another flow chart of a link embrace identification method according to an embodiment of the present invention;

[0037] Figure 3 1 is a schematic diagram of the structure of the first improved YOLOv8m network in an embodiment of the present invention;

[0038] Figure 4 2 is a schematic diagram of the structure of the second improved YOLOv8m-Seg network in an embodiment of the present invention;

[0039] Figure 5 2 is a schematic structural diagram of a chain embrace identification system according to an embodiment of the present invention. DETAILED DESCRIPTION

[0040] The principles and spirit of the present invention will be described below with reference to several exemplary embodiments. It should be understood that these embodiments are provided solely to enable those skilled in the art to better understand and implement the present invention, and are not intended to limit the scope of the present invention in any way. Rather, these embodiments are provided to make this disclosure more thorough and complete, and to fully convey the scope of the present disclosure to those skilled in the art.

[0041] Those skilled in the art will appreciate that embodiments of the present invention may be implemented as a system, apparatus, device, method, or computer program product. Therefore, the present disclosure may be implemented in the following forms: entirely in hardware, entirely in software (including firmware, resident software, microcode, etc.), or in a combination of hardware and software. It should be understood that any number of elements in the figures is for illustrative purposes only and not limiting, and any nomenclature is for distinction only and does not have any limiting meaning.

[0042] Chain-engagement recognition requires automatic identification and location of components such as the brake chain and cylinder chain under the vehicle's manual brake mechanism for different vehicle models. It also determines changes in the brake chain's morphology and calculates the degree of bending. By setting conditions and rules such as the brake chain fit score and the recognition region, combined with a set threshold, it identifies vehicles with engaged brakes and issues a warning. Considering the large number of irrelevant regions in high-resolution images and the long, strip-like structure of the target, which only accounts for a small proportion of the total image pixels, the target features are easily diluted, making it difficult for a single model to accurately identify the target. Therefore, chain-engagement recognition divides the process into two parts: object detection and instance segmentation. In the object detection phase, the first modified YOLOv8m network architecture is used to obtain the chain's ROI. In the instance segmentation phase, based on the chain region located in the object detection phase, the second modified YOLOv8m-Seg network is used to accurately segment the brake chain.

[0043] Therefore, the embodiment of the present invention proposes a chain embrace identification method, such as Figures 1 and 2 As shown, the method includes:

[0044] S1. Input the image to be detected into the trained target detection model based on the first improved YOLOv8m network for detection to determine whether there is a chain in the image to be detected;

[0045] S2. If there are chains in the image to be detected, extract the chain detection frame area image, input the chain detection frame area image into the trained instance segmentation model based on the second improved YOLOv8m-Seg network to perform chain segmentation, and obtain a chain mask image;

[0046] S3. For the chain mask image, a parabola fitting algorithm is used to calculate the chain centerline, and the curvature of the chain is evaluated according to the ratio of the chain arc depth to the chain chord length;

[0047] S4. If the bending degree is greater than the preset bending threshold, a brake risk warning is issued.

[0048] The method begins with S1. In S1, the image to be detected is input into the trained object detection model based on the first improved YOLOv8m network for detection to determine whether there is a chain in the image to be detected.

[0049] According to an embodiment of the present invention, the process of training the target detection model based on the first improved YOLOv8m network includes:

[0050] S11. Obtain a training dataset. The target detection model is trained and tested using a self-made dataset, which is compiled from on-site data collected from multiple stations and divided into training and test sets in an 8:2 ratio. The dataset is carefully annotated by a professional team according to strict annotation standards, ensuring high data quality and annotation accuracy. During the data collection process, the actual environmental differences between different stations were fully considered to enhance the model's adaptability to diverse scenarios. For example, the collection tasks cover a variety of lighting conditions, including strong daylight, low light at night, and shadowed areas; a variety of equipment; and complex backgrounds, including raindrops, leaves, and debris obstructing the view.

[0051] S12. Preprocess the training data. To further improve the model's generalization capabilities for diverse scenarios, perform data augmentation on the training data. These augmentations include rotation, flipping, scaling, and brightness adjustment, enabling the model to adapt to complex real-world environments.

[0052] S13. Input the preprocessed training data set into the target detection model based on the first improved YOLOv8m network for training to obtain a trained target detection model.

[0053] To balance model accuracy and inference speed, the more mature single-stage model, YOLOv8m, was selected as the model's foundational framework. As a member of the YOLO family, YOLOv8m also consists of three networks: the backbone, the neck, and the head. Specifically, the backbone network effectively captures both local and global image information through a C2f feature extraction module designed with stacked multi-gradient flows. The neck network, employing the concept of a path aggregation network, extracts effective information from multiple layers of feature maps by contracting and expanding paths, thereby enhancing the detection capabilities of small objects and multi-scale targets. The head network is responsible for completing the final object detection task, including category prediction, bounding box regression, and object confidence estimation.

[0054] In complex scenes, since the background and the target share similar texture or color features, the feature extraction process of the model is easily disturbed, resulting in false detection and missed detection. In order to better meet the detection requirements of the task target, the target detection model proposed in this paper has been targetedly improved and optimized based on YOLOv8m. The specific architecture of the improved model is as follows: Figure 3 The improved model structure still consists of the backbone, neck, and head. Improvements include the introduction of a spatial pyramid pooling module and a partial self-attention module in the fourth stage of the backbone network, and a coordinate attention mechanism in the third stage of the neck network. This aims to further enhance the model's ability to accurately locate object boundaries, enabling it to better meet the system's actual detection accuracy requirements while maintaining efficient inference speed.

[0055] The backbone network is responsible for extracting multi-level, multi-scale universal features from the input image, forming a feature map with high-level semantic information to support subsequent detection tasks. First, the C2f feature extraction module, as the basic unit of the network, is responsible for extracting local features at each stage and enhancing the expression of multi-scale information through feature fusion. This module adopts the CSP structure design and divides the input feature map into two branches through 1×1 convolution: one branch is through multiple residual modules. The first branch performs processing, while the other branch directly performs convolution operations. The outputs of the two branches are then concatenated using the Concat operation. After batch normalization (BN) and SiLU activation functions, the features are finally organized through convolution operations to obtain the final output. By stacking multiple C2f modules, the network can learn rich feature expressions at different levels, while optimizing the fusion of high-level semantic information and low-level detail information at different stages. The formula for the C2f feature extraction module can be expressed as follows:

[0056]

[0057] in, is the input feature, is the output feature, represents 1×1 convolution, Represents the channel splitting operation, is the residual module, Represents a feature concatenation operation. It is the intermediate feature of the module processing process and is the output obtained after N times of residual module processing.

[0058] Subsequently, the Spatial Pyramid Pooling-Fast (SPPF) module aggregates features of different scales, allowing the network to better retain semantic information when processing complex scenes and generate richer feature representations. The SPPF module first processes the input feature map through a convolutional layer, batch normalization (BN), and SiLU activation function to obtain a feature map with half the number of channels. Next, the feature map passes through three 5×5 maximum pooling layers in sequence, and the size of the feature map is gradually reduced after each pooling. The output of each pooling layer serves as the input for the next pooling. This serial pooling process can effectively capture information at different scales and improve the model's ability to capture details and global information in the image. Finally, by splicing the feature maps after pooling at different scales in the channel dimension, the SPPF module can integrate feature information from different scales to form a multi-scale feature representation. The formula of the spatial pyramid pooling module can be expressed as follows:

[0059]

[0060] in, is the input feature, is the output feature of the module, represents the maximum pooling operation of 5×5, Represents the output obtained after k times of maximum pooling.

[0061] When separating target and background features, the model inevitably introduces unnecessary background information that interferes with the target features, leading to false detections and missed detections. To address this issue, a common approach is to introduce an attention mechanism deep within the model to highlight the target features. However, existing mainstream attention mechanisms are mostly implemented through convolution operators. The inherent limitations of convolution operators make it difficult to establish long-range dependencies between features, which are crucial for accurately locating targets. In contrast, the Transformer structure, thanks to its self-attention mechanism, can efficiently capture global long-range dependencies. However, the Transformer has high computational complexity and memory usage, which often results in significant time overhead in real-time inference tasks.

[0062] Therefore, a partial self-attention (PSA) module is introduced into the backbone network. The PSA module is designed to effectively enhance the network's ability to model long-range dependencies while avoiding the computational overhead associated with the global self-attention mechanism. Specifically, the input feature map is evenly split into two parts, keeping the complexity of the self-attention computation low. One part of the feature map is fed into the self-attention module for global information modeling, capturing long-range dependencies and enhancing contextual understanding of the target. Global information modeling is performed through matrix operations between the query vector (Query), key vector (Key), and value vector (Value) to capture long-range correlations between features. The other part of the feature map is fused with the output of the self-attention module via skip connections. Furthermore, to further improve inference efficiency, the PSA module optimizes the dimensions of the query and key vectors in the self-attention mechanism, setting them to half that of the value vector, thereby reducing computational overhead. Furthermore, the module uses batch normalization (BN) instead of layer normalization (LN) for normalization, improving model speed and stability. This module is placed after the fourth stage, which has the lowest resolution in the model, and focuses on extracting and sorting out key information from high-level abstract features. Due to the lower feature resolution of the fourth stage, the quadratic complexity of self-attention calculations is significantly reduced, so the overall inference speed can still meet real-time requirements. The formula for some self-attention modules can be expressed as follows:

[0063]

[0064] in, is the input feature, is the output feature, Represents the Transformer module. Through the above operations, without significantly increasing the computational overhead, we can effectively capture the long-range dependencies between features, thereby better processing the semantic information deep in the network.

[0065] The neck network is used to integrate the features extracted from the backbone network for fusion, enhancement and processing to help the model better detect objects of different scales. Specifically, the neck network receives the features of the P3, P4 and P5 levels in the backbone network as input, first enhances the low-level features through a bottom-up information transmission path, and then fuses them with high-level features to ensure that the network can obtain rich multi-level feature representations. Subsequently, the network diffuses the detail information to deeper levels of the network through a top-down information transmission path, enhances the interaction between high-level semantic features and low-level detail features, and further improves the model's sensitivity to details. The combination of bottom-up and top-down information flow ensures that the network maintains efficient detection performance in complex scenarios, especially showing significant advantages in multi-scale object detection tasks.

[0066] To further improve the model's ability to extract target location information, a coordinate attention mechanism (CA) is introduced before the third-stage output of the neck network and the head network to enhance the ability to accurately locate target boundaries. The coordinate attention mechanism extracts compressed features in different directions by performing global average pooling in the horizontal and vertical directions. Subsequently, these features are subjected to a set of convolutional layers with shared weights for information interaction and integration, thereby improving the expressive power of the features. The fused features are re-divided into horizontal and vertical parts, and the corresponding attention matrices are generated through 1×1 convolution and Sigmoid activation functions respectively. Finally, the original features are weighted using the attention matrix through element-wise multiplication to enhance the position information in the feature map, effectively improving the model's accuracy in locating target boundaries. The formula for the coordinate attention mechanism can be expressed as follows:

[0067]

[0068] in, is the input feature, is the output feature, and Represents the average pooling operation in the horizontal and vertical directions of the feature, is the Sigmoid activation function.

[0069] The head network is responsible for converting feature maps processed from the trunk and neck into final object detection results. Specifically, the network consists of three detection heads, each of which can be divided into two parts: two 3×3 convolutions and one 1×1 convolution, which are used to predict coordinate regression information and category confidence information, respectively.

[0070] Furthermore, in order to prevent the model from overfitting, regularization techniques such as weight decay and Dropout are introduced during the model training process to improve the robustness of the model.

[0071] The image to be detected is then fed into the trained object detection model based on the first improved YOLOv8m network to determine whether there is a chain in the image to be detected. If an object is detected in the current image frame to be detected, the ROI region is extracted based on the output of the detection model and used as the input for the segmentation model.

[0072] Then execute S2. In S2, if there are chains in the image to be detected, extract the chain detection frame area image, input the chain detection frame area image into the trained instance segmentation model based on the second improved YOLOv8m-Seg network to perform chain segmentation, and obtain a chain mask image.

[0073] According to an embodiment of the present invention, the process of training an instance segmentation model based on the second improved YOLOv8m-Seg network includes:

[0074] S21. Obtain training data sets. Use multiple field-collected data sets for training and testing.

[0075] S22. Preprocess the training data. To further improve the model's generalization ability for diverse scenarios, perform data augmentation on the training data.

[0076] S23. Input the preprocessed training data set into the instance segmentation model based on the second improved YOLOv8m-Seg network for training to obtain a trained instance segmentation model.

[0077] Since the local area of ​​slender targets is extremely narrow and occupies only a small number of pixels, it is easily obscured by complex background interference during feature transfer. In addition, the aspect ratio of slender targets is much larger than that of conventional targets. Due to the limitation of the detector's receptive field, it is difficult to generate complete target features, which affects the model's accurate segmentation of slender targets. In view of the special morphology of slender targets, the instance segmentation model proposed in this paper is improved and optimized based on the YOLOv8m-Seg network. The specific model structure is as follows: Figure 4 Specifically, a spatial pyramid pooling module and a partial self-attention module are added to the original YOLOv8m-Seg backbone network; a multi-scale sequence fusion module is added between the backbone network and the neck network; and a feature fusion module is added between the neck network and the head network. The multi-scale sequence fusion module aggregates feature maps of different scales in the backbone network to generate a more detailed global information representation, thereby enhancing the network's sensitivity to target edge details and morphological changes. In the third stage of the neck network, the feature fusion module effectively fuses global context information with position information and passes it down layer by layer, allowing the downstream segmentation head to use more accurate information to complete fine segmentation.

[0078] The backbone network plays a core role in the task. It extracts multi-level, multi-scale common features from the input image and generates feature maps with high-level semantic information, providing a solid foundation for subsequent tasks. First, the C2f feature extraction module, as the basic unit of the network, is responsible for extracting local features at each stage and enhancing the representation of multi-scale information through feature fusion.

[0079] Subsequently, the Spatial Pyramid Pooling-Fast (SPPF) module aggregates features from different scales to enhance the network's semantic understanding capabilities when processing complex scenes. The SPPF module first processes the input feature map through a convolutional layer, batch normalization (BN), and the SiLU activation function to produce a feature map with half the number of channels. The feature map then passes through three 5×5 max pooling layers, with the size of the feature map gradually decreasing after each pooling. The output of each pooling layer serves as the input for the next pooling. This serial pooling process effectively captures information at different scales and improves the model's ability to capture both detailed and global information in the image. Finally, by concatenating the feature maps pooled at different scales along the channel dimension, the SPPF module integrates feature information from different scales to form a multi-scale feature representation.

[0080] Subsequently, the backbone network introduces a partial self-attention (PSA) module. The input feature map is evenly split into two parts, keeping the complexity of the self-attention computation low. A portion of the features is fed into the self-attention module for global information modeling, capturing long-range dependencies and enhancing contextual understanding of the target. Global information modeling is performed through matrix operations between the query vector (Query), key vector (Key), and value vector (Value) to capture long-range dependencies between features. The remaining features are fused with the output of the self-attention module via skip connections.

[0081] To further improve segmentation accuracy, a multi-scale sequence fusion module (MSF) was designed. This module aggregates multi-scale features to generate global contextual information, effectively compensating for the limitations of the receptive field. Specifically, this module extracts feature maps at levels P2 to P5 from the backbone network. After resizing, it initially fuses low-level local details with high-level global semantic information through element-by-element concatenation to generate a comprehensive description of the target features. To avoid information loss during excessive upsampling or downsampling, feature maps at all levels are resized to the same size as the P3 feature map. Subsequently, channel dimensionality reduction is performed using 1×1 convolutions, and features are mapped to a higher-level feature space to simplify computational complexity while preserving key information. Furthermore, a RepVGG module with convolution kernels of 3 and 5 is used to extract edge details and contextual information of the target at different receptive fields, further enhancing the representation of global context and ensuring that the network accurately captures the complete features of the target even in complex backgrounds. The formula for the multi-scale sequence fusion module can be expressed as follows:

[0082]

[0083] in, represents the feature map of level i, is the output feature of the module, Represents the RepVGG module.

[0084] The neck network, composed of a bottom-up expansion path and a top-down contraction path, is used to integrate, enhance, and process features extracted from the backbone network to help the model better detect objects of different scales. The neck network receives features from the P3, P4, and P5 levels of the backbone network as input. It first enhances low-level features through a bottom-up expansion path and then fuses them with high-level features to ensure the network can obtain rich multi-level feature representations. Through a top-down contraction path, the neck network diffuses detailed information to deeper layers of the network, enhancing the interaction between high-level semantic features and low-level detail features, further improving the model's sensitivity to detail. The combination of bottom-up and top-down information flow ensures the neck network's detection performance in the segmentation of slender objects.

[0085] Subsequently, the feature fusion module (FFM) is used to fuse the feature maps from the multi-scale sequence fusion module and the P3 level of the neck network expansion path to enhance the network's accurate segmentation ability. Specifically, the module first splices the two feature maps in the channel dimension to initially achieve information complementarity. Then, a feature weight set is generated through 3×3 convolution and Sigmoid activation function to reflect the importance of each feature. According to the feature weight set, the features to be fused are adjusted separately to enhance the response of key features and suppress redundant information. Finally, the two parts of the features are fused by element-wise addition to ensure that the final output has stronger feature expression capabilities, thereby improving the network's segmentation effect on slender targets. The formula of the feature fusion module is:

[0086]

[0087] in, and is the feature to be fused, is the output feature, is the Sigmoid activation function.

[0088] The head network is responsible for converting the feature maps fused by the feature fusion module into the final instance segmentation result. The head network consists of a segmentation head and three prediction heads. The segmentation head's primary task is to generate a high-resolution native mask of the object. Composed of multiple convolutional layers, it gradually extracts and restores spatial information to produce accurate segmentation results. First, the input feature map is processed using 3×3 convolutions to extract spatial information and enhance feature detail. Subsequently, the feature map is upsampled using transposed convolutions to restore the spatial resolution of the image, preserving the fine-grained information of the object. Finally, a 1×1 convolutional layer performs channel dimensionality reduction to generate a high-resolution object mask. The prediction head is designed for feature maps at different levels and is primarily responsible for predicting object attributes, including the location of the object bounding box, category information, and segmentation-related mask coefficients. 3×3 convolutions are first used to enhance the perception of the object region, followed by 1×1 convolutions for object category classification, bounding box regression, and prediction of mask coefficients. Finally, the outputs of the segmentation and prediction heads are used to generate the object segmentation mask.

[0089] Then, S3 is executed. In S3, a parabola fitting algorithm is used to calculate the chain mask image to obtain the chain centerline, and the curvature of the chain is evaluated according to the ratio of the chain arc depth to the chain chord length.

[0090] According to an embodiment of the present invention, the segmentation results are approximated by parabola fitting, and the curvature of the brake chain is calculated based on the ratio of the brake chain arc depth to the chain chord length. The segmentation results are analyzed and the calculated curvature is compared with a preset threshold. Curvature changes are used to identify potential risk situations. All segmented images within the same regression frame are treated as the same target to avoid the effects of occlusion. To accurately obtain the target's geometric characteristics, a parabola fitting algorithm based on the least squares method is used to obtain the chain centerline: Subsequently, the curvature of the chain centerline is calculated using a mathematical derivation method to quantitatively reflect the degree of chain deformation. The specific formula is:

[0091]

[0092] in, is the curvature of the chain centerline, represents the chord length of the chain's centerline, Indicates the maximum depth of the chain's centerline relative to the chord length.

[0093] Then, S4 is executed. In S4, if the bending degree is greater than the preset bending threshold, a brake risk warning is issued.

[0094] The technical effects of the present invention are further verified through experiments.

[0095] A chain lock detection system was constructed, with data acquisition equipment pre-installed on the side of the train. When the magnetic steel detects a train signal, the system acquires continuous video frames as input. This video data is then processed by both the main program and the intelligent detection module. During this process, the main program receives information about the curvature of the brake chain from the image transmitted by the intelligent detection module. The main program further calculates the proportion of all curvature values ​​less than 0.035. If this proportion exceeds 50%, a brake lock risk is identified and a warning is issued.

[0096] Multiple datasets collected in the field were used for training and testing. The chain detection dataset contains 16,051 images, while the chain segmentation dataset contains 14,832 images. To ensure the effectiveness of the models in real-world applications, these datasets were rigorously labeled and screened, covering targets in different scenarios and lighting conditions. Both the detection and segmentation models were built using the PyTorch deep learning framework. The batch size was set to 16, and the SGD optimizer was used with an initial learning rate of 0.01 and a momentum of 0.937 for a total of 300 epochs. To ensure the stability and convergence of the training process, a learning rate scheduling strategy was adopted, reducing the learning rate by a factor of 10 every 100 epochs to achieve a balance between model exploration and convergence.

[0097] To achieve optimal detection accuracy, detailed adjustments and optimizations were made to data augmentation. The final data augmentation parameters for the detection model and segmentation model are shown in Tables 1 and 2.

[0098] Table 1 Data augmentation parameters of the detection model

[0099]

[0100] Table 2 Data augmentation parameters of the segmentation model

[0101]

[0102] To validate the advanced nature of the segmentation model, the proposed method was comprehensively compared with several mainstream methods in the field, including Mask R-CNN, YOLACT, YOLOv5m-Seg, and YOLOv8m-Seg. To ensure a fair and authoritative comparison, the network structures of these methods were reproduced using their publicly available code, and the models were trained and evaluated under the same training environment. The experiments strictly standardized hyperparameter settings, datasets, and evaluation metrics to ensure comparability of results. As shown in Table 3, the proposed segmentation model outperformed other instance segmentation methods across all evaluation metrics, fully demonstrating its superiority in instance segmentation tasks.

[0103] Table 3 Comparison of the segmentation model of the present invention with other instance segmentation methods

[0104]

[0105] In order to verify the effectiveness of the introduced modules, an ablation experiment was conducted. The experiment used YOLOv8m-Seg as the baseline model, and added the multi-scale sequence fusion module and the feature fusion module in turn to verify the contribution and effectiveness of each module. The experimental results are shown in Table 4. Among them, the multi-scale sequence fusion module significantly enriched the diversity of feature expression by modeling and fusing information of different scales, so that the model has higher recognition ability and robustness when dealing with complex scenes or diverse targets. The feature fusion module effectively fuses features from different sources while passing key information downward through the contraction path, further enhancing the transmission and representation capabilities of semantic information, thereby optimizing the performance of the model in the segmentation task. These results show that the design of each module has practical value, and their combination can significantly improve the overall model performance, verifying the effectiveness and rationality of the method of the present invention.

[0106] Table 4 Experimental results

[0107]

[0108] To increase inference speed, the Lianbao recognition model underwent a PyTorch to TensorRT conversion to fully leverage hardware acceleration. The inference GPU deployed on-site was the NVIDIA GeForce RTX 4070, which supports FP16 and FP32 computations, both delivering 29.15 TFLOPS of computing performance. Through TensorRT optimization, the model's inference speed was significantly improved, with the inference time for a single image (including pre- and post-processing) being approximately 25ms. To further improve processing efficiency, multi-threaded inference was adopted during deployment to enhance concurrent processing capabilities. During actual on-site operation, the optimized and deployed Lianbao recognition system was able to process image data captured by multiple cameras in real time at a speed of 50 frames per second, meeting the needs of real-time monitoring and detection.

[0109] After comprehensive testing, the chain brake identification system has demonstrated excellent performance, with a detection accuracy exceeding 95%. Regardless of complex environments or changing conditions, the system can efficiently and accurately identify and detect brake risks, providing a strong guarantee for the safe operation of trains.

[0110] This paper uses YOLOv8m as its core algorithm and makes customized improvements based on task requirements. For target detection, a spatial pyramid pooling module and a partial self-attention module are added to the original YOLOv8m backbone network. A coordinate attention mechanism is added after the neck network and before the head network to meet the high requirements for precise positioning in the task. By adaptively adjusting feature weights, the model can more accurately focus on the target boundary, thereby achieving efficient detection and positioning of the chain. For target segmentation, a spatial pyramid pooling module and a partial self-attention module are added to the original YOLOv8m-Seg backbone network for the special morphology of the human brake handbrake chain. A multi-scale sequence fusion module is added between the backbone network and the neck network. A feature fusion module is added between the neck network and the head network. By effectively fusing global context information with position information, the model's ability to analyze complex morphologies at different scales is enhanced, thereby achieving accurate target segmentation.

[0111] Another embodiment of the present invention provides a chain hugging identification system, such as Figure 5 As shown, the system includes:

[0112] A chain detection module 510 is configured to input the image to be detected into the trained object detection model based on the first improved YOLOv8m network for detection to determine whether there is a chain in the image to be detected;

[0113] A chain segmentation module 520 is configured to extract a chain detection frame region image if a chain exists in the image to be detected, and input the chain detection frame region image into a trained instance segmentation model based on the second improved YOLOv8m-Seg network to perform chain segmentation to obtain a chain mask image;

[0114] The chain lock identification module 530 is configured to use a parabola fitting algorithm to calculate the chain centerline for the chain mask image, and evaluate the degree of chain bending based on the ratio of the chain arc depth to the chain chord length; if the degree of bending is greater than a preset bending threshold, a brake risk warning is issued.

[0115] For the undetailed parts of a chain hugging identification system according to an embodiment of the present invention, please refer to the above detailed description of the method embodiment.

[0116] The method of the present invention can be executed in an electronic device. The electronic device can be any device with storage and computing capabilities, such as a server, workstation, or personal computer such as a desktop computer or laptop computer, or a terminal device such as a mobile phone, tablet computer, smart wearable device, or Internet of Things device, but is not limited thereto.

[0117] An electronic device may include: a processor, a memory, an input / output interface, a communication interface, and a bus. The processor, memory, input / output interface, and communication interface are interconnected within the electronic device via the bus. The processor may be implemented as a general-purpose CPU, a microprocessor, an application-specific integrated circuit, or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this specification. The memory may be implemented as a ROM, RAM, static storage device, dynamic storage device, etc. The memory may store an operating system and other application programs. When implementing the technical solutions provided in the embodiments of this specification via software or firmware, the relevant program code is stored in the memory and called and executed by the processor. The input / output interface is used to connect to an input / output module to enable information input and output. The input / output module can be configured as a component within the electronic device or can be externally connected to the electronic device to provide corresponding functions. Input devices may include a keyboard, a mouse, a touch screen, a microphone, various sensors, etc., and output devices may include a display, a speaker, a vibrator, an indicator light, etc. The communication interface is used to connect to a communication module to enable communication between the electronic device and other devices. The communication module may communicate via wired or wireless communication. A bus comprises a pathway that transmits information between components of an electronic device.

[0118] An embodiment of the present invention also provides a non-transitory readable storage medium, which stores instructions, and the instructions are used to enable the electronic device to execute the method according to the embodiment of the present invention. The readable medium of this embodiment includes permanent and non-permanent, removable and non-removable media, and information storage can be achieved by any method or technology. The information can be a computer-readable instruction, a data structure, a program module or other data. Examples of readable storage media include, but are not limited to: phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, tape and disk storage, etc.

[0119] It should be noted that the terms used in the present invention are only for describing specific embodiments and are not intended to limit the scope of this application. As shown in the present specification, unless the context clearly indicates an exception, the words "one", "a", "a kind of" and / or "the" do not specifically refer to the singular and may also include the plural. The terms "comprise", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that the process, method or device comprising a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method or device. In the absence of further restrictions, the elements defined by the sentence "comprise a..." do not exclude the presence of other identical elements in the process, method or device comprising the elements.

[0120] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the technical solutions of the embodiments of the present invention.

Claims

1. A chain hugging identification method, characterized in that: include: Inputting the image to be detected into the trained object detection model based on the first improved YOLOv8m network for detection to determine whether there is a chain in the image to be detected; wherein the improvements of the first improved YOLOv8m network include: adding a spatial pyramid pooling module and a partial self-attention module to the original YOLOv8m backbone network; and adding a coordinate attention mechanism after the neck network and before the head network; If there are chains in the image to be detected, extract the chain detection frame area image, input the chain detection frame area image into the trained instance segmentation model based on the second improved YOLOv8m-Seg network for chain segmentation, and obtain a chain mask image; wherein, the improvements of the second improved YOLOv8m-Seg network include: adding a spatial pyramid pooling module and a partial self-attention module to the original YOLOv8m-Seg backbone network; adding a multi-scale sequence fusion module between the backbone network and the neck network; adding a feature fusion module between the neck network and the head network; inputting the chain detection frame area image into the trained instance segmentation model based on the second improved YOLOv8m-Seg network for chain segmentation includes: The chain detection box area image is input into the backbone network for feature extraction, including: using multiple C2f feature extraction modules in the backbone network to extract local features of the image and perform feature fusion; then, the features of different scales are aggregated through the spatial pyramid pooling module, including: processing the features of different scales through convolutional layers, batch normalization and SiLU activation functions to obtain feature maps with half the number of channels; passing through multiple maximum pooling layers in sequence to gradually reduce the size of the feature maps; the feature maps after pooling at different scales are spliced ​​in the channel dimension; then, through the partial self-attention module, the aggregated feature maps are evenly divided into two parts, one part of the feature map enters the self-attention module and performs global information modeling through matrix operations between the query vector, key vector and value vector; the other part of the feature map is fused with the output of the self-attention module through jump connections; The features extracted by the backbone network are fused using a multi-scale sequence fusion module. This involves extracting feature maps at levels P2 to P5 from the backbone network, resizing them, and initially fusing low-level local detail information with high-level global semantic information through element-wise splicing to generate a comprehensive description of the target features. Subsequently, 1×1 convolution is used to reduce the channel dimension and map the features to a higher-level feature space. Finally, the RepVGG module with convolution kernels of 3 and 5 is used to extract edge details and contextual information of the target under different receptive fields. The neck network is used to fuse, enhance and process the features extracted by the backbone network; The fusion features extracted by the multi-scale sequence fusion module are fused with the feature map of the neck network P3 level using a feature fusion module, including: splicing the two feature maps in the channel dimension; generating a feature weight set through 3×3 convolution and Sigmoid activation function; adjusting the features to be fused according to the feature weight set; and fusing the adjusted two features through element-wise addition. The head network is used to convert the feature maps fused by the feature fusion module into the final instance segmentation result. The head network includes a segmentation head and a prediction head. The segmentation head is used to generate a high-resolution native mask of the target, and the prediction head is used to predict target attributes, such as the target box position, category information, and mask coefficients related to segmentation. For the chain mask image, a parabola fitting algorithm is used to extract the center line of the chain, and the curvature of the chain is evaluated based on the ratio of the chain arc depth to the chain chord length. The formula is: Where κ is the curvature of the chain centerline, l is the chord length of the chain centerline, and h is the maximum depth of the chain centerline relative to the chord length; If the bending degree is greater than the preset bending threshold, a brake risk warning is issued.

2. A chain embrace identification method according to claim 1, characterized in that: Inputting the image to be detected into the trained target detection model based on the first improved YOLOv8m network for detection to determine whether there is a chain in the image to be detected includes: The image to be detected is input into the backbone network for feature extraction, including: using multiple C2f feature extraction modules in the backbone network to extract local features of the image and perform feature fusion; then, the features of different scales are aggregated through the spatial pyramid pooling module, including: processing the features of different scales through convolutional layers, batch normalization and SiLU activation functions to obtain feature maps with half the number of channels; passing through multiple maximum pooling layers in sequence to gradually reduce the size of the feature maps; the feature maps after pooling at different scales are spliced ​​in the channel dimension; then, through the partial self-attention module, the feature maps aggregated by the spatial pyramid pooling module are evenly divided into two parts, one part of the feature map enters the self-attention module to perform global information modeling through matrix operations between the query vector, key vector and value vector; the other part of the feature map is fused with the output of the self-attention module through jump connections; The neck network is used to fuse, enhance and process the features extracted by the backbone network; The coordinate attention mechanism is used to generate feature maps with enhanced position information. The method includes the following steps: first, global average pooling is performed on the features output by the neck network in the horizontal and vertical directions to extract compressed features in different directions; then, these features are fused through a set of convolutions with shared weights; the fused features are re-divided into horizontal and vertical parts, and the attention matrix is ​​generated by 1×1 convolution and sigmoid activation function respectively; finally, the features output by the neck network are weighted by the attention matrix through element-wise multiplication to obtain a feature map with enhanced position information; The head network is used to transform the feature map enhanced with position information into the final object detection result.

3. A chain embrace identification method according to claim 2, characterized in that: The dimensions of the query vector and key vector in the partial self-attention module are half of the value vector; batch normalization is used instead of layer normalization for normalization.

4. A chain hugging identification system, characterized in that: include: A chain detection module is configured to input an image to be detected into a trained object detection model based on a first improved YOLOv8m network to detect whether a chain exists in the image to be detected; wherein the improvements of the first improved YOLOv8m network include: adding a spatial pyramid pooling module and a partial self-attention module to the original YOLOv8m backbone network; and adding a coordinate attention mechanism after the neck network and before the head network; A chain segmentation module is configured to extract a chain detection frame area image if a chain exists in the image to be detected, input the chain detection frame area image into a trained instance segmentation model based on the second improved YOLOv8m-Seg network for chain segmentation, and obtain a chain mask image; wherein the improvements of the second improved YOLOv8m-Seg network include: adding a spatial pyramid pooling module and a partial self-attention module to the original YOLOv8m-Seg backbone network; adding a multi-scale sequence fusion module between the backbone network and the neck network; adding a feature fusion module between the neck network and the head network; inputting the chain detection frame area image into the trained instance segmentation model based on the second improved YOLOv8m-Seg network for chain segmentation includes: The chain detection box area image is input into the backbone network for feature extraction, including: using multiple C2f feature extraction modules in the backbone network to extract local features of the image and perform feature fusion; then, the features of different scales are aggregated through the spatial pyramid pooling module, including: processing the features of different scales through convolutional layers, batch normalization and SiLU activation functions to obtain feature maps with half the number of channels; passing through multiple maximum pooling layers in sequence to gradually reduce the size of the feature maps; the feature maps after pooling at different scales are spliced ​​in the channel dimension; then, through the partial self-attention module, the aggregated feature maps are evenly divided into two parts, one part of the feature map enters the self-attention module and performs global information modeling through matrix operations between the query vector, key vector and value vector; the other part of the feature map is fused with the output of the self-attention module through jump connections; The features extracted by the backbone network are fused using a multi-scale sequence fusion module. This involves extracting feature maps at levels P2 to P5 from the backbone network, resizing them, and initially fusing low-level local detail information with high-level global semantic information through element-wise splicing to generate a comprehensive description of the target features. Subsequently, 1×1 convolution is used to reduce the channel dimension and map the features to a higher-level feature space. Finally, the RepVGG module with convolution kernels of 3 and 5 is used to extract edge details and contextual information of the target under different receptive fields. The neck network is used to fuse, enhance and process the features extracted by the backbone network; The fusion features extracted by the multi-scale sequence fusion module are fused with the feature map of the neck network P3 level using a feature fusion module, including: splicing the two feature maps in the channel dimension; generating a feature weight set through 3×3 convolution and Sigmoid activation function; adjusting the features to be fused according to the feature weight set; and fusing the adjusted two features through element-wise addition. The head network is used to convert the feature maps fused by the feature fusion module into the final instance segmentation result. The head network includes a segmentation head and a prediction head. The segmentation head is used to generate a high-resolution native mask of the target, and the prediction head is used to predict target attributes, such as the target box position, category information, and mask coefficients related to segmentation. The chain embrace recognition module is configured to calculate the chain centerline using a parabola fitting algorithm for the chain mask image, and evaluate the degree of chain curvature based on the ratio of the chain arc depth to the chain chord length. The formula is: Where κ is the curvature of the chain centerline, l is the chord length of the chain centerline, and h is the maximum depth of the chain centerline relative to the chord length; If the bending degree is greater than the preset bending threshold, a brake risk warning is issued.

5. An electronic device, characterized in that: include: A memory, a processor, and a computer program; wherein the computer program is stored in the memory and is configured to be executed by the processor to implement the chain holding identification method according to any one of claims 1 to 3.

6. A computer-readable storage medium, characterized in that The storage medium stores a computer program; the computer program is executed by a processor to implement the chain embrace identification method described in any one of claims 1 to 3.

Images