A railway turnout structure defect monitoring method and system based on an intelligent camera and computer vision, and a computer readable storage medium

Abstract

The embodiment of the application discloses a railway turnout structure defect monitoring method and system based on an intelligent camera and computer vision, and a computer readable storage medium, comprising the following steps: acquiring a railway turnout defect training data set, and performing a pretreatment operation on the training data set; a railway turnout structure defect monitoring model based on YOLOv8 is constructed based on a deep learning framework; the model comprises a Backbone layer, a Neck layer and a Head layer; a C2F module in the Backbone layer is replaced with a C2FM module, and a mixed attention module is embedded in the Neck layer; the training data set is input into the railway turnout structure defect monitoring model for training, and the model parameters are updated; railway turnout pictures collected by the intelligent camera are input into the railway turnout structure defect monitoring model, and the result of railway turnout structure defect monitoring is obtained. Compared with the prior art, the accuracy and efficiency of railway turnout structure defect monitoring can be improved.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a method for monitoring structural defects in railway turnouts based on intelligent cameras and computer vision. Background Technology

[0002] Railway turnouts are critical steering devices in railway track systems, and their performance directly determines the safety and stability of train operation. During long-term service, turnouts are susceptible to various defects due to factors such as train impact, environmental corrosion (e.g., rainwater, sandstorms), and mechanical wear. These defects mainly include cracks in the switch rails, wear on the frogs, loose bolts, missing fasteners, and corrosion of the slide plates. Failure to monitor and repair these defects in a timely manner can lead to major safety accidents such as train derailment and overturning. Therefore, efficient and accurate monitoring of turnout defects is a core requirement for railway operation and maintenance.

[0003] Currently, turnout defect monitoring mainly relies on the following two types of methods: Manual monitoring versus traditional equipment monitoring: Manual inspection requires staff to walk along the track to observe or to conduct localized monitoring using handheld devices (such as ultrasonic flaw detectors), which has problems such as low efficiency, high missed detection rate, high labor intensity, and significant influence from light / weather. Traditional automated equipment (such as lidar carried by track monitoring vehicles) can achieve rapid scanning, but it is expensive and lacks accuracy in identifying minute defects on the surface of railway turnouts, such as cracks less than 0.2mm wide.

[0004] Deep learning-based monitoring methods: In recent years, convolutional neural networks (CNNs) such as the YOLO series and Faster R-CNN have been applied to turnout defect monitoring, improving monitoring efficiency to some extent. However, existing deep learning methods still have some problems: 1) Poor adaptability to multi-scale defects: The scale of turnout defects varies greatly. For example, the width of cracks on the switch rail surface is only 0.1 to 0.5 mm, while the area of ​​wear zone on the frog can reach 0.1 to 0.5 m². The C2F module of the traditional YOLOv8 model relies on ordinary convolution, which has a limited receptive field and cannot simultaneously and accurately capture the features of fine-grained small defects and coarse-grained large defects, resulting in missed detection of small defects and inaccurate location of large defects. 2) The contradiction between lightweight model and real-time performance: Switch monitoring needs to be deployed on mobile devices (such as track inspection robots and embedded monitoring terminals), but existing deep learning models have a large number of parameters and slow inference speed. For example, the inference time for a single frame image exceeds 50ms, which cannot meet the real-time requirement of "inspection speed of 30 kilometers per hour". 3) Insufficient ability of the model to cope with harsh environments or complex scenarios: In real-world scenarios, there are often harsh weather conditions, such as rain, snow, or low light conditions, which reduce the accuracy of model monitoring and result in insufficient robustness of the model.

[0005] Therefore, the existing technical problems are the low accuracy of monitoring defects in railway turnout structures and how to improve the robustness of the monitoring model. Summary of the Invention

[0006] In view of this, embodiments of the present invention provide a method for monitoring structural defects of railway turnouts based on intelligent cameras and computer vision. This method can solve the technical problem of low accuracy in monitoring structural defects of railway turnouts in the prior art.

[0007] The first aspect of this invention provides a method for monitoring structural defects in railway turnouts based on intelligent cameras and computer vision, comprising the following steps: S1: Obtain the training dataset of railway turnout defects and perform preprocessing operations on the training dataset; S2: Construct a railway turnout structural defect monitoring model based on YOLOv8 using a deep learning framework; the model includes a Backbone layer, a Neck layer, and a Head layer; replace the C2F module in the Backbone layer with a C2FM module, and embed a Hybrid Attention Module (HAM) in the Neck layer. S3: Input the training dataset into the railway turnout structural defect monitoring model for training, and update its model parameters; S4: Obtain the railway turnout images captured by the intelligent camera, input them into the railway turnout structural defect monitoring model, and obtain the results of railway turnout structural defect monitoring.

[0008] As a further limitation of the present invention, the C2FM module is an improvement on the C2F module, and consists of a CBS module, a Split module, n Focal modules, a Concat module, and a CBS module connected in sequence, wherein the outputs of the Split module and the n Focal modules are also input to the Concat module.

[0009] As a further limitation of the present invention, the Focal module is composed of a depthwise separable convolution module (DWConv), a SUM1 module, a dilated depthwise separable convolution module (DDWConv), a SUM2 module, a Conv2d module, and a SUM3 module connected in sequence. The SUM1-SUM3 modules are summation modules. The input features are respectively input to the depthwise separable convolution module, the SUM1 module, and the SUM3 module. The output of the SUM1 module is also input to the SUM2 module.

[0010] As a further limitation of the present invention, the size is The input features (where W is width, H is height, and C is the number of channels) are average pooled along the channel dimension to obtain a value of [value]. Features Average pooling along the width dimension yields a size of Features Average pooling along the height dimension yields a value of [value]. Features ; Features Input to Features obtained from Conv2d , will feature After transposition and features The input is concatenated using Concat to obtain a size of [size missing]. The features are then input into the concatenated feature set. Conv2d, SiLu and Conv2d, resulting in a size of The characteristics are divided into sizes using the Split module. Features and Features , will feature , and The inputs are fed into SiLu respectively, and their outputs are multiplied to obtain the final feature map.

[0011] As a further limitation of the present invention, the preprocessing operation includes image enhancement.

[0012] As a further limitation of the present invention, the loss function used in the model is: , , ,in For loss function, This is a weighted CIOU loss function, where CIOU represents the CIOU loss function. Let be the confidence loss function of the model. Let be the model's confidence level, and r be a hyperparameter controlling the penalty intensity in low-confidence regions. It is a weighting factor. .

[0013] The second aspect of the present invention provides a railway turnout structural defect monitoring system based on intelligent camera and computer vision, including a training dataset acquisition module, a model building module, a model training module and a result output module; The training dataset acquisition module is used to acquire a training dataset of railway turnout defects and to perform preprocessing operations on the training dataset. The model building module is used to build a railway turnout structural defect monitoring model based on YOLOv8 based on a deep learning framework; the model includes a Backbone layer, a Neck layer and a Head layer; the C2F module in the Backbone layer is replaced with a C2FM module, and a Hybrid Attention Module (HAM) is embedded in the Neck layer. The model training module is used to input the training dataset into the railway turnout structural defect monitoring model for training and to update its model parameters; The result output module is used to acquire railway turnout images captured by the smart camera, input them into the railway turnout structural defect monitoring model, and obtain the results of railway turnout structural defect monitoring.

[0014] A third aspect of the present invention provides a computer-readable storage medium storing a computer program that can be executed by a processor to implement the method described above.

[0015] Compared with the prior art, the present invention has at least the following beneficial effects: This invention replaces the traditional C2F module in the Backbone layer of the YOLOv8 model with a C2FM module. The C2FM module uses dilated depthwise separable convolution, which expands the receptive field while preserving coarse-grained global features, while ordinary convolution and skip connections preserve fine-grained local information. This achieves the fusion of coarse-grained and fine-grained features in the YOLOv8 model, further improving the accuracy of defect features and ensuring the precision of the YOLOv8 model in monitoring railway turnout structural defects. In addition, this invention also embeds a HAM attention module in the Neck layer. Through multi-dimensional attention fusion and lightweight design, it achieves the goals of monitoring accuracy, speed balance, and generalization adaptability, strengthens the representation of defect features, ensures real-time monitoring speed, balances accuracy and efficiency, adapts to multi-scale defects, and enhances feature discrimination. By fusing three-dimensional attention of channel, width, and height, it can adapt to defect types of different scales and spatial distributions, effectively solving the scale difference problem and ensuring the accuracy of the model in monitoring railway turnout structural defects. Attached Figure Description

[0016] The above and other objects, features and advantages of the present invention will become clearer from the following description of embodiments of the invention with reference to the accompanying drawings, in which: Figure 1 This is a flowchart of a railway turnout structural defect monitoring method based on intelligent cameras and computer vision, provided by an embodiment of the present invention.

[0017] Figure 2The structure of the railway turnout structural defect monitoring model based on YOLOv8 provided in the embodiments of the present invention.

[0018] Figure 3 The structure of the C2FM module provided in the embodiments of the present invention.

[0019] Figure 4 The structure of the Hybrid Attention Module (HAM) provided in this embodiment of the invention. Detailed Implementation

[0020] The present application is described below based on embodiments, but it is not limited to these embodiments. In the detailed description of the present application below, certain specific details are described in detail. Those skilled in the art can fully understand the present application without these details. To avoid obscuring the substance of the present application, well-known methods, processes, flows, elements, and circuits are not described in detail.

[0021] Furthermore, those skilled in the art should understand that the accompanying drawings provided herein are for illustrative purposes only and are not necessarily drawn to scale.

[0022] Unless the context explicitly requires it, words such as "including" or "contains" throughout the application should be interpreted as including rather than exclusive or exhaustive; that is, meaning "including but not limited to".

[0023] In the description of this application, it should be understood that the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance. Furthermore, in the description of this application, unless otherwise stated, "a plurality of" means two or more.

[0024] See Figure 1-4 As shown, the first aspect of the present invention provides a method for monitoring structural defects of railway turnouts based on intelligent cameras and computer vision, comprising the following steps: S1: Obtain the training dataset of railway turnout defects and perform preprocessing operations on the training dataset; S2: Construct a railway turnout structural defect monitoring model based on YOLOv8 using a deep learning framework; the model includes a Backbone layer, a Neck layer, and a Head layer; replace the C2F module in the Backbone layer with a C2FM module, and embed a Hybrid Attention Module (HAM) in the Neck layer. S3: Input the training dataset into the railway turnout structural defect monitoring model for training, and update its model parameters; S4: Obtain the railway turnout images captured by the intelligent camera, input them into the railway turnout structural defect monitoring model, and obtain the results of railway turnout structural defect monitoring.

[0025] As a further limitation of the present invention, the C2FM module is an improvement on the C2F module, and consists of a CBS module, a Split module, n Focal modules, a Concat module, and a CBS module connected in sequence, wherein the outputs of the Split module and the n Focal modules are also input to the Concat module.

[0026] As a further limitation of the present invention, the Focal module is composed of a depthwise separable convolution module (DWConv), a SUM1 module, a dilated depthwise separable convolution module (DDWConv), a SUM2 module, a Conv2d module, and a SUM3 module connected in sequence. The SUM1-SUM3 modules are summation modules. The input features are respectively input to the depthwise separable convolution module, the SUM1 module, and the SUM3 module. The output of the SUM1 module is also input to the SUM2 module.

[0027] As a further limitation of the present invention, the size is The input features (where W is width, H is height, and C is the number of channels) are average pooled along the channel dimension to obtain a value of [value]. Features Average pooling along the width dimension yields a size of Features Average pooling along the height dimension yields a value of [value]. Features ; Features Input to Features obtained from Conv2d , will feature After transposition and features The input is concatenated using Concat to obtain a size of [size missing]. The features are then input into the concatenated feature set. Conv2d, SiLu and Conv2d, resulting in a size of The characteristics are divided into sizes using the Split module. Features and Features , will feature , and The inputs are fed into SiLu respectively, and their outputs are multiplied to obtain the final feature map.

[0028] As a further limitation of the present invention, the preprocessing operation includes image enhancement.

[0029] The modules of the YOLOv8 model in this invention will be described below with reference to the accompanying drawings: Conv: Convolutional layer, is the basic structure in convolutional neural networks, such as two-dimensional convolution conv2d.

[0030] CBS Module: CBS is an abbreviation for Conv-BN-SiLU, representing a standard convolutional block that is very common and efficient in modern convolutional neural networks (especially YOLOv5, v7, v8, v11, etc.). It consists of three consecutive layers: Conv (convolutional layer), BN (batch normalization layer), and SiLU (activation function layer). This module is the most basic component for building the backbone and neck of the YOLO model.

[0031] SPPF module: SPPF stands for Spatial Pyramid Pooling - Fast, a fast version of spatial pyramid pooling. Its core purpose is to enable convolutional neural networks to efficiently process input images of different sizes and aggregate multi-scale contextual information without significantly increasing computational costs, thereby significantly improving the model's ability to detect targets of different sizes.

[0032] The C2F module's main components include convolutional layers, followed by batch normalization, SiLU activation functions, and multiple Bottleneck blocks. This module contains two different types of Bottleneck blocks: when the "add" parameter is set to "true," the Bottleneck block includes residual shortcut addition, similar to ResNet blocks; if the "add" parameter is set to "false," the Bottleneck block is simply a stack of two convolutional layers. Structurally, the C2F module combines the advantages of Efficient Layer Aggregation Networks (ELANs) and Cross-Stage Partial Connectivity Networks (CSPNets).

[0033] However, the following problems exist in railway turnout defect monitoring: insufficient multi-dimensional feature capture; defect features are spatially dispersed and have large scale differences; traditional attention modules (such as CBAM focusing on "channel + global space" and Coordinate attention focusing on "coordinate space") cannot simultaneously cover the three-dimensional information of "channel importance, width direction features, and height direction features", resulting in inaccurate feature capture of irregular, multi-scale defects; in addition, the low feature discrimination caused by background noise interference; traditional attention modules lack the ability to fuse multi-dimensional features and have difficulty distinguishing between "defect features and noise features", resulting in low feature discrimination and high risk of false positives.

[0034] This invention replaces the traditional C2F module in the Backbone layer of the YOLOv8 model with a C2FM module. The C2FM module uses dilated depthwise separable convolution, which expands the receptive field while preserving coarse-grained global features, while ordinary convolution and skip connections preserve fine-grained local information. This achieves the fusion of coarse-grained and fine-grained features in the YOLOv8 model, further improving the accuracy of defect features and ensuring the precision of the YOLOv8 model in monitoring railway turnout structural defects. In addition, this invention also embeds a HAM attention module in the Neck layer. Through multi-dimensional attention fusion and lightweight design, it achieves the goals of monitoring accuracy, speed balance, and generalization adaptability, strengthens the representation of defect features, ensures real-time monitoring speed, balances accuracy and efficiency, adapts to multi-scale defects, and enhances feature discrimination. By fusing three-dimensional attention of channel, width, and height, it can adapt to defect types of different scales and spatial distributions, effectively solving the scale difference problem and ensuring the accuracy of the model in monitoring railway turnout structural defects.

[0035] As a further limitation of the present invention, the loss function used in the model is: , , ,in For loss function, This is a weighted CIOU loss function, where CIOU represents the CIOU loss function. Let be the confidence loss function of the model. Let be the model's confidence level, and r be a hyperparameter controlling the penalty intensity in low-confidence regions. It is a weighting factor. .

[0036] In the training of YOLO models, the CIOU loss function is a commonly used loss function. It considers factors such as overlapping region, center distance, and aspect ratio. The CIOU loss value measures the overlap between the target and the ground truth bounding box through IoU, measured by distance and the angle corresponding to the aspect ratio. The formula for calculating the CIOU loss function is as follows: in This represents the distance between the center points of the predicted bounding box and the ground truth bounding box. This represents the diagonal distance between the smallest closed region containing both the predicted and ground truth bounding boxes. Indicates the weighting coefficient. The similarity coefficient represents the aspect ratio of the predicted bounding box and the ground truth bounding box. It is the width of the actual frame. It is the actual height of the bounding box. It is the width of the prediction box. It is the height of the prediction box.

[0037] However, in real-world scenarios, severe weather conditions such as rain, snow, or low light often occur, leading to reduced model accuracy. To improve the model's robustness under low light conditions, weights are introduced, defined as follows: (Where confidence is the model's confidence level). This weight assigns greater importance to scenarios with low confidence, thereby effectively enhancing the model's ability to handle complex scenarios.

[0038] Furthermore, the model's confidence score reflects the model's certainty regarding the judgment that "the predicted bounding box contains defective features." To enhance the model's sensitivity to uncertain prediction results, this embodiment of the invention introduces a loss function based on the model's confidence score, and incorporates an adaptive weighting factor into this loss function. This factor increases in the low-confidence region, and its calculation formula is as follows: Here, r is a hyperparameter that controls the intensity of the penalty for low-confidence regions. It is usually set to 2. This setting can effectively highlight the importance of low-confidence regions, thereby enhancing the robustness of the model under complex conditions and helping the model achieve better monitoring performance in harsh environments.

[0039] The advantages of the loss function designed in this invention are reflected in the following aspects: 1. Focus on areas of uncertainty: The new loss function, by weighting low-light conditions and adaptive confidence levels, makes the model pay more attention to low-confidence and uncertain prediction regions. For example, prediction box B has a low confidence level, so its loss is greater, and the model will work harder to optimize this prediction box.

[0040] 2. Enhance robustness: In complex environments, such as low light and occlusion conditions, the new loss function can better handle uncertain prediction regions, thereby improving the model's robustness. For example, the loss is greater for predicted box B in complex environments, and the model will pay more attention to this region, thus improving the accuracy of monitoring.

[0041] 3. Balance the loss of different prediction boxes: The new loss function balances the losses of different predicted boxes, preventing the model from overemphasizing high-confidence predicted boxes while ignoring low-confidence ones during optimization. For example, if predicted box A has a smaller loss but predicted box B has a larger loss, the model will work harder to optimize predicted box B.

[0042] The second aspect of the present invention provides a railway turnout structural defect monitoring system based on intelligent camera and computer vision, including a training dataset acquisition module, a model building module, a model training module and a result output module; The training dataset acquisition module is used to acquire a training dataset of railway turnout defects and to perform preprocessing operations on the training dataset. The model building module is used to build a railway turnout structural defect monitoring model based on YOLOv8 based on a deep learning framework; the model includes a Backbone layer, a Neck layer and a Head layer; the C2F module in the Backbone layer is replaced with a C2FM module, and a Hybrid Attention Module (HAM) is embedded in the Neck layer. The model training module is used to input the training dataset into the railway turnout structural defect monitoring model for training and to update its model parameters; The result output module is used to acquire railway turnout images captured by the smart camera, input them into the railway turnout structural defect monitoring model, and obtain the results of railway turnout structural defect monitoring.

[0043] A third aspect of the present invention provides a computer-readable storage medium storing a computer program that can be executed by a processor to implement the method described in the above embodiments.

[0044] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A method for monitoring structural defects in railway turnouts based on intelligent cameras and computer vision, characterized in that: Includes the following steps: S1: Obtain the training dataset of railway turnout defects and perform preprocessing operations on the training dataset; S2: Construct a railway turnout structural defect monitoring model based on YOLOv8 using a deep learning framework; the model includes a Backbone layer, a Neck layer, and a Head layer; replace the C2F module in the Backbone layer with a C2FM module, and embed a Hybrid Attention Module (HAM) in the Neck layer. S3: Input the training dataset into the railway turnout structural defect monitoring model for training, and update its model parameters; S4: Obtain the railway turnout images captured by the intelligent camera, input them into the railway turnout structural defect monitoring model, and obtain the results of railway turnout structural defect monitoring; The C2FM module is an improvement on the C2F module, consisting of a CBS module, a Split module, n Focal modules, a Concat module, and a CBS module connected in sequence. The outputs of the Split module and the n Focal modules are also input to the Concat module. The Focal module is composed of a depthwise separable convolutional module (DWConv), a SUM1 module, a dilated depthwise separable convolutional module (DDWConv), a SUM2 module, a Conv2d module, and a SUM3 module connected in sequence. The SUM1-SUM3 modules are summation modules. The input features are respectively input to the depthwise separable convolutional module, the SUM1 module, and the SUM3 module. The output of the SUM1 module is also input to the SUM2 module. The size is The input features are average pooled along the channel dimension to obtain a value of . Features Where W is the width, H is the height, and C is the number of channels, average pooling is performed along the width dimension to obtain a size of Features Average pooling along the height dimension yields a value of [value]. Features ; Features Input to Features obtained from Conv2d , will feature After transposition and features The input is concatenated using Concat to obtain a size of [size missing]. The features are then input into the concatenated feature set. Conv2d, SiLu and Conv2d, resulting in a size of The characteristics are divided into sizes using the Split module. Features and Features , will feature , and The inputs are fed into SiLu respectively, and their outputs are multiplied to obtain the final feature map.

2. The method according to claim 1, characterized in that, The preprocessing operations include image enhancement.

3. The method according to claim 1, characterized in that, The loss function used in the model is: , , ,in For loss function, This is a weighted CIOU loss function, where CIOU represents the CIOU loss function. Let be the confidence loss function of the model. Let be the model's confidence level, and r be a hyperparameter controlling the penalty intensity in low-confidence regions. It is a weighting factor. .

4. A railway turnout structural defect monitoring system based on intelligent cameras and computer vision, characterized in that: It includes a training dataset acquisition module, a model building module, a model training module, and a result output module; The training dataset acquisition module is used to acquire a training dataset of railway turnout defects and to perform preprocessing operations on the training dataset. The model building module is used to build a railway turnout structural defect monitoring model based on YOLOv8 based on a deep learning framework; the model includes a Backbone layer, a Neck layer and a Head layer; the C2F module in the Backbone layer is replaced with a C2FM module, and a Hybrid Attention Module (HAM) is embedded in the Neck layer. The model training module is used to input the training dataset into the railway turnout structural defect monitoring model for training and to update its model parameters; The result output module is used to acquire railway turnout images captured by the smart camera, input them into the railway turnout structural defect monitoring model, and obtain the results of railway turnout structural defect monitoring. The C2FM module is an improvement on the C2F module, consisting of a CBS module, a Split module, n Focal modules, a Concat module, and a CBS module connected in sequence. The outputs of the Split module and the n Focal modules are also input to the Concat module. The Focal module is composed of a depthwise separable convolutional module (DWConv), a SUM1 module, a dilated depthwise separable convolutional module (DDWConv), a SUM2 module, a Conv2d module, and a SUM3 module connected in sequence. The SUM1-SUM3 modules are summation modules. The input features are respectively input to the depthwise separable convolutional module, the SUM1 module, and the SUM3 module. The output of the SUM1 module is also input to the SUM2 module. The size is The input features are average pooled along the channel dimension to obtain a value of . Features Where W is the width, H is the height, and C is the number of channels, average pooling is performed along the width dimension to obtain a size of Features Average pooling along the height dimension yields a value of [value]. Features ; Features Input to Features obtained from Conv2d , will feature After transposition and features The input is concatenated using Concat to obtain a size of [size missing]. The features are then input into the concatenated feature set. Conv2d, SiLu and Conv2d, resulting in a size of The characteristics are divided into sizes using the Split module. Features and Features , will feature , and The inputs are fed into SiLu respectively, and their outputs are multiplied to obtain the final feature map.

5. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that can be executed by a processor to implement the method as described in any one of claims 1-3.

Images