Trackside equipment apparent defect detection method, device, equipment, medium and product

Abstract

The invention relates to a trackside equipment apparent defect detection method and device, computer equipment, a computer readable storage medium and a computer program product. The method comprises the following steps: acquiring a to-be-detected image containing trackside equipment; performing image enhancement processing on the to-be-detected image to obtain a first feature image after performing visual content optimization on the trackside equipment; inputting the first feature image into a pre-trained target extraction model, positioning and extracting trackside equipment in the to-be-detected image through a target detection model, and obtaining a second feature image without non-trackside equipment elements; inputting the second feature image into a pre-trained defect detection model to obtain a defect detection result for the trackside equipment; the defect detection model is obtained by performing sample expansion on the basis of a real apparent defect image and then performing training as a training sample. By adopting the method, the defect detection precision can be improved.

Description

Technical Field

[0001] This application relates to the field of railway equipment inspection technology, and in particular to a method, device, computer equipment, computer-readable storage medium, and computer program product for detecting apparent defects in trackside equipment. Background Technology

[0002] With the rapid development of railway transportation, the scale of railway construction continues to expand. Railway trackside electrical equipment, installed on both sides of the track, is used for transmitting dynamic train information or for the operation and maintenance of the railway transportation system, and is a key component of railway lines. However, due to long-term exposure to the outdoor environment, abnormal defects are prone to appear on the surface of trackside electrical equipment, seriously threatening the safety of the railway transportation system. Therefore, it is crucial to detect surface defects in trackside electrical equipment.

[0003] However, existing methods struggle to detect rare or subtle defects in the complex imaging environments of trackside electrical equipment. Furthermore, the dispersed and diverse distribution of trackside electrical equipment in the railway transportation environment leads to uncertainty in the number and type of equipment in each image acquired by the inspection vehicle's data acquisition system, further resulting in low detection accuracy for surface defects in railway trackside electrical equipment. Summary of the Invention

[0004] Therefore, it is necessary to provide a method, apparatus, computer equipment, computer-readable storage medium, and computer program product for detecting apparent defects of railway trackside electrical equipment with high precision, in response to the above-mentioned technical problems.

[0005] In a first aspect, this application provides a method for detecting apparent defects in trackside equipment, including:

[0006] Acquire the image to be detected, which includes trackside equipment;

[0007] The image to be detected is subjected to image enhancement processing to obtain a first feature image after visual content optimization for the trackside equipment;

[0008] The first feature image is input into a pre-trained target extraction model, and the target detection model is used to locate and extract the trackside equipment in the image to be detected, resulting in a second feature image with non-trackside equipment elements removed.

[0009] The second feature image is input into a pre-trained defect detection model to obtain the defect detection result for the trackside equipment; the defect detection model is trained based on real appearance defect images after sample expansion.

[0010] In one embodiment, the step of obtaining the target extraction model includes:

[0011] A target extraction network to be trained is constructed based on YOLOv8. During the training process of the target extraction network to be trained using training images, the regression loss between the predicted bounding box and the real bounding box of the trackside equipment area in the training image is obtained through the CIoU loss calculation model. The category loss and confidence loss of the trackside electrical equipment are calculated through Focal loss.

[0012] The target extraction model is obtained by training the target extraction network based on the regression loss, the discrimination loss, and the confidence loss.

[0013] In one embodiment, the step of obtaining the defect detection model includes:

[0014] Acquire a pre-prepared first training image containing apparent defects and a second training image not containing apparent defects; the trackside devices contained in the first training image and the second training image are of the same type;

[0015] The appearance defects in the first training image are used to process the second training image to obtain a second training image containing the appearance defects.

[0016] The first training image containing the appearance defect and the second training image containing the appearance defect are used as training samples.

[0017] The defect detection model is trained using the training samples to obtain the defect detection model.

[0018] In one embodiment, processing the second training image using appearance defects in the first training image to obtain a second training image containing the appearance defects includes:

[0019] The defective pixel regions in the first training image are extracted using the SAM model.

[0020] The defective pixel region is fused into the second training image, and the defective region edge is optimized based on Poisson fusion to obtain a second training image containing the apparent defect.

[0021] In one embodiment, the step of obtaining the defect detection model to be trained includes:

[0022] Obtain the YOLOv8 model;

[0023] The original backbone network in the YOLOv8 model was replaced with the lightweight attention network MobileNet-SE to obtain the first detection model;

[0024] In the first detection model, a multi-scale feature branch is constructed based on the spatial void pyramid structure to obtain the defect detection model to be trained; wherein, the multi-scale feature branch is connected to the Neck module in the YOLOv8 model.

[0025] In one embodiment, the step of performing image enhancement processing on the image to be detected to obtain a first feature image after visual content optimization for the trackside equipment includes:

[0026] The image to be detected is sequentially subjected to histogram equalization, power transformation, grayscale transformation and high-pass filtering to obtain the first feature image after visual content optimization for the trackside equipment.

[0027] Secondly, this application also provides a trackside equipment appearance defect detection device, comprising:

[0028] The acquisition module is used to acquire images of the trackside equipment to be detected;

[0029] The optimization module is used to perform image enhancement processing on the image to be detected to obtain a first feature image after visual content optimization for the trackside equipment;

[0030] The extraction module is used to input the first feature image into a pre-trained target extraction model, and use the target detection model to locate and extract the trackside equipment in the image to be detected, thereby obtaining a second feature image with non-trackside equipment elements removed.

[0031] The detection module is used to input the second feature image into a pre-trained defect detection model to obtain the defect detection result for the trackside equipment; the defect detection model is trained based on real appearance defect images after sample expansion.

[0032] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the method described in the first aspect.

[0033] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method described in the first aspect.

[0034] Fifthly, this application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the method described in the first aspect.

[0035] The aforementioned methods, devices, computer equipment, computer-readable storage media, and computer program products for detecting apparent defects in trackside equipment improve the original image quality of railway trackside equipment through image enhancement processing, solving the problem of low image quality under complex lighting conditions; they utilize target extraction models to remove complex background interference, focusing on trackside equipment and improving the accuracy and robustness of defect detection; and the defect detection model based on sample expansion training can effectively address the problems of sample imbalance and scarcity, improving defect detection accuracy. Attached Figure Description

[0036] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0037] Figure 1 This is a flowchart illustrating a method for detecting apparent defects in trackside equipment in one embodiment;

[0038] Figure 2 This is a flowchart illustrating the steps for obtaining a defect detection model in one embodiment;

[0039] Figure 3 This is a schematic diagram of the sample augmentation process in one embodiment;

[0040] Figure 4 This is a schematic diagram of the framework of a defect detection model in one embodiment;

[0041] Figure 5 This is a schematic diagram of multiple types of images containing trackside equipment originally acquired in one embodiment;

[0042] Figure 6 This is a schematic diagram illustrating the detection effect of track capacitor defect detection in one embodiment;

[0043] Figure 7 This is a schematic diagram illustrating the detection effect of a communication cable leaky clamp defect detection in one embodiment;

[0044] Figure 8 This is a schematic diagram illustrating the detection effect of transponder defect detection in one embodiment;

[0045] Figure 9 This is a structural block diagram of a trackside equipment apparent defect detection device in one embodiment;

[0046] Figure 10 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0047] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0048] To enable those skilled in the art to better understand this application, the relevant technologies are first introduced below.

[0049] With the rapid development of railway transportation, the scale of railway construction continues to expand. Railway trackside electrical equipment, installed on both sides of the track, is used for transmitting dynamic train information or for the operation and maintenance of the railway transportation system, and is a key component of railway lines. However, due to long-term exposure to the outdoor environment, abnormal defects are prone to appear on the surface of trackside electrical equipment, seriously threatening the safety of the railway transportation system. Therefore, it is crucial to detect surface defects in trackside electrical equipment.

[0050] Currently, computer vision-based automatic railway surface defect detection methods are mainly divided into traditional image processing methods and deep learning methods. Traditional image processing methods rely on manually designed features to extract key visual information from railway images and combine them with prior knowledge to identify defects. The applicant found that these methods have low feature robustness and the detection accuracy needs further improvement. Deep learning methods, on the other hand, design deep networks to automatically extract surface features of trackside electrical equipment based on a large amount of data and detect defects through sufficient model training. However, for surface images of trackside electrical equipment in complex imaging environments, existing methods struggle to detect rare or subtle defects. Furthermore, in the railway transportation environment, the applicant found that trackside electrical equipment is scattered and of varying types, resulting in uncertainty in the number and type of trackside electrical equipment in each image acquired by the inspection vehicle data acquisition system. Additionally, trackside electrical equipment with surface defects occupies a smaller area in the image, and the defect area is even smaller, leading to an imbalance between defective and normal surface data. Directly detecting surface defects in the original acquired trackside electrical equipment images is difficult to achieve accurate results.

[0051] Therefore, to address the problems of insufficient accuracy, imbalanced defect ratios, and difficulty in detecting minute defects in existing railway trackside electrical equipment appearance defect detection technologies, and considering the critical role of railway trackside electrical equipment in the railway transportation system and its long-term exposure to the field, which makes it prone to defects and thus threatens transportation safety, existing computer vision-based detection methods are insufficient to effectively address the challenges of low image quality, significant background interference, and small defect areas in trackside electrical equipment. Therefore, this application aims to provide an efficient and accurate railway trackside electrical equipment appearance defect detection solution. Through innovative technical means, it improves detection accuracy and reliability, providing strong support for ensuring the safe and stable operation of the railway transportation system.

[0052] In one embodiment, such as Figure 1 As shown, a method for detecting apparent defects in trackside equipment is provided. This embodiment illustrates the application of this method to a server. It is understood that this method can also be applied to a terminal, or to a system including both a terminal and a server, and implemented through interaction between the terminal and the server. In this embodiment, the method includes the following steps:

[0053] Step S102: Obtain the image to be detected, which includes the trackside equipment.

[0054] Trackside equipment refers to various electronic and electrical devices and facilities installed on both sides of railway lines, near the tracks, or between the tracks to ensure train operation safety, improve transportation efficiency, and enable train control and communication. It can also be called trackside electrical equipment. However, these devices are prone to surface defects due to long-term exposure to the elements.

[0055] Among them, the images to be inspected that include trackside equipment can refer to images that include trackside equipment collected by the integrated inspection train. These images may have problems such as uneven illumination and noise interference, which can affect defect detection.

[0056] For example, a comprehensive inspection train can be used to collect images of the trackside electrical equipment along the railway line and upload the collected images to a server as images to be inspected containing the trackside equipment; the server can directly retrieve the stored images to be inspected containing the trackside equipment from its internal storage; or the comprehensive inspection train can acquire the collected images to be inspected containing the trackside equipment in real time.

[0057] Step S104: Perform image enhancement processing on the image to be detected to obtain the first feature image after visual content optimization for the trackside equipment.

[0058] Image enhancement processing refers to performing a series of operations on the original image, such as histogram equalization, power transform, grayscale transformation, and frequency domain filtering, to improve the visual effect and enhance feature quality, facilitating subsequent defect detection. Visual content optimization for trackside equipment refers to improving the visualization of trackside equipment in the image by enhancing the detailed features of the trackside equipment area.

[0059] For example, the server performs image enhancement processing on the image to be detected using a preset image processing algorithm or pre-installed image processing software to obtain a first image after visual content optimization for the trackside equipment.

[0060] Step S106: Input the first feature image into the pre-trained target extraction model, and use the target detection model to locate and extract the trackside device in the image to be detected, thereby obtaining the second feature image with non-trackside device elements removed.

[0061] The target extraction model can refer to a pre-trained model used to locate and extract trackside devices in the image to be detected, which can remove non-trackside device elements and focus the detection range.

[0062] For example, the server inputs the first feature image into a pre-trained target extraction model, which identifies the location of trackside equipment in the image and extracts the trackside equipment based on the feature patterns learned during training, thereby removing background elements of surrounding non-trackside equipment to obtain the second feature image.

[0063] Step S108: Input the second feature image into the pre-trained defect detection model to obtain the defect detection results for the trackside equipment. The defect detection model is trained using real-world defect images augmented with additional samples.

[0064] Among them, defect detection results can refer to output information indicating whether there are apparent defects in the trackside equipment and the specific details of the defects. Apparent defects can refer to abnormal conditions appearing on the surface of the trackside equipment, such as cracks, wear, etc.

[0065] For example, the server inputs the second feature image into a pre-trained defect detection model, which determines whether there are apparent defects and the nature of the defects in the trackside equipment based on the learned defect feature patterns, and outputs the defect detection results for the trackside equipment.

[0066] Because trackside electrical equipment is scattered and of different types in the railway transportation environment, the number and type of trackside electrical equipment in each image collected by the inspection vehicle data acquisition system are uncertain. Furthermore, trackside electrical equipment with apparent defects occupies a smaller area in the image, and the area of ​​its apparent defect is even smaller. The ratio of defect appearance data to normal appearance data is unbalanced. In this embodiment, the sample is expanded based on real appearance defect images and used as training samples to train the defect detection model to be trained, thus obtaining a trained defect detection model.

[0067] In the above-mentioned method for detecting apparent defects in trackside equipment, image enhancement processing is used to improve the original image quality of railway trackside equipment, solving the problem of low image quality under complex lighting conditions; a target extraction model is used to remove complex background interference, focusing on the trackside equipment and improving the accuracy and robustness of defect detection; the defect detection model based on sample expansion training can effectively cope with the problems of sample imbalance and scarcity, and improve the accuracy of defect detection.

[0068] In an exemplary embodiment, the step of obtaining the target extraction model includes:

[0069] A target extraction network to be trained is constructed based on YOLOv8. During the training process of the target extraction network to be trained using training images, the regression loss between the predicted bounding box and the true bounding box of the trackside equipment area in the training image is obtained through the CIoU loss calculation model. The category loss and confidence loss of the trackside electrical equipment are calculated through Focal loss. The target extraction network to be trained is trained based on the regression loss, category loss and confidence loss to obtain the target extraction model.

[0070] The training image can refer to an image containing trackside equipment and its actual bounding box information.

[0071] For example, firstly, a target extraction network framework to be trained is built based on the YOLOv8 algorithm structure. A large number of training images containing trackside equipment are prepared and input into the network. The network predicts the bounding boxes and categories of the trackside equipment. The CIoU (Complete Intersection over Union Loss) loss model is used to calculate the regression loss based on the positional relationship between the predicted and ground truth bounding boxes. Simultaneously, the Focal (Focal Loss) loss is used to calculate the category loss and confidence loss. Then, the calculated regression loss, category loss, and confidence loss are backpropagated to adjust the parameters of the target extraction network to be trained. Training is iterated continuously until the model converges, resulting in the target extraction model.

[0072] In another possible example, when building the target extraction network framework to be trained based on YOLOv8, the weights of the YOLOv8 model pre-trained on the COCO dataset can be transferred to the target extraction network framework to be trained. When training with training images, the total number of training epochs is set to 100, the batch size to 64, the initial learning rate to 0.001, and the Adam (Adaptive Moment Estimation) optimizer is used. The CIoU loss model is used to calculate the regression loss based on the positional relationship between the predicted bounding box and the ground truth bounding box; simultaneously, the Focal loss is used to calculate the class loss and confidence loss. The calculated regression loss, class loss, and confidence loss are then backpropagated to adjust the parameters of the target extraction network to be trained, and training is iterated until the model converges, resulting in the target extraction model. After training, the trained target extraction model is used to locate trackside devices in the image, and the corresponding ROI (Region of Interest) is extracted based on the detected position coordinates, thereby removing background areas of non-trackside devices from the image and reducing the interference of other elements in the background on the defect detection algorithm.

[0073] In this embodiment, a target extraction network to be trained is constructed based on YOLOv8. During training, regression loss is calculated using CIoU loss, which can more accurately measure the positional difference between the predicted bounding box and the ground truth bounding box, making the target extraction model more accurate in locating trackside equipment. Focal loss effectively addresses the sample imbalance problem, improves the model's ability to identify different types of trackside equipment and the confidence level of the prediction results, and further enhances the performance of the target extraction model.

[0074] In one exemplary embodiment, such as Figure 2 As shown, the steps for obtaining the defect detection model include:

[0075] Step S202: Obtain a pre-prepared first training image containing appearance defects and a second training image not containing appearance defects.

[0076] The trackside devices included in the first training image and the second training image are of the same type.

[0077] The first training image containing apparent defects can be a feature image obtained by processing an image containing apparent defects of trackside equipment through the aforementioned image enhancement and target extraction model. The second training image not containing apparent defects can be a feature image obtained by processing an image containing trackside equipment but without apparent defects through the aforementioned image enhancement and target extraction model (i.e., ROI region extraction).

[0078] Step S204: Use the appearance defects in the first training image to process the second training image to obtain a second training image containing appearance defects.

[0079] In this context, processing the second training image using appearance defects in the first training image can refer to fusing the appearance defects in the first training image into the second training image through image processing methods.

[0080] Step S206: Use the first training image containing the appearance defect and the second training image containing the appearance defect as training samples.

[0081] Step S208: Train the defect detection model to be trained using training samples to obtain the defect detection model.

[0082] For example, a first training image containing apparent defects and a second training image without apparent defects can be selected from a collected training image library, ensuring that the trackside equipment type is the same in both types of images; all images in the training image library have undergone image enhancement processing and target extraction model processing; then, through a pre-configured model, the apparent defects in the first training image are fused into the second training image to obtain a second training image containing apparent defects; the first training image and the processed second training image containing apparent defects are combined to form a training sample set for training the defect detection model; the training samples are input into the defect detection model to be trained, and through continuous iterative training, the model parameters are adjusted so that the model learns the apparent defect features until the model converges, thus obtaining the defect detection model.

[0083] In this embodiment, by processing the second training image that does not contain apparent defects, a second training image containing apparent defects is obtained, which expands the types and number of defect samples, effectively solving the problem of sample imbalance and scarcity in the apparent defect detection task, enabling the defect detection model to learn richer defect features and improve detection accuracy.

[0084] In an exemplary embodiment, processing a second training image using appearance defects in a first training image to obtain a second training image containing appearance defects includes:

[0085] The defective pixel region in the first training image is extracted using the SAM model; the defective pixel region is fused into the second training image, and the defective region edge is optimized in the fused second training image based on Poisson fusion to obtain the second training image containing the apparent defect.

[0086] Here, SAM model can refer to Segment Anything Model, which can identify and segment any object based on text commands or image recognition. Defective pixel regions can refer to the pixel regions in the first training image corresponding to apparent defects.

[0087] For example, the first training image is input into the SAM model, which identifies and segments defective pixel regions in the image based on the defect feature patterns it has learned. The extracted defective pixel regions are then fused into the second training image according to certain rules. Poisson fusion technology is then used to optimize the edges of the defective regions in the fused image, making the edge transitions natural, resulting in a second training image that contains apparent defects and is of higher quality.

[0088] In this embodiment, the SAM model can accurately extract defect pixel regions, providing accurate defect information for sample expansion; Poisson fusion is then used to ensure the quality of the edges of the fused defect regions, making the expanded samples closer to the real situation, which helps the defect detection model to better learn defect features and further improve detection accuracy.

[0089] In one exemplary embodiment, such as Figure 3 As shown, the sample augmentation process can specifically include: First, based on the SAM large model and designing corresponding prompts, extracting the apparent defect region from the trackside equipment ROI image (the defect image, i.e., the first training image). Using the extracted defect region, obtain the corresponding mask image, and segment the defect pixel region according to the defect mask to obtain the defect region image. Then, based on the obtained defect mask image, multiply the normal trackside electrical equipment ROI image (the normal image, i.e., the second training image) without apparent defects pixel by pixel, and add the defect region image to it pixel by pixel, thereby fusing the apparent defect with the normal image. Subsequently, use the Poisson fusion method to optimize the defect region edges of the fused augmented defect sample to ensure that the edges of the fused defect region are not distorted. Finally, obtain the augmented trackside electrical equipment defect sample and construct the training dataset required for the defect detection model. In this embodiment, the SAM model is used to extract the defect pixel region in the defect image. Combined with the Poisson fusion technique, the defect region is seamlessly fused into the normal trackside electrical equipment image, which expands the types and number of defect samples and solves the problem of sample imbalance and scarcity in the apparent defect detection task.

[0090] In an exemplary embodiment, the step of obtaining the defect detection model to be trained includes:

[0091] Obtain the YOLOv8 model; replace the original backbone network in the YOLOv8 model with the lightweight attention network MobileNet-SE to obtain the first detection model; construct multi-scale feature branches based on the spatial void pyramid structure in the first detection model to obtain the defect detection model to be trained; wherein, the multi-scale feature branches are connected to the Neck module in the YOLOv8 model respectively.

[0092] The lightweight attention network MobileNet-SE can refer to a network obtained by integrating the MobileNet architecture with the Squeeze-and-Excitation (SE) attention module. The original backbone network can refer to the main network structure originally used for feature extraction in the YOLOv8 model.

[0093] For example, the steps for obtaining the defect detection model to be trained can be as follows: Figure 4As shown, to achieve efficient detection of trackside devices, the feature extraction part of the YOLOv8 model is structurally improved by introducing a lightweight attention network, MobileNet-SE, as the backbone network. This design maintains the real-time requirements of the model in edge device deployment while effectively improving the semantic representation ability and detection accuracy of the target region (ROI). The overall network structure retains the original YOLOv8 framework, replacing the traditional backbone network with MobileNet-SE, thus achieving a balance between model computational complexity and feature modeling capability.

[0094] The MobileNet-SE backbone network is primarily built upon the MobileNet architecture. Its core idea is to decompose standard convolutions into Depthwise (DW) and Pointwise (PW) convolutions using Depthwise Separable Convolution (DSConv). Using DW and PW significantly reduces the model's parameter size and computational cost. Let the input feature map dimension be... The number of output channels is The kernel size is Then when and At that time, depthwise separable convolution has a computational complexity approximately reduced compared to standard convolution. For example, when This reduces processing time by approximately 87.5%, significantly improving efficiency. In MobileNet, each Bottleneck structure consists of continuous... PW convolution, DW convolution, To further enhance feature representation capabilities, we construct the model using PW convolutions. After extracting intermediate features at each Bottleneck, we add a Squeeze-and-Excitation (SE) attention module. The SE module is a channel attention mechanism that dynamically assigns different weights to each channel based on global context information, improving the response of key channels. This adaptively learns the importance of each feature channel, strengthening features useful to the model and suppressing redundant features. The modeling process of the SE module is as follows:

[0095] Let the intermediate output feature map be , its first Each channel is denoted as The Squeeze operation then compresses the spatial dimension of the features through global average pooling, obtaining the global features for each channel:

[0096]

[0097] The Excitation operation will obtain the channel description vector. Input a nonlinear transformation containing two fully connected layers to generate weight vectors for each feature channel. :

[0098]

[0099] in, , ; For channel compression ratio, It is the ReLU activation function. This is the Sigmoid function.

[0100] Finally, the Hadamard product is used to weight each channel:

[0101]

[0102] Weighted results As the output of this SE, adaptive feature enhancement in the channel dimension is achieved.

[0103] Integrating the SE module constructed above with the MobileNet structure forms the MobileNet-SE backbone network.

[0104] Furthermore, to obtain multi-scale information at different levels, a multi-scale feature branch was constructed based on the Spatial Hollow Pyramid (ASPP) structure. This was applied to the input ROI image. After downsampling and convolutional transformation of the shallow network, the extracted shallow features are enhanced to fully utilize high-level semantic features and low-level details. Specifically, dilated convolutional layers are stacked in the feature extraction network, with three densely connected dilated convolutional layers. Each layer has a 3×3 kernel, and the dilation rates are 2, 3, and 6 from top to bottom. The output of each dilated layer is as follows:

[0105]

[0106] in, For kernel size, For the first The void ratio of the layer This represents dilated convolution, which yields three multi-scale feature maps at different resolutions. , , Among them, low-level features Preserve more shallow texture details in areas of equipment defects, medium-level features Capture structural information, high-level features This is used to represent high-level semantic content. Simultaneously, it maximizes the receptive field of the spatial void pyramid module. The resolution is set to 99, and the input ROI resolution is 1000×1000, resulting in a 100×100 feature map input to ASPP. Finally, a 1×1 convolution operation is performed on all features from dilated convolutional layers to aggregate them into a multi-scale feature map, effectively enhancing the feature information. Simultaneously, the number of output channels of ASPP is adjusted to reduce the additional parameters introduced by dense connections.

[0107] Finally, these multi-scale features are input into the YOLOv8 Neck module (composed of FPN+PAN modules) for cross-scale information fusion, ultimately forming the multi-scale enhanced features required by the detection head. Through this design, the improved network not only significantly reduces the parameter size and FLOPs of YOLOv8, but also adaptively enhances the multi-scale features extracted by the model through the SE module and ASPP structure, thereby improving the detection accuracy and robustness of target defect regions on electrical equipment, especially demonstrating stronger discrimination capabilities in scenarios with small targets and complex backgrounds. The final improved network maintains the end-to-end target detection framework while balancing the dual requirements of lightweight deployment and detection performance.

[0108] In this embodiment, the feature extraction part of the YOLOv8 model is structurally improved by replacing the original backbone network with MobileNet-SE. This maintains the model's lightweight and real-time performance while enhancing its semantic representation ability of the target region, enabling the model to better extract features of trackside equipment. Furthermore, a multi-scale feature branch is constructed based on ASPP to acquire multi-scale information at different levels. This information is then fused across scales by connecting with the Neck module, improving the model's detection accuracy and robustness against defects at different scales, especially in scenarios with small targets and complex backgrounds.

[0109] In an exemplary embodiment, the image to be detected is subjected to image enhancement processing to obtain a first feature image after visual content optimization for the trackside equipment. This includes: sequentially performing histogram equalization processing, power transformation processing, grayscale transformation processing, and high-pass filtering processing on the image to be detected to obtain the first feature image after visual content optimization for the trackside equipment.

[0110] like Figure 5 As shown, the image to be detected can be as follows: Figure 5 The original image is shown; specifically, the image enhancement processing steps can be as follows:

[0111] Histogram equalization processing involves redistributing grayscale values ​​to expand the dynamic range of the original image to the full range, resulting in an image with enhanced contrast. This enhances the detail of trackside electrical equipment, thereby improving image visualization. Specifically, the input image's grayscale histogram is first... Calculate the cumulative distribution function (CDF) of image gray levels based on the gray-level histogram:

[0112]

[0113] in, These are the various gray levels of the image. It is grayscale. The probability of.

[0114] Then, the original grayscale values ​​of the image are mapped to equalized grayscale values ​​according to the cumulative distribution function:

[0115]

[0116] in, For mapping functions, It is the maximum value of the image's grayscale level (usually 256).

[0117] The power transform processing includes: adjusting pixel values ​​in different illumination areas of the image to highlight key features of trackside electrical equipment under varying lighting conditions, thereby enhancing the detail representation of the trackside electrical equipment area in the image. Specifically, the grayscale values ​​of each pixel in the image are processed as follows:

[0118]

[0119] in, To output pixel grayscale values, The input pixel grayscale value is the normalized value. It is a proportionality coefficient. It represents the power exponent.

[0120] Gray-scale transformation processing includes adjusting the gray-scale value distribution of the image through a linear mapping function to enhance the contrast of the trackside electrical equipment area. Specifically, the gray-scale values ​​of each pixel in the image are transformed as follows:

[0121]

[0122] in, This represents the original grayscale value range. The target grayscale value range.

[0123] High-pass filtering involves using a high-pass filter in the frequency domain to filter the image, thereby preserving high-frequency information while suppressing low-frequency information and emphasizing image details. Specifically, the original spatial domain image is first filtered... Convert to frequency domain:

[0124]

[0125] The acquired frequency domain image Multiply by the high-pass filter:

[0126]

[0127] in, It is a Butterworth high-pass filter.

[0128] Finally, an inverse Fourier transform is performed on the processed frequency domain image to transform it back into a spatial domain image. :

[0129]

[0130] After the above optimization process, the visual effect and feature quality of the original trackside electrical equipment images can be significantly improved. The enhanced images have higher contrast and more prominent details, providing high-quality input data for subsequent trackside electrical equipment appearance defect detection algorithms.

[0131] In this embodiment, multiple image enhancement processes are performed sequentially to comprehensively improve the contrast, brightness uniformity, and detail clarity of the image. This effectively solves the problem of low image quality under complex lighting conditions, providing high-quality image data for subsequent trackside equipment inspection and helping to improve the accuracy and reliability of the entire inspection process.

[0132] After fully training the improved defect detection model, the defect detection effect can be verified on a pre-prepared trackside equipment defect test dataset. The performance of the trackside electrical equipment defect detection algorithm can be evaluated by precision and recall. The experimental results are shown in the table below.

[0133] Trackside electrical equipment Defect detection accuracy Defect detection recall rate Track capacitor 91.4% 93.1% Communication leaky cable clamp 86.8% 95.2% transponder 86.5% 97.5%

[0134] The experimental results shown in the table above demonstrate that the trackside equipment apparent defect detection method provided in this application achieves high precision and recall rates for all three types of trackside electrical equipment defects in the test set. Specifically, the precision and recall rates for track capacitor apparent defect identification are as high as 91.4% and 93.1%, respectively; for transponder apparent defect identification, they are as high as 86.8% and 95.2%, respectively; and for communication cable clamp apparent defect identification, they are as high as 86.5% and 97.5%, respectively. It is clear that the trackside equipment apparent defect detection method of this application exhibits excellent detection performance for trackside electrical equipment apparent defects, with the best recall rate, enabling more accurate detection of defect samples. Figure 6 , Figure 7 as well as Figure 8As shown, the visualization results of three types of trackside electrical equipment appearance defect detection are presented, which can intuitively show the detection effect of trackside equipment appearance defect detection methods on different trackside equipment appearance defects.

[0135] To enable those skilled in the art to better understand the above steps, the following example illustrates the embodiments of this application, but it should be understood that the embodiments of this application are not limited thereto. This embodiment may include the following steps: acquiring an image to be detected containing trackside equipment; sequentially performing histogram equalization, power transformation, grayscale transformation, and high-pass filtering on the image to be detected to obtain a first feature image after visual content optimization for trackside equipment; inputting the first feature image into a pre-trained target extraction model, using the target detection model to locate and extract trackside equipment in the image to be detected, to obtain a second feature image with non-trackside equipment elements removed; inputting the second feature image into a pre-trained defect detection model to obtain defect detection results for trackside equipment; the defect detection model is trained based on real appearance defect images after sample augmentation; wherein, the step of acquiring the target extraction model includes: constructing a target extraction network to be trained based on YOLOv8; during the training process of the target extraction network to be trained using training images, obtaining the regression loss between the predicted bounding box and the real bounding box of the trackside equipment region in the training image through the CIoU loss calculation model, and calculating the category loss and confidence loss of trackside electrical equipment through Focal loss; training the target extraction network to be trained based on the regression loss, category loss, and confidence loss to obtain the target extraction model. The steps for obtaining the defect detection model include: acquiring a pre-prepared first training image containing apparent defects and a second training image not containing apparent defects; the trackside equipment contained in the first and second training images is of the same type; extracting defect pixel regions from the first training image using the SAM model; fusing the defect pixel regions into the second training image, and performing defect region edge optimization processing on the fused second training image based on Poisson fusion to obtain a second training image containing apparent defects; using the first and second training images containing apparent defects as training samples; and training the defect detection model to be trained using the training samples to obtain the defect detection model. The steps for obtaining the defect detection model to be trained include: acquiring a YOLOv8 model; replacing the original backbone network in the YOLOv8 model with a lightweight attention network MobileNet-SE to obtain a first detection model; constructing multi-scale feature branches based on a spatial void pyramid structure in the first detection model to obtain the defect detection model to be trained; wherein the multi-scale feature branches are connected to the Neck module in the YOLOv8 model.

[0136] In summary, the trackside equipment appearance defect detection method proposed in this application, by adopting a coarse-to-fine detection strategy—that is, first extracting images of the trackside equipment area and then performing appearance defect detection—effectively improves the detection accuracy of appearance defects in railway trackside electrical equipment, and provides an effective technical means to ensure the safety of the railway transportation system.

[0137] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages in other steps. It is understood that the steps in different embodiments can be freely combined as needed, and all non-contradictory solutions formed by such combinations are within the scope of protection of this application.

[0138] Based on the same inventive concept, this application also provides a trackside equipment appearance defect detection device for implementing the above-mentioned trackside equipment appearance defect detection method. The solution provided by this device is similar to the implementation solution described in the above method. Therefore, the specific limitations of one or more trackside equipment appearance defect detection device embodiments provided below can be found in the limitations of the trackside equipment appearance defect detection method above, and will not be repeated here.

[0139] In one exemplary embodiment, such as Figure 9 As shown, a trackside equipment appearance defect detection device is provided, comprising: an acquisition module 901, an optimization module 902, an extraction module 903, and a detection module 904, wherein:

[0140] The acquisition module 901 is used to acquire the image to be detected, which includes the trackside equipment;

[0141] Optimization module 902 is used to perform image enhancement processing on the image to be detected to obtain a first feature image after visual content optimization for trackside equipment;

[0142] The extraction module 903 is used to input the first feature image into a pre-trained target extraction model, and use the target detection model to locate and extract the trackside device in the image to be detected, thereby obtaining a second feature image with non-trackside device elements removed.

[0143] The detection module 904 is used to input the second feature image into the pre-trained defect detection model to obtain the defect detection result for the trackside equipment; the defect detection model is trained based on the real appearance defect image after sample expansion.

[0144] In an exemplary embodiment, the above-mentioned trackside equipment apparent defect detection device further includes a model acquisition module, which is used to construct a target extraction network to be trained based on YOLOv8. During the training process of the target extraction network to be trained using training images, the regression loss between the predicted bounding box and the true bounding box of the trackside equipment region in the training image is obtained through the CIoU loss calculation model, and the category loss and confidence loss of the trackside electrical equipment are calculated through Focal loss. The target extraction network to be trained is trained based on the regression loss, category loss and confidence loss to obtain the target extraction model.

[0145] In an exemplary embodiment, the model acquisition module is further configured to acquire a pre-prepared first training image containing apparent defects and a second training image not containing apparent defects; the trackside equipment contained in the first training image and the second training image are of the same type; process the second training image using the apparent defects in the first training image to obtain a second training image containing apparent defects; use the first training image containing apparent defects and the second training image containing apparent defects as training samples; train the defect detection model to be trained using the training samples to obtain a defect detection model.

[0146] In an exemplary embodiment, the model acquisition module is further configured to extract defective pixel regions in the first training image using the SAM model; fuse the defective pixel regions into the second training image; and perform defective region edge optimization processing on the fused second training image based on Poisson fusion to obtain a second training image containing apparent defects.

[0147] In an exemplary embodiment, the model acquisition module is further configured to acquire a YOLOv8 model; replace the original backbone network in the YOLOv8 model with a lightweight attention network MobileNet-SE to obtain a first detection model; construct multi-scale feature branches based on a spatial void pyramid structure in the first detection model to obtain a defect detection model to be trained; wherein the multi-scale feature branches are respectively connected to the Neck module in the YOLOv8 model.

[0148] In an exemplary embodiment, the optimization module 902 is further configured to sequentially perform histogram equalization, power transformation, grayscale transformation and high-pass filtering on the image to be detected to obtain a first feature image after visual content optimization for the trackside equipment.

[0149] Each module in the aforementioned trackside equipment surface defect detection device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of a computer device in hardware form or independent of it, or stored in the memory of a computer device in software form, so that the processor can call and execute the corresponding operations of each module.

[0150] In one exemplary embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 10 As shown, the computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communication with external terminals via a network connection. When the computer program is executed by the processor, it implements a method for detecting apparent defects in trackside equipment.

[0151] Those skilled in the art will understand that Figure 10 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0152] In one embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above method embodiments.

[0153] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps in the above method embodiments.

[0154] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0155] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0156] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0157] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for detecting apparent defects in trackside equipment, characterized in that, The method includes: Acquire the image to be detected, which includes trackside equipment; The image to be detected is subjected to image enhancement processing to obtain a first feature image after visual content optimization for the trackside equipment; The first feature image is input into a pre-trained target extraction model, and the target detection model is used to locate and extract the trackside equipment in the image to be detected, resulting in a second feature image with non-trackside equipment elements removed. The second feature image is input into a pre-trained defect detection model to obtain the defect detection result for the trackside equipment; the defect detection model is trained based on real appearance defect images after sample expansion.

2. The method according to claim 1, characterized in that, The steps for obtaining the target extraction model include: A target extraction network to be trained is constructed based on YOLOv8. During the training process of the target extraction network to be trained using training images, the regression loss between the predicted bounding box and the real bounding box of the trackside equipment area in the training image is obtained through the CIoU loss calculation model. The category loss and confidence loss of the trackside electrical equipment are calculated through Focal loss. The target extraction model is obtained by training the target extraction network based on the regression loss, the discrimination loss, and the confidence loss.

3. The method according to claim 1, characterized in that, The steps for obtaining the defect detection model include: Acquire a pre-prepared first training image containing apparent defects and a second training image not containing apparent defects; the trackside devices contained in the first training image and the second training image are of the same type; The appearance defects in the first training image are used to process the second training image to obtain a second training image containing the appearance defects. The first training image containing the appearance defect and the second training image containing the appearance defect are used as training samples. The defect detection model is trained using the training samples to obtain the defect detection model.

4. The method according to claim 3, characterized in that, The step of processing the second training image using appearance defects in the first training image to obtain a second training image containing the appearance defects includes: The defective pixel regions in the first training image are extracted using the SAM model. The defective pixel region is fused into the second training image, and the defective region edge is optimized based on Poisson fusion to obtain a second training image containing the apparent defect.

5. The method according to claim 3, characterized in that, The steps for obtaining the defect detection model to be trained include: Obtain the YOLOv8 model; The original backbone network in the YOLOv8 model was replaced with the lightweight attention network MobileNet-SE to obtain the first detection model; In the first detection model, a multi-scale feature branch is constructed based on the spatial void pyramid structure to obtain the defect detection model to be trained; wherein, the multi-scale feature branch is connected to the Neck module in the YOLOv8 model.

6. The method according to claim 1, characterized in that, The step of performing image enhancement processing on the image to be detected to obtain a first feature image after visual content optimization for the trackside equipment includes: The image to be detected is sequentially subjected to histogram equalization, power transformation, grayscale transformation and high-pass filtering to obtain the first feature image after visual content optimization for the trackside equipment.

7. A trackside equipment surface defect detection device, characterized in that, The device includes: The acquisition module is used to acquire images of the trackside equipment to be detected; The optimization module is used to perform image enhancement processing on the image to be detected to obtain a first feature image after visual content optimization for the trackside equipment; The extraction module is used to input the first feature image into a pre-trained target extraction model, and use the target detection model to locate and extract the trackside equipment in the image to be detected, thereby obtaining a second feature image with non-trackside equipment elements removed. The detection module is used to input the second feature image into a pre-trained defect detection model to obtain the defect detection result for the trackside equipment; the defect detection model is trained based on real appearance defect images after sample expansion.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images