Guide rail surface defect segmentation method based on double-input network

Abstract

The invention belongs to the technical field of guide rail defect detection, and particularly relates to a guide rail surface defect segmentation method based on a double-input network, which comprises the following steps: completing data acquisition of a guide rail surface image, and preprocessing the data; building a dual-input neural network structure, and completing feature extraction; after the extraction is completed, the features of the two different input paths are merged together to realize feature fusion; performing network training by using the marked guide rail defect data set to complete optimization and adjustment; performing post-processing on a segmentation result output by the network; and carrying out subsequent defect detection on the segmentation result. According to the method, two input sources are introduced, one is an original image of the surface of the guide rail, the other is a depth image of the surface of the guide rail or other auxiliary information graphs (such as a texture information graph and a grey-scale graph), the features of different information sources are fully fused through training of the double-input network, and more accurate defect segmentation is achieved.

Description

Technical Field

[0001] The present invention relates to the technical field of guide rail defect detection, and specifically to a guide rail surface defect segmentation method based on a dual-input network. Background Art

[0002] Defect detection on the surface of guide rails is a crucial safety guarantee measure in the railway transportation system. Timely detection of defects on the guide rail surface can effectively avoid safety accidents caused by guide rail damage.

[0003] In recent years, with the development of deep learning technology, automatic defect detection methods based on deep neural networks have gradually been applied to the field of guide rail defect detection. In particular, convolutional neural networks (CNNs) have demonstrated powerful performance in image segmentation.

[0004] However, due to the diversity and complexity of guide rail surface defects and the differences in image acquisition conditions, existing single-input networks often struggle to meet the diverse detection requirements of guide rail surface defects. Therefore, proposing a new guide rail surface defect segmentation method based on a dual-input network has important research significance and practical application value;

[0005] Currently, guide rail defect detection methods mainly rely on manual inspection and traditional image processing techniques. Although these methods can detect defects to a certain extent, they have problems such as low detection efficiency, poor accuracy, and being limited by human subjective factors.

[0006] It should be noted that the above content belongs to the technical cognition scope of the inventor and does not necessarily constitute prior art. Summary of the Invention

[0007] Aiming at the deficiencies of the prior art, the present invention provides a guide rail surface defect segmentation method based on a dual-input network, which solves the existing problems.

[0008] To achieve the above object, the present invention provides the following technical solution: A guide rail surface defect segmentation method based on a dual-input network, comprising the following steps:

[0009] Step 1: Complete the data acquisition of the guide rail surface image and preprocess the data;

[0010] Step 2: Complete the construction of the dual-input neural network structure and perform feature extraction;

[0011] Step 3: After the extraction, merge the features of the two different input paths to achieve feature fusion;

[0012] Step 4: Use the labeled guide rail defect data set for network training to complete optimization and adjustment;

[0013] Step Five: Post-process the segmentation results output by the network;

[0014] Step Six: Perform subsequent defect detection on the segmentation results, calculate the area, position, and morphological features of the defect regions, and evaluate the accuracy and efficiency of the detection results.

[0015] In some embodiments, the guide rail surface image is acquired by a high-precision industrial camera;

[0016] The specific steps of the data preprocessing are as follows:

[0017] 1) Select Gaussian filtering and median filtering methods to denoise the original image of the guide rail surface, thereby removing the noise generated due to environmental light and camera noise;

[0018] 2) Standardize the image, adjust each image pixel value to a fixed value, and control the fixed value range between 0 and 1;

[0019] 3) Adopt data augmentation techniques to perform operations such as rotation, translation, scaling, and mirroring on the image, enhance the diversity of the dataset, and reduce the overfitting phenomenon;

[0020] 4) Obtain the depth image of the guide rail surface, including its texture and sharpness, and perform corresponding processing on these auxiliary images to ensure that they have the same size and resolution as the original image.

[0021] In some embodiments, the specific steps of building the dual-input neural network structure are as follows:

[0022] 1) Extract features from the original image through a convolutional layer, and at the same time, extract features from the auxiliary image through an independent convolutional layer;

[0023] 2) Each input undergoes feature extraction through an independent convolutional layer, and then the features of the two inputs are fused through a feature fusion module;

[0024] 3) Complete the segmentation of the guide rail surface defects through the decoding part of the convolutional neural network.

[0025] 4. According to the method for segmenting guide rail surface defects based on a dual-input network described in claim 1, wherein the specific steps of the feature fusion in step three are as follows:

[0026] 1) Perform weighted fusion on the features of the two inputs, and assign different weights to the features from different sources;

[0027] 2) Perform further convolutional operations on the fused feature map to fuse more high-level features.

[0028] In some of these embodiments, the optimization and adjustment steps are as follows:

[0029] 1) Select the labeled dataset of guide rail surface defects, which includes different types of defects and defect-free samples;

[0030] 2) Select the Dice system loss to measure the similarity between the prediction result and the true label;

[0031] 3) During the training process, calculate the gradient through the backpropagation algorithm and update the network parameters using the optimization algorithm;

[0032] 4) Use the early stopping technique to avoid overfitting, adopt the learning rate decay strategy to improve the training efficiency, and use data augmentation and regularization techniques to improve the generalization ability of the model.

[0033] In some of these embodiments, the specific steps for post-processing the segmentation result are as follows:

[0034] 1) Perform morphological operations on the segmentation result output by the network to remove small regional noise;

[0035] 2) Use morphological operations to improve the shape of the defect area, including smoothing the boundary and filling in the missing parts to ensure that the shape of the defect area is more reasonable;

[0036] 3) When there are multiple discontinuous defect areas in the network segmentation result, the connected component algorithm can be used to merge these areas to ensure that the detected defects are complete and coherent.

[0037] In some of these embodiments, the specific steps for defect detection and evaluation are as follows:

[0038] 1) Further analyze the segmented defect area to detect the type, location, size, and shape of the defect;

[0039] 2) Based on these features, judge the severity of the defect and classify it according to the preset rules;

[0040] 3) Use standard evaluation metrics to evaluate the performance of the model.

[0041] Compared with the prior art, the present invention provides a method for segmenting guide rail surface defects based on a dual-input network, which has the following beneficial effects:

[0042] The method for segmenting rail surface defects based on a dual-input network introduces two input sources. One is the original image of the rail surface, and the other is the depth image of the rail surface or other auxiliary information images (such as texture information images, grayscale images, etc.). Through the training of the dual-input network, the features of different information sources are fully integrated to achieve more accurate defect segmentation. It can effectively improve the automation level of rail surface defect detection and is widely applied in fields such as railway transportation, track detection, and intelligent railway maintenance. With the continuous improvement of the modernization and automation of the railway system, this technology will provide strong technical support for the safety detection and maintenance of rails, promoting the safety and stability of railway transportation. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 It is a schematic diagram of the steps of the method for segmenting rail surface defects based on a dual-input network according to the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0044] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the embodiments of the present invention and the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0045] It should be understood that the step numbers used in the text are only for convenience of description and do not limit the order of execution of the steps.

[0046] It should be understood that the terms used in the specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the specification of the present invention and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include the plural forms.

[0047] The terms "comprising" and "including" indicate the presence of the described features, wholes, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or their combinations.

[0048] The term " / and / " refers to any combination and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0049] Please refer to Figure 1 , in this embodiment: A method for segmenting rail surface defects based on a dual-input network includes the following steps:

[0050] Step 1: Complete the data acquisition of the rail surface image and preprocess the data;

[0051] The image of the guide rail surface is acquired by a high-precision industrial camera;

[0052] The specific steps of data preprocessing are as follows:

[0053] 1) Select Gaussian filtering and median filtering methods to denoise the original image of the guide rail surface, so as to remove the noise caused by environmental light and camera noise;

[0054] 2) Standardize the image, adjust each image pixel value to a fixed value, and control the fixed value range between 0-1, so that the input of the neural network is more consistent, thereby improving the convergence speed and stability of the model;

[0055] 3) Adopt data augmentation technology to perform processing methods such as rotation, translation, scaling, and mirroring on the image, enhance the diversity of the dataset, reduce overfitting, and improve the generalization ability of the model;

[0056] 4) Obtain the depth image of the guide rail surface, including its texture and sharpness, and perform corresponding processing on these auxiliary images to ensure that they have the same size and resolution as the original image;

[0057] Before performing defect segmentation on the guide rail surface, it is necessary to preprocess the original image and the auxiliary information image first to ensure the quality of the input data and improve the recognition effect of the network;

[0058] Step 2: Complete the construction of the dual-input neural network structure and complete feature extraction;

[0059] The specific steps for constructing the dual-input neural network structure are as follows:

[0060] 1) Extract features from the original image through the convolutional layer, and at the same time, the auxiliary image also extracts features through an independent convolutional layer. One input path enables the network to obtain additional information that is different from but related to the original image, supplementing the deficiencies in the original image;

[0061] 2) Each input extracts features through an independent convolutional layer, and then through the feature fusion module, the features of the two inputs are fused;

[0062] 3) Complete the segmentation of the guide rail surface defects through the decoding part of the convolutional neural network. The network will learn the relationship between the original image and the auxiliary image and extract the features that best represent the defect information;

[0063] Step 3: After extraction, merge the features of the two different input paths to achieve feature fusion;

[0064] The specific steps of feature fusion are as follows:

[0065] 1) Weightedly fuse the features of the two inputs, and assign different weights to the features from different sources;

[0066] 2) Perform further convolution operations on the fused feature map to fuse more high-level features, which can help the network learn more complex defect patterns and thus improve the effect of defect segmentation;

[0067] Step 4: Use the labeled guide rail defect dataset for network training to complete optimization and adjustment;

[0068] The specific steps of optimization and adjustment are as follows:

[0069] 1) Select the labeled guide rail surface defect dataset, which includes different types of defects and defect-free samples;

[0070] 2) Select the Dice system loss to measure the similarity between the prediction result and the true label;

[0071] 3) During the training process, calculate the gradient through the backpropagation algorithm and update the network parameters using the optimization algorithm. The goal of the network is to minimize the loss function and adjust the weights so that the segmentation result is as close as possible to the true label;

[0072] 4) Use the early stopping technique to avoid overfitting, adopt the learning rate decay strategy to improve the training efficiency, and use data augmentation and regularization techniques to improve the generalization ability of the model;

[0073] Step 5: Post-process the segmentation result output by the network;

[0074] The specific steps of post-processing the segmentation result are as follows:

[0075] 1) Perform morphological operations on the segmentation result output by the network to remove small area noise;

[0076] 2) Use morphological operations to improve the shape of the defect area, including smoothing the boundary and filling the missing part to ensure that the shape of the defect area is more reasonable;

[0077] 3) When there are multiple discontinuous defect areas in the network segmentation result, the connected component algorithm can be used to merge these areas to ensure that the detected defects are complete and coherent;

[0078] Step 6: Perform subsequent defect detection on the segmentation result, calculate the area, position and morphological features of the defect area, and evaluate the accuracy and efficiency of the detection result;

[0079] The specific steps of defect detection and evaluation are as follows:

[0080] 1) Further analyze the segmented defect areas to detect the type, location, size, and shape of the defects;

[0081] 2) Based on these features, judge the severity of the defects and classify them according to preset rules;

[0082] 3) Evaluate the performance of the model using standard evaluation metrics.

[0083] Each embodiment in this specification is described in a progressive manner. For the same or similar parts among the embodiments, reference can be made to each other. Each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, it is described relatively simply. For the relevant parts, reference can be made to the partial description of the method embodiment.

[0084] Finally, it should be noted that the above are only the preferred embodiments of the present invention and are not used to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions recorded in the foregoing embodiments or perform equivalent replacements for some of the technical features. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for segmenting guide rail surface defects based on a dual-input network, characterized in that, It includes the following steps: Step 1: Complete the data acquisition of the guide rail surface image and preprocess the data; Step 2: Complete the construction of the dual-input neural network structure and complete feature extraction; Step 3: After the extraction, merge the features of the two different input paths to achieve feature fusion; Step 4: Use the labeled guide rail defect dataset for network training to complete optimization and adjustment; Step 5: Post-process the segmentation result output by the network; Step 6: Perform subsequent defect detection on the segmentation result, calculate the area, position and morphological features of the defect area, and evaluate the accuracy and efficiency of the detection result.

2. The method for dividing guide rail surface defects based on a dual-input network according to claim 1, characterized in that In Step 1, the guide rail surface image is acquired by a high-precision industrial camera; The specific steps of the data preprocessing are as follows: 1) Select Gaussian filtering and median filtering methods to denoise the original image of the guide rail surface, so as to remove the noise generated due to environmental light and camera noise; 2) Standardize the image, adjust each image pixel value to a fixed value, and the fixed value range is controlled between 0 and 1; 3) Adopt data augmentation technology to perform processing methods such as rotation, translation, scaling and mirroring on the image, improve the diversity of the dataset, and reduce the overfitting phenomenon; 4) Obtain the depth image of the guide rail surface, including its texture and sharpness, and perform corresponding processing on these auxiliary images to ensure that they have the same size and resolution as the original image.

3. A method for segmenting surface defects of a guide rail based on a dual-input network according to claim 1, characterized in that, The specific steps of constructing the dual-input neural network structure in Step 2 are as follows: 1) Extract features from the original image through the convolutional layer, and the auxiliary image also extracts features through an independent convolutional layer; 2) Each input extracts features through an independent convolutional layer, and then through the feature fusion module, fuse the features of the two inputs; 3) Complete the segmentation of the guide rail surface defects through the decoding part of the convolutional neural network.

4. A method for dividing rail surface defects based on a dual-input network according to claim 1, characterized in that: The specific steps of the feature fusion in Step 3 are as follows: 1) Perform weighted fusion on the features of the two inputs, and assign different weights to the features from different sources; 2) Perform further convolutional operations on the fused feature map to fuse more high-level features.

5. A method for dividing rail surface defects based on a dual-input network according to claim 1, characterized in that: The specific steps of the optimization and adjustment in Step 4 are as follows: 1) Select the labeled guide rail surface defect dataset, which includes different types of defects and defect-free samples; 2) Select the Dice system loss to measure the similarity between the prediction result and the true label; 3) During the training process, calculate the gradient through the backpropagation algorithm and update the network parameters using the optimization algorithm; 4) Use the early stopping technique to avoid overfitting, adopt the learning rate decay strategy to improve the training efficiency, and use data augmentation and regularization techniques to improve the generalization ability of the model.

6. A method for dividing rail surface defects based on a dual-input network according to claim 1, characterized in that: The specific steps of the post-processing of the segmentation result in Step 5 are as follows: 1) Perform morphological operations on the segmentation result output by the network to remove small area noise; 2) Use morphological operations to improve the shape of the defect area, including smoothing the boundary and filling the missing part, to ensure that the morphology of the defect area is more reasonable; 3) When there are multiple discontinuous defect regions in the network segmentation results, the connected component algorithm can be used to merge these regions to ensure that the detected defects are complete and coherent.

7. A method for segmenting rail surface defects based on a dual-input network according to claim 1, characterized in that: The specific steps of the defect detection and evaluation described in step six are as follows: 1) Further analyze the segmented defect regions to detect the type, location, size, and shape of the defects; 2) Based on these features, judge the severity of the defects and classify them according to preset rules; 3) Use standard evaluation metrics to evaluate the performance of the model.

Images