Railway freight car cover fault identification method and device based on deep learning

Abstract

A method and device for identifying railway freight car cover faults based on deep learning relates to the field of image processing technology. The present invention is intended to solve the problem of low accuracy and efficiency in manual detection of railway freight car cross-support device covers. The present invention inputs a test image containing a test component into a trained deep learning semantic segmentation model, generates a set of coordinates of all pixel points in the result image, and uses the set as a feature vector set. The feature vectors in the feature vector set are divided into multiple categories according to the minimum distance principle; the cluster centers of each category are calculated respectively; it is determined whether the cluster centers of two adjacent iterations in any category are the same, and if so, the minimum circumscribed rectangular area of all pixel points in each category in the clustering result is calculated respectively, and the category with the minimum circumscribed rectangular area less than 50 pixels is screened out, and the image formed by the screened category is used as the segmentation result to determine whether the segmentation result is deformed.

Description

Technical Field

[0001] The present invention belongs to the field of image processing technology, and in particular relates to the recognition of covers of cross-support devices of railway freight cars. Background Art

[0002] The train braking system is an important component of the train, and its performance and braking capacity are directly related to whether the train can operate safely. The advantages of the bogie equipped with the cross-support device cover in transportation are very obvious, which is manifested in the change of the dynamic performance of the bogie, the enhancement of the anti-diameter stiffness, the realization of the elastic positioning of the bogie, and the improvement of the running speed and stability of the train. However, after running for a period of time, the cover may be deformed due to external impact, resulting in uneven force on the cover, affecting the stability of the cover and affecting driving safety. Therefore, it is necessary to detect the status of the cross-support device cover during operation. The existing technology adopts a manual inspection method of looking at pictures one by one. This method is affected by the quality and sense of responsibility of the personnel, and the problems of missed inspections occur frequently, making it difficult to guarantee the quality of the operation. At the same time, there are also problems such as huge labor costs and low efficiency. Summary of the Invention

[0003] The present invention aims to solve the problem that the current method of detecting the cover of the cross support device of railway freight cars is manual detection, which has low accuracy and low efficiency. A method and device for identifying railway freight car cover faults based on deep learning are now provided.

[0004] The railway freight car cover fault identification method based on deep learning includes the following steps:

[0005] Step 1: Input the image containing the component to be tested into the trained deep learning semantic segmentation model to obtain the result image containing the component to be identified;

[0006] Step 2: Generate a set of all pixel coordinates in the result image and use this set as the feature vector set.

[0007] Step 3: Divide the feature vectors in the feature vector set into C categories according to the minimum distance principle. The cluster centers of the C categories are expressed as: Where k represents the number of iterations and the initial value is 0;

[0008] Step 4: Calculate the cluster center of each category at the k+1th iteration

[0009] Step 5: Determine whether the cluster centers of the k+1th and kth iterations in any category are the same. If so, obtain the clustering result and execute step 6. Otherwise, set k=k+1 and return to step 3.

[0010] Step 6: Calculate the minimum bounding rectangle area of all pixels in each category in the clustering results, filter out the categories with a minimum bounding rectangle area less than 50 pixels, and use the image composed of the filtered categories as the segmentation result;

[0011] Step 7: Determine whether the segmentation result is deformed. If so, it is determined that the cover of the railway freight car cross support device is deformed; otherwise, it is not deformed.

[0012] Furthermore, the loss function of the above deep learning semantic segmentation model is a fusion loss function of the Dice loss function and the cross entropy loss function.

[0013] Furthermore, the expression of the above fusion loss function L is:

[0014] L=a×bce+(1-a)×dice,

[0015] Among them, a is the coefficient of the cross entropy loss function, bce represents the cross entropy loss function, and dice represents the Dice loss function.

[0016] The expression of the cross entropy loss function bce is:

[0017]

[0018] The expression of Dice loss function dice is:

[0019]

[0020] Above, M is the number of categories. When the predicted category and the sample category are the same, y r =1, otherwise y r =0, p r To predict the probability that the sample belongs to the rth category, P is the predicted result, T is the true result, r = 1, 2, ..., M.

[0021] Furthermore, in the initial state, k = 0, C feature vectors are randomly selected from the feature vector set as the initial cluster centers of C categories, and are expressed as

[0022] Furthermore, in step 3 above, the minimum distance principle is expressed as follows:

[0023]

[0024] in, is the characteristic vector x i The nearest cluster center of the lth class The distance between is the feature vector x i and the cluster center of the jth class The distance between them, j = 1, 2, ..., C, i = 1, 2, ..., N, N is the total number of feature vectors.

[0025] Furthermore, in the above step 4, the cluster center at the k+1th iteration is calculated according to the following formula:

[0026]

[0027] in, is the cluster center of the k+1th iteration in the jth category, is the clustering at the k+1th iteration in the jth category, For clustering The number of eigenvectors contained in .

[0028] Furthermore, in the above step 7, the specific method for determining whether the segmentation result is deformed is:

[0029] Determine whether the area difference between the two parts of the segmentation result relative to the central axis of the freight car bogie is greater than 50 pixels. If so, the part is deformed; otherwise, it is not deformed.

[0030] The railway freight car cover fault identification device based on deep learning includes the following units:

[0031] Segmentation unit: used to input the image under test containing the tested component into the trained deep learning semantic segmentation model to obtain the result image containing the component to be identified;

[0032] Set generation unit: used to generate a set of all pixel coordinates in the result image and use the set as a feature vector set.

[0033] Classification unit: It is used to divide the feature vectors in the feature vector set into C categories according to the minimum distance principle. The cluster centers of the C categories are expressed as: Where k represents the number of iterations and the initial value is 0;

[0034] Clustering unit: used to calculate the cluster center of each category at the k+1th iteration

[0035] Cluster judgment unit: used to judge whether the cluster centers of the k+1th and kth iterations in any category are the same. If yes, the clustering result is obtained and the screening unit is executed; otherwise, k=k+1 and the classification unit is executed;

[0036] Screening unit: used to calculate the minimum enclosing rectangle area of all pixels in each category in the clustering results, screen out categories with a minimum enclosing rectangle area less than 50 pixels, and use the image composed of the filtered categories as the segmentation result;

[0037] Deformation judgment unit: used to judge whether the segmentation result is deformed. If so, it is determined that the cover plate of the railway freight car cross support device is deformed; otherwise, it is not deformed.

[0038] A computer-readable storage device stores a computer program, which, when executed, implements the above-mentioned railway freight car cover fault identification method based on deep learning.

[0039] A railway freight car cover fault identification system based on deep learning includes a storage device, a processor, and a computer program stored in the storage device and runnable on the processor. The system is characterized in that the processor executes the computer program to implement the above-mentioned railway freight car cover fault identification method based on deep learning.

[0040] The beneficial effects of the present invention are as follows:

[0041] 1. Introducing automatic identification technology into truck fault detection enables automatic fault identification and alarm. Human operators only need to confirm the alarm results, effectively saving labor costs and improving work quality and efficiency.

[0042] 2. Applying deep learning algorithms to the automatic identification of deformation faults in cross-support device covers has higher accuracy and stability than traditional machine vision detection methods.

[0043] 3. Based on the semantic segmentation network Unet, the Unet network is combined with the ResNet residual network to improve the network's feature extraction effect on the deformation fault of the cross-support device cover and enhance the fault detection effect.

[0044] 4. The loss function is improved by combining the Dice loss function and the cross entropy loss function as a fusion loss function to improve the segmentation results of small targets.

[0045] 5. Based on the characteristics of semantic segmentation results, a fault area screening method based on cluster analysis is proposed, which can accurately denoise the segmentation results and is conducive to subsequent fault judgment.

[0046] In summary, the present invention is of great significance for the automated detection of deformation faults in cross-support cover plates. By combining image processing and deep learning technologies, automatic fault identification and alarm can be achieved, effectively improving the quality and efficiency of vehicle inspection operations. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 Flowchart for model training;

[0048] Figure 2 Schematic diagram of the Res-UNet network structure;

[0049] Figure 3is a schematic diagram of the structure of the residual module and the decoding module, where 3(a) is a schematic diagram of the structure of the residual module, Figure 3 (b) is a schematic diagram of the decoding module structure;

[0050] Figure 4 Flowchart of the railway freight car cover fault identification method based on deep learning. DETAILED DESCRIPTION

[0051] The following will be combined with the accompanying drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention. It should be noted that the embodiments of the present invention and the features in the embodiments can be combined with each other in the absence of conflict.

[0052] Specific implementation method 1: refer to Figures 1 to 4 Specifically describing this embodiment, the railway freight car cover fault identification method based on deep learning described in this embodiment includes the following steps:

[0053] Step 1: High-definition equipment is installed at the bottom of the truck tracks to capture images of the trucks passing at high speed. Line scanning enables seamless image stitching, generating a wide-field, high-precision 2D image. The cross-support cover is roughly located based on the truck's wheelbase and bogie type information. An image of a localized area containing the component is captured from the larger bottom image as the image to be tested, effectively reducing fault identification time and improving recognition accuracy.

[0054] The UNet network is widely used in the field of medical image segmentation. Its U-shaped structure, consisting of an encoding network and a decoding network, ensures that the network can integrate shallow and deep features, making it more suitable for identifying small objects with irregular edges. ResNet introduces cross-layer connected residual blocks, which can extract features at depth while avoiding gradient vanishing and network degradation. Res-UNet combines the advantages of the UNet network with the ResNet residual network, fully extracting and utilizing image features, avoiding gradient vanishing, and shortening training time.

[0055] Res-UNet uses ResNet as the encoding network and adds 6 cross-layer connections. It unifies the dimensions through the maximum pooling operation. The network structure can reuse shallow features and enhance the depth extension of features. The decoding network structure of the UNet network remains unchanged, realizing the restoration of feature image resolution and the combination with shallow features. The improved Res-UNet structure is as follows: Figure 2As shown. The encoding network includes 4 residual modules, such as Figure 3 As shown in (a), each module contains two convolutional layers, an activation function layer (Rectified Linear Unit, ReLU) and two batch normalization layers (BatchNormalization, BN). The size of the convolution kernel of the convolution layer is 3×3, and the skip connection can prevent the gradient from disappearing during back propagation. The decoding network includes 5 decoding modules, such as Figure 3 As shown in (b), each module includes upsampling, two Conv2d convolutional layers, and two ReLU activation function layers. The upsampling method uses nearest neighbor interpolation to double the size of the feature map. The convolution kernel size is 3×3 with a stride of 1 for feature fusion. There are four skip connections between the encoding network and the decoding network to splice features at the same level, enabling the reuse of shallow features.

[0056] In this embodiment, the image under test containing the component to be tested is input into a trained deep learning semantic segmentation model to obtain a result image containing the component to be identified. In semantic segmentation models, the Dice coefficient is often used as a loss function to train the model. Since it focuses on whether the foreground image is correctly classified and does not pay attention to background pixels, it can effectively alleviate the problem of imbalance between foreground and background samples. The expression of the Dice loss function dice is:

[0057]

[0058] However, the Dice loss function has a poor effect on the segmentation of small targets, so the binary cross entropy loss function bce is introduced to have a better effect on foreground and background classification. The expression is:

[0059]

[0060] The loss function of the deep learning semantic segmentation model in this embodiment is a fusion loss function of the Dice loss function and the cross entropy loss function. The fusion loss function is mainly based on the Dice loss function, supplemented by the cross entropy loss function, and the ratio of dice to cross entropy is set to 2:1. The expression of the fusion loss function L is:

[0061] L=a×bce+(1-a)×dice,

[0062] Among them, the coefficient of the cross entropy loss function a=0.33, M is the number of categories, when the predicted category and the sample category are the same, y r =1, otherwise y r =0, p r To predict the probability that the sample belongs to the rth category, P is the predicted result, T is the true result, r = 1, 2, ..., M.

[0063] Since the semantic segmentation results may contain noise interference such as mis-segmentation, the segmentation results need to be further screened to extract the deformed areas. Common image processing screening methods include: area size screening, contour length screening, color screening, etc. However, there is incoherence in the semantic segmentation results, and direct screening may cause inaccurate screening results. In view of the characteristics of the semantic segmentation results, this embodiment uses the segmentation result pixel point coordinate set as the feature vector set to be classified, pre-determines C categories and selects C initial cluster centers, and assigns the feature vectors to one of the C categories according to the minimum distance principle. After that, the class center is continuously calculated and the category of each feature vector is adjusted, and finally the sum of the squares of the distances from each feature vector to the center of the category to which it belongs is minimized. Specifically, there are the following steps.

[0064] Step 2: Generate a set of all pixel coordinates in the result image and use the set as the feature vector set.

[0065] Step 3: Divide the feature vectors in the feature vector set into C categories according to the minimum distance principle. The minimum distance principle is expressed as follows:

[0066]

[0067]

[0068] is the characteristic vector x i The nearest cluster center of the lth class The distance between is the feature vector x i Clustering with the jth class The cluster center The distance between is the updated cluster of the lth class, j = 1, 2, ..., C, i = 1, 2, ..., N, and N is the total number of feature vectors.

[0069] The cluster centers of C categories are expressed as: Where k represents the number of iterations, and in the initial state, k = 0. C feature vectors are randomly selected from the feature vector set as the initial cluster centers of C categories, and are expressed as

[0070] Step 4: Calculate the cluster center of each category at the k+1th iteration according to the following formula

[0071] in, is the cluster center of the k+1th iteration in the jth category, is the clustering at the k+1th iteration in the jth category, For clustering The number of eigenvectors contained in .

[0072] Step 5: Determine whether the cluster centers of the k+1th and kth iterations in any category are the same. If so, obtain the clustering result and execute step 6. Otherwise, set k=k+1 and return to step 3.

[0073] Step 6: Calculate the minimum bounding rectangle area of all pixels in each category in the clustering results, filter out the categories with a minimum bounding rectangle area less than 50 pixels, and use the image composed of the filtered categories as the segmentation result;

[0074] Step 7: Determine whether the area difference between the two parts of the segmentation result relative to the central axis of the freight car bogie is greater than 50 pixels. If so, it is determined that the cover plate of the railway freight car cross support device is deformed; otherwise, no deformation is present.

[0075] Step 8: Generate a message with the fault location and fault category based on the fault information and upload it to the alarm platform.

[0076] Furthermore, training a deep learning semantic segmentation model requires a training image dataset. Truck components may be affected by natural or man-made conditions such as rain, mud, oil stains, and black paint. Furthermore, images captured at different locations may vary. Therefore, when collecting training image datasets, diversity should be ensured, including images captured under a variety of conditions and at different locations. Each sample image dataset consists of an original image set and a labeled information set. The original image set consists of coarsely positioned images of the cross-support cover component, captured by the equipment. The labeled information set contains information about the rectangular subregions containing the component, obtained through manual labeling. There is a one-to-one correspondence between the original image set and the labeled information dataset, meaning each image corresponds to a single labeled data point. While the sample dataset includes images from a variety of conditions, data augmentation is still required to improve algorithm stability. Horizontal and vertical flipping is used to enhance image diversity; image brightness and contrast are adjusted to reduce the effects of uneven lighting and highlight the edge features of defects; histogram equalization is used to enhance image contrast; and Gaussian blur and random noise are added to enhance model generalization.

[0077] Specific embodiment 2: The railway freight car cover fault identification device based on deep learning described in this embodiment includes the following units:

[0078] Segmentation unit: used to input the image under test containing the tested component into the trained deep learning semantic segmentation model to obtain the result image containing the component to be identified;

[0079] Set generation unit: used to generate a set of all pixel coordinates in the result image and use the set as a feature vector set.

[0080] Classification unit: It is used to divide the feature vectors in the feature vector set into C categories according to the minimum distance principle. The cluster centers of the C categories are expressed as: Where k represents the number of iterations and the initial value is 0;

[0081] Clustering unit: used to calculate the cluster center of each category at the k+1th iteration

[0082] Cluster judgment unit: used to judge whether the cluster centers of the k+1th and kth iterations in any category are the same. If yes, the clustering result is obtained and the screening unit is executed; otherwise, k=k+1 and the classification unit is executed;

[0083] Screening unit: used to calculate the minimum enclosing rectangle area of all pixels in each category in the clustering results, screen out categories with a minimum enclosing rectangle area less than 50 pixels, and use the image composed of the filtered categories as the segmentation result;

[0084] Deformation judgment unit: used to judge whether the segmentation result is deformed. If so, it is determined that the cover plate of the railway freight car cross support device is deformed; otherwise, it is not deformed.

[0085] Specific embodiment three: This embodiment describes a computer-readable storage device, wherein the storage device stores a computer program, and when the computer program is executed, the method described in specific embodiment one is implemented.

[0086] Specific embodiment four: The railway freight car cover fault identification system based on deep learning described in this embodiment includes a storage device, a processor, and a computer program stored in the storage device and capable of running on the processor. The processor executes the computer program to implement the method described in specific embodiment one.

[0087] Although the present invention is described herein with reference to specific embodiments, it should be understood that these embodiments are merely illustrative of the principles and applications of the invention. It should be understood that many modifications may be made to the illustrative embodiments, and that other arrangements may be devised, without departing from the spirit and scope of the invention as defined by the appended claims. It should be understood that the various dependent claims and features described herein may be combined in ways other than those described in the original claims. It should also be understood that features described in conjunction with individual embodiments may be used in conjunction with other described embodiments.

Claims

1. A railway freight car cover fault identification method based on deep learning, characterized in that: The following steps are involved: Step 1: Input the image containing the component to be tested into the trained deep learning semantic segmentation model to obtain the result image containing the component to be identified; Step 2: Generate a set of all pixel coordinates in the result image and use the set as the feature vector set; Step 3: Divide the feature vectors in the feature vector set into C categories according to the minimum distance principle. The cluster centers of the C categories are expressed as: Where k represents the number of iterations and the initial value is 0; Step 4: Calculate the cluster center of each category at the k+1th iteration Step 5: Determine whether the cluster centers of the k+1th and kth iterations in any category are the same. If so, obtain the clustering result and execute step 6. Otherwise, set k=k+1 and return to step 3. Step 6: Calculate the minimum bounding rectangle area of all pixels in each category in the clustering results, filter out the categories with a minimum bounding rectangle area less than 50 pixels, and use the image composed of the filtered categories as the segmentation result; Step 7: Determine whether the segmentation result is deformed. If so, it is determined that the cover of the railway freight car cross support device is deformed; otherwise, it is not deformed.

2. The railway freight car cover fault identification method based on deep learning according to claim 1 is characterized in that: The loss function of the deep learning semantic segmentation model is a fusion loss function of the Dice loss function and the cross entropy loss function.

3. The railway freight car cover fault identification method based on deep learning according to claim 2 is characterized in that: The expression of the fusion loss function L is: L=a×bce+(1-a)×dice, Among them, a is the coefficient of the cross entropy loss function, bce represents the cross entropy loss function, and dice represents the Dice loss function. The expression of the cross entropy loss function bce is: The expression of the Dice loss function dice is: Above, M is the number of categories. When the predicted category and the sample category are the same, y r =1, otherwise y r =0, p r To predict the probability that the sample belongs to the rth category, P is the predicted result, T is the true result, r = 1, 2, ..., M.

4. The railway freight car cover fault identification method based on deep learning according to claim 1 is characterized in that: In the initial state, k = 0, and C feature vectors are randomly selected from the feature vector set as the initial cluster centers of C categories, and are expressed as 5. The railway freight car cover fault identification method based on deep learning according to claim 1 is characterized in that: In step 3, the minimum distance principle is expressed as follows: in, is the characteristic vector x i The nearest cluster center of the lth class The distance between is the feature vector x i and the cluster center of the jth class The distance between them, j = 1, 2, ..., C, i = 1, 2, ..., N, N is the total number of feature vectors.

6. The railway freight car cover fault identification method based on deep learning according to claim 5 is characterized in that: In step 4, the cluster center at the k+1th iteration is calculated according to the following formula: in, is the cluster center of the k+1th iteration in the jth category, is the clustering at the k+1th iteration in the jth category, For clustering The number of eigenvectors contained in .

7. The railway freight car cover fault identification method based on deep learning according to claim 1 is characterized in that: In step 7, the specific method for determining whether the segmentation result is deformed is: Determine whether the area difference between the two parts of the segmentation result relative to the central axis of the freight car bogie is greater than 50 pixels. If so, the part is deformed; otherwise, it is not deformed.

8. A railway freight car cover fault identification device based on deep learning, characterized in that: The following units are included: Segmentation unit: used to input the image under test containing the tested component into the trained deep learning semantic segmentation model to obtain the result image containing the component to be identified; Set generation unit: used to generate a set of all pixel coordinates in the result image and use the set as a feature vector set. Classification unit: used to classify the feature vectors in the feature vector set into C categories according to the minimum distance principle. The cluster centers of the C categories are expressed as: Where k represents the number of iterations and the initial value is 0; Clustering unit: used to calculate the cluster center of each category at the k+1th iteration Cluster judgment unit: used to judge whether the cluster centers of the k+1th and kth iterations in any category are the same. If yes, the clustering result is obtained and the screening unit is executed; otherwise, k=k+1 and the classification unit is executed; Screening unit: used to calculate the minimum enclosing rectangle area of all pixels in each category in the clustering results, screen out categories with a minimum enclosing rectangle area less than 50 pixels, and use the image composed of the filtered categories as the segmentation result; Deformation judgment unit: used to judge whether the segmentation result is deformed. If so, it is determined that the cover plate of the railway freight car cross support device is deformed; otherwise, it is not deformed.

9. A computer-readable storage device storing a computer program, characterized in that: When the computer program is executed, the method according to any one of claims 1 to 7 is implemented.

10. A railway freight car cover fault identification system based on deep learning, comprising a storage device, a processor, and a computer program stored in the storage device and executable on the processor, characterized in that: The processor executes the computer program to implement the method according to any one of claims 1 to 7.

Images