Deep learning-based train side apron board loss fault detection method

Abstract

The invention relates to the technical field of detection, and discloses a method for detecting a loss fault of a side apron board of a train based on deep learning, which is characterized by comprising the following steps of: acquiring images of a normal apron board and an apron board to be detected; according to the deep learning-based train side apron board loss fault detection method, a high-definition imaging device is built around a train track to obtain a to-be-detected apron board image, the to-be-detected apron board image is denoised and subjected to shape fitting, and a normal apron board image and the to-be-detected apron board image are compared and classified; the method comprises the following steps: constructing a deep convolutional neural network model, importing classified images of a to-be-detected apron board into the deep convolutional neural network model, carrying out quality evaluation separation on the apron board through the deep convolutional neural network model, carrying out evaluation on a detected apron board condition type set, and uploading and alarming a quality evaluation structure. A worker carries out corresponding processing according to an identification result, and safe operation of the train is guaranteed.

Description

Technical Field

[0001] This invention relates to the field of detection technology, specifically to a method for detecting the loss of train side skirts based on deep learning. Background Technology

[0002] The side skirts of high-speed trains contain various covers, grilles, battery boxes, and other components. These side skirts protect these components and play a vital role in the safe operation of the train. However, side skirts can easily become missing during high-speed train operation, and if not detected in time, this can seriously jeopardize train safety.

[0003] When using manual image inspection for fault detection, the high density of EMU operation and short inspection time make it easy for inspectors to become fatigued or overlook certain aspects, leading to missed or incorrect detections of side skirt loss faults and affecting train operation safety. Therefore, in order to overcome the shortcomings of manual inspection methods, existing methods use image processing algorithms to identify side skirt loss faults. However, due to the influence of external factors such as lighting and weather, the quality of the acquired train images will vary, and traditional image processing algorithms are highly dependent on image quality, resulting in poor generalization ability of fault identification algorithms. Summary of the Invention

[0004] To address the shortcomings of existing technologies, this invention provides a deep learning-based method for detecting train side skirt loss faults, which has advantages such as good detection effect and strong recognition capability, and solves the problems mentioned in the background art.

[0005] To achieve the above objectives, the present invention provides the following technical solution: a method for detecting train side skirt loss faults based on deep learning, characterized by comprising the following steps: S1: Obtain images of a normal skirt board and a skirt board to be detected, and obtain the edge pixels of the normal skirt board and the skirt board to be detected images through contour detection; S2: Perform shape fitting on the edge pixels of the normal skirt board and the skirt board to be detected image to determine the basic shape of the edge pixels of the normal skirt board and the skirt board to be detected image; S3: Compare the images of the normal skirt board with the skirt board to be tested, and classify the skirt board to be tested into five categories: scratches (A), dents (B), looseness (C), missing (D), and normal (E). S4: Set scratches A, dents B, looseness C and missing D as negative samples, and normal E as positive samples. After classification, construct a skirt database from the sample set. Divide the skirt database into training set T1, validation set T2 and test set T3 in a ratio of 7:2:1. S5: Construct a deep convolutional neural network model, including an input layer, two convolutional layers, two pooling layers, a fully connected layer, a softmax classification layer, and an output layer; S6: Use a deep convolutional neural network model to perform quality assessment and classification on the acquired skirt board images to be detected; S7: Evaluate the collection of detected skirt board conditions.

[0006] Preferably, the specific steps for obtaining the edge pixels of the normal skirt board and the skirt board to be detected in S1 are as follows: S1.1: Perform grayscale processing on the acquired images of the normal skirt board and the skirt board to be detected to obtain grayscale images; S1.2: Perform Gaussian filtering and adaptive binarization on the grayscale image to obtain a binarized image; S1.3: Identify the edge pixels of the normal skirt board and the skirt board to be detected from the binarized image.

[0007] Preferably, the specific comparison step in S3 is as follows: S3.1: Count the number of edge pixels in the skirt board image to be detected, and calculate the coordinates of the four corner edge pixels of the skirt board to be detected. If the coordinates of the four corner edge pixels of the skirt board to be detected are not detected, it is determined that D is missing. If the coordinates of the four corner edge pixels of the skirt board to be detected are detected, proceed to step S3.2. S3.2: Calculate the length m1 and width n1 of the normal skirt board and the length m2 and width n2 of the skirt board to be tested using the plane coordinate formula. Calculate the area S1 of the normal skirt board and the area S2 of the skirt board to be tested. If S1 > S2, it is determined to be loose C. If S1 = S2, proceed to step S3.3. S3.3: Perform erosion processing on the binary image of the skirt board to be inspected after smoothing, and obtain the connected region Oi of the eroded binary image. Compare the connected region Oi of the skirt board to be inspected with the connected region Pi of the normal skirt board. If the connected region Oi > the connected region Pi, it is determined that there is a scratch A or a dent B, and proceed to step S3.4. If the connected region Oi = the connected region Pi, it is determined to be normal E. S3.4: Obtain the leftmost coordinates (X1, Y1) and rightmost coordinates (X2, Y2) of the connected region Oi using the disjoint-set data structure algorithm, calculate the aspect ratio Q of the connected region Oi, compare the aspect ratio Q of the connected region Oi with the set threshold α, if the aspect ratio Q of the connected region Oi is less than the threshold α, it is considered to be a depression B, otherwise it is considered to be a scratch A, and proceed to step S3.5; S3.5: Construct a skirt panel database by identifying scratches (A), dents (B), looseness (C), missing parts (D), and normal parts (E). Divide the skirt panel database into a training set T1, a validation set T2, and a test set T3 in a ratio of 7:2:1.

[0008] Preferably, the formula for calculating the aspect ratio Q of the connected region Oi is: ; Preferably, the step of constructing the deep convolutional neural network model in S5 is as follows: S5.1: Cross-stack two convolutional layers and two pooling layers to form the main network structure, connect the softmax classification layer to the fully connected layer as the fault feature identification layer, and form a complete deep convolutional neural network model. S5.2: In a convolutional layer, for any input r, the subsequence is: r j =ρw j r j-1 Where W and ρ are the convolution operation and the nonlinear activation function, respectively; S5.3: Wj is the weight of the filter mapping, and each layer is written as the sum of the convolutions of the previous layer: ; Where u is an element of r, j = 1, 2, 3, ..., N, * is the convolution operation, k is the sample point, and g is the activation function; ; Where h is the mapping function, g is the activation function, and f is the frequency; S5.4: Calculate the gradient using the convex optimization algorithm of stochastic gradient descent and optimize the solution for weights Wj; S5.5: The target region is divided into non-overlapping regions using a pooling method. The pooling dimension is set to 3 and the number of neurons is set to 200. S5.6: Employs max pooling function to extract features and improves computational efficiency for deep convolutional neural network models; S5.7: Utilize a deep convolutional neural network structure model to extract convolutional and pooling features from the input data, train the network model structure parameters, and use the sigmoid function as the activation function in the Softmax layer. ; Where e is the base of the natural logarithm; S5.8: Train a deep convolutional neural network model using the training set T1.

[0009] Preferably, the specific operation steps of S6 are as follows: S6.1: The loss function is the cross-entropy loss function. The network is trained using the cross-entropy loss function to obtain a deep convolutional neural network model. The cross-entropy loss function is: ; Where β∈(A, B, C, D, E), β represents the true probability value of the skirt board sample. This represents the predicted probability value of the sample on the skirt panel; S6.2: Validate the model's performance using samples from the validation set T2 and fine-tune its parameters; S6.3: Use the trained deep convolutional neural network model to classify skirt board defect images of the test set T3 and statistically analyze the defect classification results.

[0010] Preferably, the evaluation expression is: ; in, Indicates the evaluation coefficient. This represents the set of occurrences of positive and negative samples in the image of the skirt panel to be detected. As weight, if Not at the threshold If the internal components fail the test, an alarm will alert maintenance personnel to perform repairs. Located at the threshold If it is inside, then the test is qualified.

[0011] Preferably, the specific method for fine-tuning in S6.2 is as follows: During the training process, the backpropagation algorithm and stochastic gradient descent method are used to update the network weights until the loss value converges to obtain the trained deep convolutional neural network model.

[0012] Compared with existing technologies, this invention provides a deep learning-based method for detecting train side skirt loss faults, which has the following advantages: This deep learning-based method for detecting missing train side skirts involves acquiring images of the skirts to be detected using high-definition imaging equipment deployed around the train track. The images are then denoised and shape-fitted. Normal skirt images are compared and classified with those of the missing skirts. A deep convolutional neural network (DCNN) model is constructed, and the classified skirt images are imported into this model. The DCNN model then performs quality assessment and separation of the skirts. Finally, the detected skirt condition categories are evaluated, and the quality assessment results are uploaded to an alarm system. Staff then take appropriate action based on the identification results to ensure safe train operation. Attached Figure Description

[0013] Figure 1 This is a schematic diagram of the detection method for train side skirt loss fault based on deep learning proposed in this invention. Figure 2 This is a schematic diagram of the S3 judgment process of a deep learning-based method for detecting train side skirt loss faults proposed in this invention. Detailed Implementation

[0014] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0015] 1. Please refer to Figure 1-2 A deep learning-based method for detecting train side skirt loss faults includes the following steps: S1: Obtain images of a normal skirt board and a skirt board to be detected, and obtain the edge pixels of the normal skirt board and the skirt board to be detected images through contour detection; S2: Perform shape fitting on the edge pixels of the normal skirt board and the skirt board to be detected image to determine the basic shape of the edge pixels of the normal skirt board and the skirt board to be detected image; S3: Compare the images of the normal skirt board with the skirt board to be tested, and classify the skirt board to be tested into five categories: scratches (A), dents (B), looseness (C), missing (D), and normal (E). S4: Set scratches A, dents B, looseness C and missing D as negative samples, and normal E as positive samples. After classification, construct a skirt database from the sample set. Divide the skirt database into training set T1, validation set T2 and test set T3 in a ratio of 7:2:1. S5: Construct a deep convolutional neural network model, including an input layer, two convolutional layers, two pooling layers, a fully connected layer, a softmax classification layer, and an output layer; S6: Use a deep convolutional neural network model to perform quality assessment and classification on the acquired skirt board images to be detected; S7: Evaluate the collection of detected skirt board conditions.

[0016] The specific steps for obtaining the edge pixels of the normal skirt board and the skirt board to be detected in S1 are as follows: S1.1: Perform grayscale processing on the acquired images of the normal skirt board and the skirt board to be detected to obtain grayscale images; S1.2: Perform Gaussian filtering and adaptive binarization on the grayscale image to obtain a binarized image; S1.3: Identify the edge pixels of the normal skirt board and the skirt board to be detected from the binarized image.

[0017] The specific comparison steps in S3 are as follows: S3.1: Count the number of edge pixels in the skirt board image to be detected, and calculate the coordinates of the four corner edge pixels of the skirt board to be detected. If the coordinates of the four corner edge pixels of the skirt board to be detected are not detected, it is determined that D is missing. If the coordinates of the four corner edge pixels of the skirt board to be detected are detected, proceed to step S3.2. S3.2: Calculate the length m1 and width n1 of the normal skirt board and the length m2 and width n2 of the skirt board to be tested using the plane coordinate formula. Calculate the area S1 of the normal skirt board and the area S2 of the skirt board to be tested. If S1 > S2, it is determined to be loose C. If S1 = S2, proceed to step S3.3. ; S3.3: Perform erosion processing on the binary image of the skirt board to be inspected after smoothing, and obtain the connected region Oi of the eroded binary image. Compare the connected region Oi of the skirt board to be inspected with the connected region Pi of the normal skirt board. If the connected region Oi > the connected region Pi, it is determined that there is a scratch A or a dent B, and proceed to step S3.4. If the connected region Oi = the connected region Pi, it is determined to be normal E. The formula for calculating the aspect ratio Q of the connected region Oi is: ; S3.4: Obtain the leftmost coordinates (X1, Y1) and rightmost coordinates (X2, Y2) of the connected region Oi using the disjoint-set data structure algorithm, calculate the aspect ratio Q of the connected region Oi, compare the aspect ratio Q of the connected region Oi with the set threshold α, if the aspect ratio Q of the connected region Oi is less than the threshold α, it is considered to be a depression B, otherwise it is considered to be a scratch A, and proceed to step S3.5; To improve query efficiency, optimization strategies such as path compression and rank-based merging can be employed. Path compression refers to attaching all nodes along the path directly to the root node during the root node search process, thereby reducing the time for subsequent queries. Rank-based merging refers to attaching the set with fewer elements to the set with more elements when merging two sets, in order to maintain the balance of the tree.

[0018] The disjoint-set data structure (DFS) algorithm can be applied to many fields, such as social networks, routing protocols, and graph algorithms. It can efficiently solve some set merging and query problems, providing an effective data structure and method for addressing these issues.

[0019] S3.5: Construct a skirt panel database by identifying scratches (A), dents (B), looseness (C), missing parts (D), and normal parts (E). Divide the skirt panel database into a training set T1, a validation set T2, and a test set T3 in a ratio of 7:2:1.

[0020] The steps for building a deep convolutional neural network model in S5 are as follows: S5.1: Cross-stack two convolutional layers and two pooling layers to form the main network structure, connect the softmax classification layer to the fully connected layer as the fault feature identification layer, and form a complete deep convolutional neural network model. S5.2: In a convolutional layer, for any input r, the subsequence is: r j =ρw j r j-1 Where W and ρ are the convolution operation and the nonlinear activation function, respectively; S5.3: Wj is the weight of the filter mapping, and each layer is written as the sum of the convolutions of the previous layer: ; Where u is an element of r, j = 1, 2, 3, ..., N, * is the convolution operation, k is the sample point, and g is the activation function; ; Where h is the mapping function, g is the activation function, and f is the frequency; S5.4: Calculate the gradient using the convex optimization algorithm of stochastic gradient descent and optimize the solution for weights Wj; S5.5: The target region is divided into non-overlapping regions using a pooling method. The pooling dimension is set to 3 and the number of neurons is set to 200. S5.6: Employs max pooling function to extract features and improves computational efficiency for deep convolutional neural network models; S5.7: Utilize a deep convolutional neural network structure model to extract convolutional and pooling features from the input data, train the network model structure parameters, and use the sigmoid function as the activation function in the Softmax layer. ; Where e is the base of the natural logarithm; S5.8: Train a deep convolutional neural network model using the training set T1.

[0021] The specific operating steps for S6 are as follows: S6.1: The loss function is the cross-entropy loss function. The network is trained using the cross-entropy loss function to obtain a deep convolutional neural network model. The cross-entropy loss function is: ; Where β∈(A, B, C, D, E), β represents the true probability value of the skirt board sample. This represents the predicted probability value of the sample on the skirt panel; S6.2: Validate the model's performance using samples from the validation set T2 and fine-tune its parameters; S6.3: Use the trained deep convolutional neural network model to classify skirt board defect images of the test set T3 and statistically analyze the defect classification results.

[0022] The evaluation expression is: ; in, Indicates the evaluation coefficient. This represents the set of occurrences of positive and negative samples in the image of the skirt panel to be detected. As weight, if Not at the threshold If the internal components fail the test, an alarm will alert maintenance personnel to perform repairs. Located at the threshold If it is inside, then the test is qualified.

[0023] The specific method for fine-tuning S6.2 is as follows: During the training process, the backpropagation algorithm and stochastic gradient descent method are used to update the network weights until the loss value converges to obtain the trained deep convolutional neural network model.

[0024] This deep learning-based method for detecting missing train side skirts involves acquiring images of the skirts to be detected using high-definition imaging equipment deployed around the train track. The images are then denoised and shape-fitted. Normal skirt images are compared and classified with those of the missing skirts. A deep convolutional neural network (DCNN) model is constructed, and the classified skirt images are imported into this model. The DCNN model then performs quality assessment and separation of the skirts. Finally, the detected skirt condition categories are evaluated, and the quality assessment results are uploaded to an alarm system. Staff then take appropriate action based on the identification results to ensure safe train operation.

[0025] It should be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0026] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A method for detecting a train side skirt loss fault based on deep learning, characterized in that, The method comprises the following steps: S1: obtaining normal skirt and to-be-detected skirt images, and obtaining edge pixel points of the normal skirt and to-be-detected skirt images through contour detection; S2: performing shape fitting on the edge pixel points of the normal skirt and to-be-detected skirt images to determine the basic shape of the edge pixel points of the normal skirt and to-be-detected skirt images; S3: comparing the normal skirt and to-be-detected skirt images, and dividing the to-be-detected skirt into five cases of scratch A, depression B, loosening C, loss D and normal E; S4: setting the scratch A, depression B, loosening C and loss D as negative samples, setting the normal E as a positive sample, constructing a skirt database after sample set classification, dividing the skirt database into a training set T1, a verification set T2 and a test set T3, and the proportion is 7:2:1; S5: constructing a deep convolutional neural network model, comprising an input layer, two convolutional layers, two pooling layers, a full connection layer, a softmax classification layer and an output layer; S6: using the deep convolutional neural network model to perform quality evaluation and classification on the obtained to-be-detected skirt image; S7: evaluating the detected skirt case categories.

2. The method of claim 1, wherein the method is based on deep learning. The specific steps for obtaining the edge pixel points of the normal skirt and to-be-detected skirt images in S1 are as follows: S1.1: performing gray processing on the obtained normal skirt and to-be-detected skirt images to obtain a gray image; S1.2: performing Gaussian filtering and adaptive binarization processing on the gray image to obtain a binary image; S1.3: identifying the edge pixel points of the normal skirt and to-be-detected skirt images from the binary image.

3. The method of claim 1, wherein the method further comprises: The specific comparison steps in S3 are as follows: S3.1: counting the number of edge pixel points of the to-be-detected skirt image, and calculating the coordinates of the edge pixel points of the four corners of the to-be-detected skirt, if the coordinates of the edge pixel points of the four corners of the to-be-detected skirt are not detected, it is judged as loss D, if the coordinates of the edge pixel points of the four corners of the to-be-detected skirt are detected, step S3.2 is entered; S3.2: calculating the length m1 and width n1 of the normal skirt, the length m2 and width n2 of the to-be-detected skirt, the area S1 of the normal skirt and the area S2 of the to-be-detected skirt through the plane coordinate formula, if S1>S2, it is judged as loosening C, if S1=S2, step S3.3 is entered; S3.3: performing erosion processing on the binary image of the to-be-detected skirt after smoothing processing, obtaining the connected region Oi of the binary image after erosion, comparing the connected region Oi of the to-be-detected skirt with the connected region Pi of the normal skirt, if the connected region Oi>the connected region Pi, it is judged as existing scratch A or depression B, step S3.4 is entered, if the connected region Oi=the connected region Pi, it is judged as normal E; S3.4: obtaining the leftmost coordinate (X1, Y1) and the rightmost coordinate (X2, Y2) in the connected region Oi through the union-find set algorithm, calculating the length-width ratio Q of the connected region Oi, comparing the length-width ratio Q of the connected region Oi with the set threshold value α, if the length-width ratio Q of the connected region Oi is less than the threshold value α, it is considered as depression B, otherwise it is considered as scratch A, step S3.5 is entered; S3.5: The scratch A, the dent B, the looseness C, the loss D and the normal E are constructed into a skirtboard database, and the skirtboard database is divided into a training set T1, a verification set T2 and a test set T3, and the ratio is 7:2:

1.

4. The method of claim 3, wherein the method further comprises: The aspect ratio Q of the communication area Oi is calculated by the following formula: 。 5. The method of claim 1, wherein the method is based on deep learning. The step of constructing the deep convolutional neural network model in S5 is: S5.1: Two convolutional layers and two pooling layers are cross-stacked to form a network main structure, and a softmax classification layer is connected to a fully connected layer as a fault feature recognition layer to form a complete deep convolutional neural network model; S5.2: In the convolutional layer, for any input r, the subsequence is: r j = pw j r j-1 Wherein, W and p are convolution operation and nonlinear activation function respectively; S5.3: Wj is the weight of filter mapping, and each layer is written as the convolution sum of the previous layer: ； Wherein, u is an element of r, j=1, 2, 3,..., N, * is convolution operation, k is sample point, and g is activation function; ； Wherein, h is a mapping function, g is an activation function, and f is a frequency; S5.4: The gradient is calculated by the convex optimization algorithm of the stochastic gradient descent method, and the weight Wj is optimized and solved; S5.5: The target area is divided into non-overlapping areas by using the pooling method, the pooling dimension is selected as 3, and the number of neurons is 200; S5.6: The maximum pooling function is used to extract features and improve the calculation efficiency of the deep convolutional neural network model; S5.7: The convolution features and pooling features of the input data are extracted by using the deep convolutional neural network structure model, the network model structure parameters are trained, and the activation function of the Softmax layer uses sigmoid function: ； Wherein, e is the base of natural logarithm; S5.8: The training set T1 is used to train the deep convolutional neural network model.

6. The method of claim 1, wherein the method further comprises: The specific operation steps of S6 are: S6.1: The loss function is cross-entropy loss function, the network is trained by using cross-entropy loss function, and the deep convolutional neural network model is obtained, wherein the cross-entropy loss function is: ； Wherein, β ∈ (A, B, C, D, E), β represents the real probability value of the skirt panel sample, represents the first skirt panel sample prediction probability value; S6.2: The performance of the model is verified by using the samples of the verification set T2, and the parameters are fine-tuned; S6.3: The trained deep convolutional neural network model is used to classify the skirtboard defect image of the test set T3, and the defect classification result is counted.

7. The method of claim 1, wherein the method further comprises: The evaluation expression is: ； wherein, represents an evaluation coefficient, represents a set of positive and negative sample times in the skirtboard image to be detected, is a weight, if is not located within the threshold , the detection is unqualified, which can be reminded by an alarm to the maintenance personnel for maintenance, if is located within the threshold , the detection is qualified. 8.The method of claim 6, wherein the method further comprises: The fine-tuning method of S6.2 is: In the process of training the model, the network weight is updated by using the back propagation algorithm and the stochastic gradient descent method until the loss value converges to obtain the trained deep convolutional neural network model.

Images