Method and device for identifying foreign objects on train skirts based on deep learning
Through edge detection and fusion image processing, combined with an improved deep learning network and an efficient channel attention module, the problem of insufficient accuracy of traditional networks in foreign object detection on EMU skirts is solved, and efficient and accurate foreign object recognition is achieved.
Patent Information
- Application Number
- CN202310782807.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-06-29
- Publication Date
- 2025-09-30
- Estimated Expiration
- 2043-06-29
AI Technical Summary
The traditional Cascade-rcnn deep learning network is not accurate enough in detecting foreign object faults on EMU skirts and cannot meet actual needs.
The edge detection algorithm is used to process the original image, which is then fused with the Canny feature map. An improved deep learning network is used, including a new activation function New_Relu(x) and a new loss function New_Loss=0.5*(SmoothL1_Loss+DIoU). An efficient channel attention module ECA-NET is added after the FPN network.
The accuracy and stability of foreign body detection on EMU skirt panels are improved, the learning ability of the network is enhanced, the detection ability and feature extraction ability of the model are improved, and the accuracy and applicability of the detection results are ensured.
Smart Images
Figure CN116805412B_ABST
Abstract
Description
Technical Field
[0001] The invention belongs to the technical field of motor vehicle fault detection, and in particular relates to the detection of foreign matter on a motor vehicle skirt plate. Background Art
[0002] When detecting foreign object faults on EMU skirt panels, manual inspection of images of the object being tested is often used. This method is not only inefficient but also unstable and inaccurate due to the varying levels of responsibility and capabilities of different operators.
[0003] In recent years, with the gradual development and maturity of deep learning and artificial intelligence technologies, a method using deep learning to identify foreign object faults in EMU skirt panels has emerged, effectively improving detection efficiency and stability. However, due to the cluttered background and numerous similar parts of EMU skirt panel images, the traditional Cascade-rcnn deep learning network for fault detection cannot meet practical requirements. Summary of the Invention
[0004] The present invention aims to solve the problem that the traditional Cascade-rcnn deep learning network cannot meet the actual needs for identifying foreign body faults in EMU skirt panels. A method for identifying foreign body faults in EMU skirt panels based on deep learning is now provided.
[0005] The foreign body recognition method for EMU skirt panels based on deep learning is as follows:
[0006] The original image is processed using an edge detection algorithm to obtain an edge detection feature map, wherein the original image is a grayscale image containing the skirt of the motor vehicle to be tested;
[0007] Fusing the edge detection feature map with the original image to obtain the image under test;
[0008] The image to be tested is input into the trained deep learning network to obtain the detection results and realize the identification of foreign objects in the skirt panel of the tested EMU.
[0009] Furthermore, the method of fusing the edge detection feature map with the original image includes:
[0010] Multiply the pixel values of all corresponding pixels in the edge detection feature map and the original image by 0.5 to obtain the modified edge detection feature map and the original image, respectively. The corresponding pixel points are the pixel points with corresponding positions in the edge detection feature map and the original image.
[0011] The modified edge detection feature map is added to the corresponding pixel points of the original image to obtain the measured image.
[0012] Furthermore, the activation function New_Relu(x) of the above deep learning network is expressed as:
[0013]
[0014] Among them, x is the output result of the previous convolutional layer of the activation function New_Relu(x).
[0015] Furthermore, the regression loss function of the above deep learning network and its RPN network both adopts the New_Loss loss function.
[0016] The expression of the New_Loss loss function is:
[0017] New_Loss=0.5*(SmoothL1_Loss+DIoU),
[0018] in, y is the input of the New_Loss loss function;
[0019] IoU is the intersection over union ratio of the predicted target box selection area and the real target box selection area, b and b gt are the center coordinates of the predicted target box selection area and the true target box selection area, respectively, ρ is the Euclidean distance between the center points of the predicted target box selection area and the true target box selection area, and c is the diagonal length of the minimum closure area.
[0020] Furthermore, when the New_Loss loss function is used as the loss function of the deep learning network, its input y is the predicted position output by each pooling unit of the deep learning network, and the output of the New_Loss loss function is the difference between the predicted target frame area and the true target frame area. The predicted position corresponding to the output with the smallest difference is selected as the detection result obtained by the deep learning network.
[0021] Furthermore, when the New_Loss loss function is used as the regression loss function of the RPN network, its input y is the output of the RPN network, and the output of the New_Loss loss function is used to distinguish the foreground and background of the image under test.
[0022] Furthermore, an efficient channel attention module is set after the FPN network of the above deep learning network.
[0023] The output of the efficient channel attention module is used as the input of the RPN network.
[0024] The deep learning-based foreign body recognition device for train skirts includes the following units:
[0025] Edge detection unit: uses edge detection algorithm to process the original image to obtain edge detection feature map, the original image is a grayscale image containing the tested motor vehicle skirt;
[0026] Fusion unit: fuses the edge detection feature map with the original image to obtain the tested image;
[0027] Foreign object recognition unit: The image to be tested is input into the trained deep learning network to obtain the detection results, thereby realizing the recognition of foreign objects on the skirt panel of the tested EMU.
[0028] Furthermore, in the above fusion unit, the method of fusing the edge detection feature map with the original image includes:
[0029] Multiply the pixel values of all corresponding pixels in the edge detection feature map and the original image by 0.5 to obtain the modified edge detection feature map and the original image, respectively. The corresponding pixel points are the pixel points with corresponding positions in the edge detection feature map and the original image.
[0030] The modified edge detection feature map is added to the corresponding pixel points of the original image to obtain the measured image.
[0031] Furthermore, in the above-mentioned foreign object recognition unit, the activation function New_Relu(x) of the deep learning network is expressed as:
[0032]
[0033] Among them, x is the output result of the previous convolutional layer of the activation function New_Relu(x).
[0034] Furthermore, in the above-mentioned foreign object recognition unit, the regression loss function of the deep learning network and its RPN network both adopts the New_Loss loss function.
[0035] The expression of the New_Loss loss function is:
[0036] New_Loss=0.5*(SmoothL1_Loss+DIoU),
[0037] in, y is the input of the New_Loss loss function;
[0038] IoU is the intersection over union ratio of the predicted target box selection area and the real target box selection area, b and b gt are the center coordinates of the predicted target box selection area and the true target box selection area, respectively, ρ is the Euclidean distance between the center points of the predicted target box selection area and the true target box selection area, and c is the diagonal length of the minimum closure area.
[0039] Furthermore, in the above-mentioned foreign object recognition unit, when the New_Loss loss function is used as the loss function of the deep learning network, its input y is the predicted position output by each pooling unit of the deep learning network, and the output of the New_Loss loss function is the difference between the predicted target frame area and the real target frame area. The predicted position corresponding to the output with the smallest difference is selected as the detection result obtained by the deep learning network.
[0040] Furthermore, in the above-mentioned foreign object recognition unit, when the New_Loss loss function is used as the regression loss function of the RPN network, its input y is the output of the RPN network, and the output of the New_Loss loss function is used to distinguish the foreground and background of the image under test.
[0041] Furthermore, in the above-mentioned foreign object recognition unit, an efficient channel attention module is provided after the FPN network of the deep learning network, and the output of the efficient channel attention module serves as the input of the RPN network.
[0042] A computer-readable storage medium stores a computer program, which, when executed, implements the above-mentioned method for identifying foreign objects on a motor vehicle skirt panel based on deep learning.
[0043] An electronic device includes a storage medium, a processor, and a computer program stored in the storage medium and executable on the processor. The processor executes the computer program to implement the above-mentioned method for identifying foreign objects on a motor vehicle skirt panel based on deep learning.
[0044] The beneficial effects of the present invention are as follows:
[0045] 1. The present invention proposes a new activation function to replace the original ReLU activation function. When the network training gradient is reversely conducted, the parameters can be updated normally, thereby enhancing the learning ability of the network.
[0046] 2. This paper proposes a new loss function that combines the advantages of DIoU Loss and SmoothL1Loss, minimizes the distance between the two target boxes, improves the convergence speed during training and can achieve good convergence, thereby improving the accuracy of the model.
[0047] 3. Fuse the Canny features with the original image to enhance the characteristics of the original image, making the features extracted by the backbone network richer and improving the detection ability of the model.
[0048] 4. The ECA-NET structure is added after FPN to ensure that the attention mechanism obtains more accurate feature maps. BRIEF DESCRIPTION OF THE DRAWINGS
[0049] Figure 1This is a flowchart of the method for identifying foreign objects on EMU skirt panels based on deep learning;
[0050] Figure 2 This is a schematic diagram of the structure of the deep learning network described in the first embodiment;
[0051] Figure 3 is the original image;
[0052] Figure 4 The image under test is the fusion of the original image and the edge detection feature map;
[0053] Figure 5 Schematic diagram of the structure of the efficient channel attention module. DETAILED DESCRIPTION
[0054] The following will be combined with the accompanying drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention. It should be noted that the embodiments of the present invention and the features in the embodiments can be combined with each other in the absence of conflict.
[0055] Specific implementation method 1: refer to Figures 1 to 5 Specifically describing this embodiment, the method for identifying foreign matter on a train skirt panel based on deep learning described in this embodiment is as follows:
[0056] 1. Create a sample dataset
[0057] High-definition equipment is installed around the freight train tracks. High-definition images are captured after the freight train passes through the equipment. These images are clear grayscale images and serve as the original images. Freight train components can be affected by natural or man-made conditions such as rain, mud, oil, and black paint, and images taken at different locations can vary greatly, leading to significant variations in images of foreign objects on EMU skirts. Therefore, when collecting data on foreign objects on EMU skirts, it is important to ensure diversity and capture images of foreign objects under a variety of conditions.
[0058] Foreign objects on EMU skirts can appear in different forms depending on the bogie and vehicle type. However, due to significant differences in frequency of occurrence between different types, collecting images of foreign objects on EMU skirts for less common bogie types and models is difficult. Therefore, all types of foreign objects on EMU skirts are collectively referred to as the "Foreign Objects on EMU Skirts" category.
[0059] Screenshots of foreign objects on EMU skirts include a grayscale image set and a labeled image set. The grayscale image set consists of high-definition grayscale images captured by the device. The labeled image set consists of grayscale images marked with the location of the foreign object on the EMU skirts, acquired through manual labeling. There is a one-to-one correspondence between the grayscale image set and the labeled image set; each grayscale image corresponds to a labeled image.
[0060] Although the sample dataset includes images under various conditions, data augmentation is still required to improve the stability of the algorithm. Augmentation includes operations such as image translation, scaling, and brightness adjustment. Each operation is performed under random conditions to maximize sample diversity and applicability.
[0061] This embodiment uses the edge detection feature map (Canny feature map) to fuse with the original image to process the training data set. First, the original image is processed using the edge detection algorithm to obtain the corresponding Canny feature map. Then the original image is fused with the corresponding Canny feature map. The original image is as follows Figure 3 As shown, the image after the original image and the Canny feature map are fused is as follows Figure 4 As shown in the figure, after comparison, it can be seen that the edge of the foreign body in the fused image becomes more obvious, and the characteristics of the foreign body are highlighted, which enables the convolutional neural network to better extract the characteristics of the foreign body on the train skirt.
[0062] 2. Build a deep learning network
[0063] Figure 2 The figure shows the improved Cascade-rcnn network structure of this embodiment. This network is an improvement based on the traditional Faster-rcnn-resnet50 network. Its feature extraction network uses the Resnet50 convolution module, where Conv1 to Conv5 (convolution layers 1 to 5) represent the Resnet50 convolution module. Except for the activation function, the parameter settings of the feature extraction network, such as the number of network convolution layers and the number of convolution kernels per convolution layer, are consistent with the feature extraction network Resnet50 of Cascade-rcnnn-resnet50.
[0064] The activation function of the traditional Faster-rcnn-resnet50 network uses the Relu activation function, which is expressed as:
[0065]
[0066] When the input of the Relu activation function is negative, its output is zero and its derivative is also zero, which causes the parameters to be unable to be updated during the reverse conduction of the network training gradient, reducing the learning ability of the network.
[0067] The activation function New_Relu(x) of the deep learning network in this embodiment is expressed as:
[0068]
[0069] Among them, x is the output result of the previous convolutional layer of the activation function New_Relu(x).
[0070] When the input parameter of the activation function New_Relu(x) is negative, the output of the network is not zero, its derivative is not zero, and the parameters can be updated normally when the network training gradient is reversely conducted, thereby improving the learning ability of the network.
[0071] In addition, this implementation uses the New_Loss loss function instead of the original SmoothL1_Loss. When used to calculate the Bounding Box Loss (bounding box regression loss function) for object detection, the original SmoothL1_Loss is less sensitive to outliers (points far from the center) and outliers than other loss functions, and can control the magnitude of the gradient to prevent errors during training.
[0072] In this implementation, the New_Loss loss function expression is:
[0073] New_Loss=0.5*(SmoothL1_Loss+DIoU),
[0074] in, y is the input of the New_Loss loss function;
[0075] IoU is the intersection over union ratio of the predicted target box selection area and the real target box selection area, b and b gt are the center coordinates of the predicted target box selection area and the true target box selection area, respectively, ρ is the Euclidean distance between the center points of the predicted target box selection area and the true target box selection area, and c is the diagonal length of the minimum closure area.
[0076] DIoU can provide the optimization direction of the bounding box when the predicted target box selection area and the real target box selection area do not intersect; and DIoU minimizes the distance between the two target boxes, improves the convergence speed during training and can achieve good convergence.
[0077] The New_Loss loss function is used in two places. First, in the RPN network structure, when the New_Loss loss function is used as the regression loss function of the RPN network, its input y is the output of the RPN network, and the output of the New_Loss loss function is used to distinguish the foreground and background of the image being tested. Second, when the candidate box is finally positioned, when the New_Loss loss function is used as the loss function of the deep learning network, its input y is the predicted position output by each pooling unit of the deep learning network, and the output of the New_Loss loss function is the difference between the predicted target box area and the actual target box area. The predicted position corresponding to the output with the smallest difference is selected as the detection result obtained by the deep learning network.
[0078] This embodiment also adds an efficient channel attention module (ECA-NET) after the FPN network, as shown in Figure 5 The main idea of this module is to prevent dimensionality reduction after image processing. Specifically, the local cross-channel interaction strategy autonomously selects a single-kernel convolution kernel during image processing to ensure that the image does not deform during processing and that the attention mechanism obtains more accurate feature maps. This effectively solves the problem of loss of detailed information caused by dimensionality changes in feature maps.
[0079] 3. Training Deep Learning Network
[0080] The sample data set is used to train the improved deep learning network to obtain the improved trained deep learning network.
[0081] 4. Actual testing
[0082] When the train under test passes through the high-definition imaging equipment installed around the railway, an image of the train passing by is captured, and an image of the train skirt is captured as the original image. The original image is a grayscale image containing the train skirt under test.
[0083] The original image is processed using the edge detection algorithm to obtain the edge detection feature map.
[0084] The pixel values of all corresponding pixel points in the edge detection feature map and the original image are multiplied by 0.5 to obtain the modified edge detection feature map and the original image respectively. The corresponding pixel points are the pixel points whose positions in the edge detection feature map and the original image correspond to each other. The modified edge detection feature map and the corresponding pixel points in the original image are added to obtain the measured image.
[0085] The image to be tested is input into the trained deep learning network to obtain the detection result. If a foreign object is detected on the EMU skirt, the image is considered a fault image and a fault message is uploaded; otherwise, the next image is tested.
[0086] Specific embodiment 2: The motor vehicle skirt foreign body identification device based on deep learning described in this embodiment includes the following units:
[0087] Edge detection unit: uses edge detection algorithm to process the original image to obtain edge detection feature map, the original image is a grayscale image containing the tested motor vehicle skirt;
[0088] Fusion unit: fuses the edge detection feature map with the original image to obtain the tested image;
[0089] Foreign object recognition unit: The image to be tested is input into the trained deep learning network to obtain the detection results, thereby realizing the recognition of foreign objects on the skirt panel of the tested EMU.
[0090] Specific embodiment three: This embodiment describes a computer-readable storage medium, wherein the storage medium stores a computer program, and when the computer program is executed, the method described in specific embodiment one is implemented.
[0091] Specific embodiment 4: The electronic device described in this embodiment includes a storage medium, a processor, and a computer program stored in the storage medium and executable on the processor. The processor executes the computer program to implement the method described in specific embodiment 1.
[0092] Although the present invention is described herein with reference to specific embodiments, it should be understood that these embodiments are merely illustrative of the principles and applications of the invention. It should be understood that many modifications may be made to the illustrative embodiments, and that other arrangements may be devised, without departing from the spirit and scope of the invention as defined by the appended claims. It should be understood that the various dependent claims and features described herein may be combined in ways other than those described in the original claims. It should also be understood that features described in conjunction with individual embodiments may be used in conjunction with other described embodiments.
Claims
1. A method for identifying foreign objects on train skirts based on deep learning, characterized in that: Specifically: The original image is processed using an edge detection algorithm to obtain an edge detection feature map, wherein the original image is a grayscale image containing the skirt of the motor vehicle to be tested; Fusing the edge detection feature map with the original image to obtain the image under test; The image to be tested is input into the trained deep learning network to obtain the test results and realize the identification of foreign objects on the skirt of the tested EMU; The method for fusing the edge detection feature map with the original image comprises: Multiply the pixel values of all corresponding pixels in the edge detection feature map and the original image by 0.5 to obtain the modified edge detection feature map and the original image, respectively. The corresponding pixel points are the pixel points with corresponding positions in the edge detection feature map and the original image. Add the modified edge detection feature map to the corresponding pixel points of the original image to obtain the tested image; The regression loss function of the deep learning network and its RPN network are both loss function, described The expression of the loss function is: ， in, , for Input to the loss function; , is the intersection-over-union ratio of the predicted target frame area and the true target frame area, and are the center point coordinates of the predicted target frame area and the real target frame area, respectively. The Euclidean distance between the center point of the predicted target frame area and the true target frame area, is the diagonal length of the minimum enclosed region.
2. The method for identifying foreign matter on a train skirt based on deep learning according to claim 1 is characterized in that: The activation function of the deep learning network The expression is: ， in, is the activation function The output of the previous convolutional layer.
3. The method for identifying foreign matter on a train skirt based on deep learning according to claim 1 is characterized in that: when When the loss function is used as the loss function of the deep learning network, its input is the predicted position output by each pooling unit of the deep learning network, The output of the loss function is the difference between the predicted target frame area and the true target frame area. The predicted position corresponding to the output with the smallest difference is selected as the detection result obtained by the deep learning network.
4. The method for identifying foreign matter on a train skirt based on deep learning according to claim 1 is characterized in that: when When the loss function is used as the regression loss function of the RPN network, its input is the output of the RPN network, The output of the loss function is used to distinguish the foreground and background of the image under test.
5. The method for identifying foreign matter on a train skirt based on deep learning according to claim 1 is characterized in that: The FPN network of the deep learning network is followed by an efficient channel attention module. The output of the efficient channel attention module is used as the input of the RPN network.
6. The foreign body identification device for train skirts based on deep learning is characterized by: The following units are included: Edge detection unit: uses edge detection algorithm to process the original image to obtain edge detection feature map, the original image is a grayscale image containing the tested motor vehicle skirt; Fusion unit: fuses the edge detection feature map with the original image to obtain the tested image; Foreign object recognition unit: Input the tested image into the trained deep learning network to obtain the detection results, thereby realizing the recognition of foreign objects on the tested EMU skirt; The fusing of the edge detection feature map with the original image includes: multiplying the pixel values of all corresponding pixel points in the edge detection feature map and the original image by 0.5 to obtain a modified edge detection feature map and the original image, respectively, wherein the corresponding pixel points are pixel points in the edge detection feature map and the original image that have corresponding positions, and adding the modified edge detection feature map and the corresponding pixel points in the original image to obtain the measured image; The regression loss function of the deep learning network and its RPN network are both loss function, described The expression of the loss function is: ， in, , for Input to the loss function; , is the intersection-over-union ratio of the predicted target frame area and the true target frame area, and are the center point coordinates of the predicted target frame area and the real target frame area, respectively. The Euclidean distance between the center point of the predicted target frame area and the true target frame area, is the diagonal length of the minimum enclosed region.
7. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed, the method according to any one of claims 1 to 5 is implemented.
8. An electronic device comprising a storage medium, a processor, and a computer program stored in the storage medium and executable on the processor, wherein: The processor executes the computer program to implement the method according to any one of claims 1 to 5.