Wagon bottom foreign matter lightweight detection method based on deep learning, storage medium and equipment

Abstract

The invention discloses a rail wagon bottom foreign matter lightweight detection method based on deep learning, a storage medium and equipment, and belongs to the technical field of rail wagon bottom foreign matter detection. The problems that a traditional detection model is high in computing resource requirement, low in detection accuracy and incapable of conducting real-time detection are solved. According to the method, the MCA attention mechanism is combined on the basis of YOLOv8, the perception ability of the model to the global context is enhanced, and more geometric characteristics are considered by designing a loss function, so that the accuracy and robustness of detection of foreign matters with different shapes are improved. And the model is pruned, so that the foreign matter detection efficiency can be improved, and the occupation of computing resources can be reduced. The method can be applied to railway wagon bottom foreign matter detection.

Description

Technical Field

[0001] The present invention belongs to the technical field of foreign object detection at the bottom of railway freight cars, and specifically relates to a lightweight foreign object detection method, storage medium and equipment at the bottom of railway freight cars based on deep learning. Background Art

[0002] With the rapid development of my country's high-speed railways, the operation and maintenance of trains have gradually become a key issue of concern, and the detection of foreign objects on the bottom of railway freight cars is particularly important. These foreign objects may cause safety hazards, equipment damage and operational delays, seriously affecting transportation efficiency and safety. Although existing fault detection systems (such as TFDS) provide a large amount of valuable data for train inspection research, they still rely on manual foreign object judgment, resulting in slow and inefficient foreign object detection. This detection mode is difficult to meet the requirements of modern railways for safety and efficiency, so the automated detection of foreign objects on the bottom of railway freight cars is of great significance.

[0003] However, deploying existing detection models often faces difficulties, especially in field environments with limited computing resources. Traditional detection models are often bulky, resulting in a large amount of computation for the model and cannot be effectively run in real-time monitoring. Moreover, bloated detection models increase the demand for computing resources, while the detection accuracy of traditional detection models is still relatively limited. Therefore, in order to solve the problems of high computing resource requirements, low detection accuracy and inability to perform real-time detection in traditional detection models, it is necessary to propose a lightweight detection model that is easy to deploy on various devices to achieve fast and accurate foreign body detection. Summary of the invention

[0004] The purpose of the present invention is to solve the problems of high computing resource requirements, low detection accuracy and inability to perform real-time detection in traditional detection models, and to propose a lightweight detection method, storage medium and device for foreign objects at the bottom of railway freight cars based on deep learning.

[0005] The technical solution adopted by the present invention to solve the above technical problems is:

[0006] A method for detecting lightweight foreign objects at the bottom of a railway freight car based on deep learning, the method comprising the following steps:

[0007] Step 1: Obtain a dataset of foreign object images on the bottom of railway freight cars, mark the foreign objects in the images on the bottom of railway freight cars, and obtain a training set of foreign object images on the bottom of railway freight cars;

[0008] Step 2: Build a foreign object detection network and train the foreign object detection network using a training set of foreign object images at the bottom of a railway freight car;

[0009] During the training of the foreign object detection network, the working process of the foreign object detection network is:

[0010] In the foreign object detection network, the input image first passes through the first CBS module, and then the output of the first CBS module is used as the input of the second CBS module;

[0011] The output of the second CBS module is used as the input of the first C2f module, and the output of the first C2f module is used as the input of the third CBS module;

[0012] The output of the third CBS module is used as the input of the second C2f module, and the output of the second C2f module is used as the input of the fourth CBS module;

[0013] The output of the fourth CBS module is used as the input of the third C2f module, and the output of the third C2f module is used as the input of the fifth CBS module;

[0014] The output of the fifth CBS module is used as the input of the fourth C2f module, and the output of the fourth C2f module is used as the input of the SPPF module;

[0015] The output of the SPPF module is used as the input of the first upsampling module, and the output of the first upsampling module and the output of the third C2f module are concatenated, and the concatenated result is used as the input of the fifth C2f module;

[0016] The output of the fifth C2f module is used as the input of the second upsampling module, and the output of the second upsampling module and the output of the second C2f module are concatenated, and the concatenated result is used as the input of the sixth C2f module;

[0017] Pass the output of the sixth C2f module through the first MCA attention module, use the output of the first channel of the first MCA attention module as the input of the sixth CBS module, and use the output of the second channel of the first MCA attention module as the input of the seventh CBS module;

[0018] The output of the sixth C2f module is used as the input of the eighth CBS module, the output of the fifth C2f module is spliced ​​with the output of the eighth CBS module, and the spliced ​​result is used as the input of the seventh C2f module;

[0019] Pass the output of the seventh C2f module through the second MCA attention module, use the output of the first channel of the second MCA attention module as the input of the ninth CBS module, and use the output of the second channel of the second MCA attention module as the input of the tenth CBS module;

[0020] The output of the seventh C2f module is used as the input of the eleventh CBS module. After splicing the output of the eleventh CBS module and the output of the SPPF module, the splicing result passes through the eighth C2f module, and then the output of the eighth C2f module passes through the third MCA attention module. The output of the first channel of the third MCA attention module is used as the input of the twelfth CBS module, and the output of the second channel of the third MCA attention module is used as the input of the thirteenth CBS module;

[0021] The output of the sixth CBS module passes through the first convolutional layer, and the predicted box of the feature map with a size of W1×H1×C1 is output through the first convolutional layer; where W1 is the width of the feature map, H1 is the height of the feature map, and C1 is the number of channels of the feature map;

[0022] The output of the seventh CBS module passes through the second convolutional layer, and the second convolutional layer outputs the predicted category of the feature map with a size of W1×H1×C1;

[0023] The output of the ninth CBS module passes through the third convolutional layer, and the third convolutional layer outputs the prediction box of the feature map with a size of W2×H2×C2;

[0024] The output of the tenth CBS module passes through the fourth convolutional layer, and the fourth convolutional layer outputs the predicted category of the feature map with a size of W2×H2×C2;

[0025] The output of the twelfth CBS module passes through the fifth convolutional layer, and the fifth convolutional layer outputs the prediction box of the feature map with a size of W3×H3×C3;

[0026] The output of the thirteenth CBS module passes through the sixth convolutional layer, and the sixth convolutional layer outputs the predicted category of the feature map with a size of W3×H3×C3;

[0027] The loss function L is calculated according to the outputs of the first convolutional layer to the sixth convolutional layer, and the training is stopped when the loss function converges to obtain a trained foreign body detection network;

[0028] Step 3: Prune the trained foreign body detection network to obtain a pruned foreign body detection network;

[0029] Step 4: Use the pruned foreign object detection network to identify the bottom image of the railway freight car to be inspected, and obtain the foreign object detection result of the bottom image of the railway freight car to be inspected.

[0030] Furthermore, the specific process of obtaining the foreign body image dataset at the bottom of railway freight cars is as follows:

[0031] The constructed imaging equipment is used to photograph the bottom of a passing railway freight car to obtain a railway freight car bottom image dataset, and then all railway freight car bottom foreign object images are selected from the obtained railway freight car bottom image dataset, and the railway freight car bottom foreign object image dataset is formed using all the selected railway freight car bottom foreign object images.

[0032] Furthermore, the foreign objects in the image of the bottom of the railway freight car are marked, specifically:

[0033] Mark the minimum bounding rectangle of the foreign object in each railway truck bottom image.

[0034] Furthermore, before using the railway freight car bottom foreign object image training set to train the foreign object detection network, the railway freight car bottom foreign object image training set needs to be amplified, and then the amplified railway freight car bottom foreign object image training set is used to train the foreign object detection network.

[0035] Furthermore, the first CBS module includes a convolutional layer, a BN layer and a SILU activation function layer; and the working process of the first CBS module is:

[0036] In the first CBS module, the input image first passes through the convolution layer, and then the output of the convolution layer is used as the input of the BN layer;

[0037] Use the output of the BN layer as the input of the SILU activation function layer;

[0038] The output of the SILU activation function layer is used as the output of the first CBS module.

[0039] Furthermore, the first C2f module includes a fourteenth CBS module, a fifteenth CBS module, and a first Bottleneck unit to an nth Bottleneck unit; wherein:

[0040] First, the input of the first C2f module is used as the input of the fourteenth CBS module, and then the output of the fourteenth CBS module is divided into two parts, wherein one part of the output passes through the first Bottleneck unit, the output of the first Bottleneck unit passes through the second Bottleneck unit, and the output of the second Bottleneck unit passes through the third Bottleneck unit;

[0041] And so on, the output of the n-1th Bottleneck unit is used as the input of the nth Bottleneck unit;

[0042] Then, another part of the output of the fourteenth CBS module is spliced ​​with the outputs of the first Bottleneck unit to the nth Bottleneck unit;

[0043] Finally, the splicing result is used as the input of the fifteenth CBS module, and the output of the fifteenth CBS module is used as the output of the first C2f module;

[0044] The first Bottleneck unit includes a sixteenth CBS module and a seventeenth CBS module. The input of the first Bottleneck unit is used as the input of the sixteenth CBS module, and the output of the sixteenth CBS module is used as the input of the seventeenth CBS module. The input of the first Bottleneck unit and the output of the seventeenth CBS module are added, and the addition result is used as the output of the first Bottleneck unit.

[0045] Furthermore, the working process of the first MCA attention module is:

[0046] In the first MCA attention module, the input F of the first MCA attention module is first used as the input of the first channel and the second channel in parallel;

[0047] In the first channel, the input F first passes through the LN layer, and then the output of the LN layer passes through three parallel convolutional layers, and the outputs of the three parallel convolutional layers are spliced, and the spliced ​​result is used as the input of the convolutional layer to obtain the output F of the convolutional layer. x ;

[0048] Then for F x Perform linear transformation to generate value vector V1, key vector K1 and query vector Q1;

[0049] In the second channel, the input F first passes through the LN layer, and then the output of the LN layer passes through three parallel convolutional layers, and the outputs of the three parallel convolutional layers are spliced, and the spliced ​​result is used as the input of the convolutional layer to obtain the output F of the convolutional layer. y ;

[0050] Then for F y Perform linear transformation to generate value vector V2, key vector K2 and query vector Q2;

[0051] Pass the value vector V1, key vector K1 and query vector Q2 through the Reshape layer respectively, multiply the output of the key vector K1 through the Reshape layer and the output of the query vector Q2 through the Reshape layer to obtain the multiplication result a; then multiply a with the output of the value vector V1 through the Reshape layer to obtain the multiplication result b;

[0052] Then the multiplication result b passes through the Reshape layer and the convolution layer in turn to obtain the output F of the first channel T ;

[0053] Pass the value vector V2, key vector K2 and query vector Q1 through the Reshape layer respectively, multiply the output of the key vector K2 through the Reshape layer and the output of the query vector Q1 through the Reshape layer to obtain the multiplication result c; then multiply c with the output of the value vector V2 through the Reshape layer to obtain the multiplication result d;

[0054] Then the multiplication result d is passed through the Reshape layer and the convolution layer in turn to obtain the output F of the second channel B .

[0055] Furthermore, the loss function L is calculated as:

[0056]

[0057] Where λ represents the balancing factor, Γ represents the set of all BN layers in the foreign object detection network, γ is the scaling factor added by the BN layer, and L SIoU is the SIoU loss;

[0058] After adding the scaling factor, the working process of the BN layer is:

[0059]

[0060] In the formula, z in Represents the input of the BN layer with the scaling factor γ added; represents an intermediate variable; z out represents the output of the BN layer with the scaling factor γ added; β represents the offset coefficient of the BN layer; ε represents the offset; μ B It represents the mean of all activation values ​​of the current BN layer after the training samples of the current batch pass through the current BN layer; It shows the standard deviation of all activation values ​​of the current BN layer after the training samples of the current batch pass through the current BN layer.

[0061] A computer storage medium stores at least one instruction, and the at least one instruction is loaded and executed by a processor to implement a method for lightweight detection of foreign objects at the bottom of a railway freight car based on deep learning.

[0062] A railway freight car bottom foreign body lightweight detection device based on deep learning, the device comprises a processor and a memory, the memory stores at least one instruction, the at least one instruction is loaded and executed by the processor to implement the railway freight car bottom foreign body lightweight detection method based on deep learning.

[0063] The beneficial effects of the present invention are:

[0064] 1. Automatic identification technology is introduced to realize automatic identification and alarm of foreign objects at the bottom of railway freight cars. Manual personnel only need to confirm the alarm results, which effectively saves labor costs and improves operation quality and efficiency.

[0065] 2. Applying the YOLOv8 algorithm to the detection of foreign objects at the bottom of railway freight cars has higher accuracy and real-time performance than traditional machine vision detection methods.

[0066] 3. Combining the MCA attention mechanism based on YOLOv8 enhances the model's ability to perceive the global context, thereby more accurately detecting foreign objects.

[0067] 4. The SIoU loss function is used to replace CIoU. The SIoU loss function considers more geometric characteristics, such as the shape loss of foreign objects, thereby improving the accuracy and robustness of foreign object detection of various shapes.

[0068] 5. Improve detection efficiency and reduce computing resource usage through model pruning to facilitate subsequent deployment. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] Figure 1 It is a flow chart of a method for lightweight detection of foreign matter at the bottom of a railway freight car based on deep learning of the present invention;

[0070] Figure 2 It is a schematic diagram of the structure of the MCA attention module;

[0071] Figure 3 is a schematic diagram of the structure of the foreign body detection network of the present invention;

[0072] Figure 4 It is a schematic diagram of the loss function calculation process. DETAILED DESCRIPTION

[0073] It should be particularly noted that, in the absence of conflict, the various embodiments disclosed in this application can be combined with each other.

[0074] Specific implementation method 1: Combination Figure 1 and Figure 3 This embodiment is a method for detecting lightweight foreign matter at the bottom of a railway freight car based on deep learning. The specific process of the method is as follows:

[0075] Step 1: Obtain a dataset of foreign object images on the bottom of railway freight cars, mark the foreign objects in the images on the bottom of railway freight cars, and obtain a training set of foreign object images on the bottom of railway freight cars;

[0076] Step 2: Build a foreign object detection network and train the foreign object detection network using a training set of foreign object images at the bottom of a railway freight car;

[0077] During the training of the foreign object detection network, the working process of the foreign object detection network is:

[0078] In the foreign object detection network, the input image first passes through the first CBS module, and then the output of the first CBS module is used as the input of the second CBS module;

[0079] The output of the second CBS module is used as the input of the first C2f module, and the output of the first C2f module is used as the input of the third CBS module;

[0080] The output of the third CBS module is used as the input of the second C2f module, and the output of the second C2f module is used as the input of the fourth CBS module;

[0081] The output of the fourth CBS module is used as the input of the third C2f module, and the output of the third C2f module is used as the input of the fifth CBS module;

[0082] The output of the fifth CBS module is used as the input of the fourth C2f module, and the output of the fourth C2f module is used as the input of the SPPF module;

[0083] The output of the SPPF module is used as the input of the first upsampling module, and the output of the first upsampling module and the output of the third C2f module are concatenated, and the concatenated result is used as the input of the fifth C2f module;

[0084] The output of the fifth C2f module is used as the input of the second upsampling module, and the output of the second upsampling module and the output of the second C2f module are concatenated, and the concatenated result is used as the input of the sixth C2f module;

[0085] Pass the output of the sixth C2f module through the first MCA attention module, use the output of the first channel of the first MCA attention module as the input of the sixth CBS module, and use the output of the second channel of the first MCA attention module as the input of the seventh CBS module;

[0086] The output of the sixth C2f module is used as the input of the eighth CBS module, the output of the fifth C2f module is spliced ​​with the output of the eighth CBS module, and the spliced ​​result is used as the input of the seventh C2f module;

[0087] Pass the output of the seventh C2f module through the second MCA attention module, use the output of the first channel of the second MCA attention module as the input of the ninth CBS module, and use the output of the second channel of the second MCA attention module as the input of the tenth CBS module;

[0088] The output of the seventh C2f module is used as the input of the eleventh CBS module. After splicing the output of the eleventh CBS module and the output of the SPPF module, the splicing result passes through the eighth C2f module, and then the output of the eighth C2f module passes through the third MCA attention module. The output of the first channel of the third MCA attention module is used as the input of the twelfth CBS module, and the output of the second channel of the third MCA attention module is used as the input of the thirteenth CBS module;

[0089] The output of the sixth CBS module passes through the first convolutional layer, and the prediction box of the feature map with a size of W1×H1×C1 (80×80×256) is output through the first convolutional layer; where W1 is the width of the feature map, H1 is the height of the feature map, and C1 is the number of channels of the feature map;

[0090] The output of the seventh CBS module passes through the second convolutional layer, and the second convolutional layer outputs the predicted category of the feature map with a size of W1×H1×C1 (80×80×256);

[0091] The output of the ninth CBS module passes through the third convolutional layer, and the third convolutional layer outputs the prediction box of the feature map with a size of W2×H2×C2 (40×40×512);

[0092] The output of the tenth CBS module passes through the fourth convolutional layer, and the fourth convolutional layer outputs the predicted category of the feature map with a size of W2×H2×C2 (40×40×512);

[0093] The output of the twelfth CBS module passes through the fifth convolutional layer, and the fifth convolutional layer outputs the prediction box of the feature map with a size of W3×H3×C3 (20×20×512);

[0094] The output of the thirteenth CBS module passes through the sixth convolutional layer, and the sixth convolutional layer outputs the predicted category of the feature map with a size of W3×H3×C3 (20×20×512);

[0095] The first convolutional layer, the third convolutional layer, and the fifth convolutional layer are used to output prediction boxes of feature maps of different sizes, and the second convolutional layer, the fourth convolutional layer, and the sixth convolutional layer are used to output prediction categories of feature maps of different sizes; the loss function L is calculated according to the outputs of the first to sixth convolutional layers, and the training is stopped when the loss function converges to obtain a trained foreign body detection network;

[0096] Step 3: Prune the trained foreign body detection network according to the scaling coefficient of each BN layer when the loss function converges (i.e., compare the scaling coefficient of each BN layer with the set threshold, remove the BN layer with a scaling coefficient less than the threshold, and the threshold can be set according to the actual situation), and obtain the pruned foreign body detection network;

[0097] Step 4: Use the pruned foreign object detection network to identify the bottom image of the railway freight car to be inspected, and obtain the foreign object detection result of the bottom image of the railway freight car to be inspected.

[0098] Specific implementation method 2: This implementation method is different from the specific implementation method 1 in that the specific process of obtaining the foreign body image data set at the bottom of the railway freight car is as follows:

[0099] The bottom of the passing railway freight cars is photographed using the built high-definition imaging equipment to obtain a railway freight car bottom image dataset, and then all railway freight car bottom foreign object images are selected from the obtained railway freight car bottom image dataset, and the railway freight car bottom foreign object image dataset is formed using all the selected railway freight car bottom foreign object images.

[0100] The other steps and parameters are the same as those in the first embodiment.

[0101] The present invention uses a high-definition device built at the bottom of the freight car track to capture images of the bottom of a railway freight car passing at high speed. It adopts a line scanning method, which can achieve seamless image splicing and generate a two-dimensional railway freight car bottom image with a large field of view and high precision. Since the parts of railway freight cars may be affected by natural conditions or artificial conditions such as rain, mud stains, oil stains, black paint, etc., and there may be differences in the images taken at different stations. Therefore, in the process of collecting image data sets, it is necessary to ensure diversity and try to collect images under various conditions and different stations.

[0102] Specific implementation method three: This implementation method is different from specific implementation method one or two in that: marking foreign objects in the bottom image of the railway freight car is performed as follows:

[0103] Mark the minimum bounding rectangle of the foreign object in each railway truck bottom image.

[0104] The other steps and parameters are the same as those in the first or second embodiment.

[0105] The marking method that can be used in this embodiment includes but is not limited to manual marking, and the foreign body targets in the bottom image of the railway freight car are marked, and each foreign body target corresponds to a marking data.

[0106] Specific implementation method four: This implementation method is different from any one of specific implementation methods one to three in that before using the training set of foreign object images on the bottom of railway freight cars to train the foreign object detection network, it is necessary to amplify the training set of foreign object images on the bottom of railway freight cars, and then use the amplified training set of foreign object images on the bottom of railway freight cars to train the foreign object detection network.

[0107] The other steps and parameters are the same as those in Specific Embodiments 1 to 3.

[0108] Although the training set includes images under various conditions, data augmentation is still needed to improve the stability of the algorithm. Use horizontal and vertical flipping to enhance the diversity of images; adjust image brightness and contrast to reduce the impact of uneven lighting and highlight the edge features of defects; use histogram equalization to enhance the contrast of images and improve the generalization ability of the model; use random noise to simulate working conditions in rainy and snowy weather.

[0109] Specific implementation method 5: This implementation method is different from any one of the specific implementation methods 1 to 4 in that: the first CBS module includes a convolution layer, a BN layer and a SILU activation function layer; and the working process of the first CBS module is:

[0110] In the first CBS module, the input image first passes through the convolution layer, and then the output of the convolution layer is used as the input of the BN layer;

[0111] Use the output of the BN layer as the input of the SILU activation function layer;

[0112] The output of the SILU activation function layer is used as the output of the first CBS module.

[0113] The other steps and parameters are the same as those in Specific Embodiments 1 to 4.

[0114] Specific implementation method 6: This implementation method is different from any one of the specific implementation methods 1 to 5 in that: the first C2f module includes the fourteenth CBS module, the fifteenth CBS module, the first Bottleneck unit to the nth Bottleneck unit; wherein:

[0115] First, the input of the first C2f module is used as the input of the fourteenth CBS module, and then the output of the fourteenth CBS module is divided into two parts, wherein one part of the output passes through the first Bottleneck unit, the output of the first Bottleneck unit passes through the second Bottleneck unit, and the output of the second Bottleneck unit passes through the third Bottleneck unit;

[0116] And so on, the output of the n-1th Bottleneck unit is used as the input of the nth Bottleneck unit;

[0117] Then, another part of the output of the fourteenth CBS module is spliced ​​with the outputs of the first Bottleneck unit to the nth Bottleneck unit;

[0118] Finally, the splicing result is used as the input of the fifteenth CBS module, and the output of the fifteenth CBS module is used as the output of the first C2f module;

[0119] The first Bottleneck unit includes a sixteenth CBS module and a seventeenth CBS module. The input of the first Bottleneck unit is used as the input of the sixteenth CBS module, and the output of the sixteenth CBS module is used as the input of the seventeenth CBS module. The input of the first Bottleneck unit and the output of the seventeenth CBS module are added, and the addition result is used as the output of the first Bottleneck unit.

[0120] The other steps and parameters are the same as those in Specific Implementation Methods 1 to 5.

[0121] In the present invention, the structure and working process of each CBS module are the same as those of the first CBS module; the structure and working process of each C2f module are the same as those of the first C2f module, and the structure and working process of the second Bottleneck unit to the nth Bottleneck unit are the same as those of the first Bottleneck unit.

[0122] Specific implementation method seven: Combination Figure 2 The present embodiment is different from the first to sixth embodiments in that the working process of the first MCA attention module is:

[0123] In the first MCA attention module, the input F of the first MCA attention module is first used as the input of the first channel and the second channel in parallel;

[0124] In the first channel, the input F first passes through the LN (Layer Norm) layer, and then the output of the LN layer passes through three parallel convolutional layers, and the outputs of the three parallel convolutional layers are spliced, and the spliced ​​result is used as the input of the convolutional layer to obtain the output F of the convolutional layer. x ;

[0125] Then for F x Perform linear transformation to generate value vector V1, key vector K1 and query vector Q1;

[0126] In the second channel, the input F first passes through the LN (Layer Norm) layer, and then the output of the LN layer passes through three parallel convolutional layers, and the outputs of the three parallel convolutional layers are spliced, and the spliced ​​result is used as the input of the convolutional layer to obtain the output F of the convolutional layer. y ;

[0127] Then for F y Perform linear transformation to generate value vector V2, key vector K2 and query vector Q2;

[0128] Pass the value vector V1, key vector K1 and query vector Q2 through the Reshape layer respectively, multiply the output of the key vector K1 through the Reshape layer and the output of the query vector Q2 through the Reshape layer to obtain the multiplication result a; then multiply a with the output of the value vector V1 through the Reshape layer to obtain the multiplication result b;

[0129] Then the multiplication result b passes through the Reshape layer and the convolution layer in turn to obtain the output F of the first channel T ;

[0130] Pass the value vector V2, key vector K2 and query vector Q1 through the Reshape layer respectively, multiply the output of the key vector K2 through the Reshape layer and the output of the query vector Q1 through the Reshape layer to obtain the multiplication result c; then multiply c with the output of the value vector V2 through the Reshape layer to obtain the multiplication result d;

[0131] Then the multiplication result d is passed through the Reshape layer and the convolution layer in turn to obtain the output F of the second channel B .

[0132] The other steps and parameters are the same as those in Specific Embodiments 1 to 6.

[0133] In view of the problem that there are many types and styles of foreign objects to be detected at the bottom of railway freight cars, the MCA attention module improves the traditional axial attention mechanism by introducing multi-scale features using strip convolution to better locate the foreign object target area. At the same time, a double cross attention between the two spatial axial attentions is established to better utilize multi-scale features and identify fuzzy boundaries. MCA can effectively encode the global context while taking into account the various sizes and shapes of foreign objects. By integrating the attention of multiple scales, the model's perception of the global context is enhanced, thereby more accurately identifying foreign objects. In addition, the number of small model parameters of the MCA method is only 0.14M, which is flexible and lightweight enough.

[0134] Specific implementation method eight: Combination Figure 4 The present embodiment is described below. The present embodiment is different from the first to seventh embodiments in that the loss function L is calculated as follows:

[0135]

[0136] Where λ represents the balancing factor, Γ represents the set of all BN layers in the foreign object detection network, γ is the scaling factor added by the BN layer, and L SIoU is the SIoU loss;

[0137] After adding the scaling factor, the working process of the BN layer is:

[0138]

[0139] In the formula, z in Represents the input of the BN layer with the scaling factor γ added; represents an intermediate variable; z out represents the output of the BN layer with the scaling factor γ added; β represents the offset coefficient of the BN layer; ε represents the offset; μ B It represents the mean of all activation values ​​of the current BN layer after the training samples of the current batch pass through the current BN layer; It shows the standard deviation of all activation values ​​of the current BN layer after the training samples of the current batch pass through the current BN layer.

[0140] The other steps and parameters are the same as those in Specific Embodiments 1 to 7.

[0141] There are a large number of redundant parameters in the foreign object detection network, and their existence has little effect on the accuracy of the network. In order to reduce the number of model parameters and enable the model to effectively complete the real-time detection of foreign objects at the bottom of the truck, the network model of the present invention is subjected to channel pruning to implement model compression: first, the scale factor γ introduced in the BN layer is used to perform L1 regularization training on the model, so that the scale factor γ of the redundant channel tends to 0 during training, and the importance of the channel in the convolutional layer is judged according to the sorting result of γ from small to large, and the channels that contribute little to the network are removed at the same time, so as to ensure network performance and effectively reduce model complexity.

[0142] The calculation method of SIoU loss in the present invention is:

[0143] L SIoU =λ1·L′ SIoU +λ2·L″ SIoU +λ3·L″′ SIoU

[0144] Among them, L′ SIoU is the loss calculated based on the first and second convolutional layers, L″ SIoU is the loss calculated based on the third and fourth convolutional layers, L″′ SIoU is the loss calculated based on the fifth and sixth convolutional layers; λ1, λ2, and λ3 are fusion weights;

[0145] L′ SIoU , L″ SIoU , L″′ SIoU The calculation process is similar to that of SIoU Take this as an example to illustrate, L′ SIoU It is the sum of the prediction box loss and the classification loss. The calculation process of the prediction box loss L′ is:

[0146]

[0147] γ′=2-Λ

[0148]

[0149] Among them, Λ is the angle loss, Δ is the distance loss, Ω is the shape loss, IoU is the intersection over union loss, c h is an intermediate variable, is the coordinate of the center point of the real frame, is the coordinate of the center point of the prediction box, γ′ is an intermediate variable, e is the base of the natural logarithm, and w gt is the width of the real box, h gt is the height of the real box, w is the width of the predicted box, h is the height of the predicted box, c w It is the length of the rectangle with the line connecting the center points of the real frame and the center points of the predicted frame as the diagonal. After the rectangle is formed with the line connecting the center points of the real frame and the center points of the predicted frame as the diagonal, α is the angle between the long side of the rectangle and the diagonal, and θ is a constant. The θ value is used to define the shape cost, and its value is unique for each data set. This value is a very important item in this loss, which controls how much attention is required for the shape cost. If the θ value is set to 1, it means that the shape is optimized immediately, thereby affecting the free movement of the shape. The general θ value range for each data set is 2 to 6, and the present invention defines the θ value as 4.

[0150] By redefining the distance loss and taking into account the angle loss, a more comprehensive evaluation of the position, size and direction of the object is performed. Specifically, the distance loss is calculated by the distance between the center position of the object and the center position of the predicted bounding box, which not only includes the Euclidean distance but also takes into account the angle between the two. When the directions of the two boxes do not match, the distance loss will increase even if the two positions are very close. At the same time, shape loss is also considered, which has stronger robustness to changes in the shapes of various foreign objects to a certain extent, helps to improve the model's adaptability to target deformation and solve the deformation problem in traffic sign image acquisition. In addition, the loss function of the present invention can be directly applied to multi-category target detection problems without introducing additional complexity, ensuring the speed of the algorithm. It is also insensitive to changes in the size of the target, and will not introduce excessive errors due to changes in the size of the target. The differentiability of the loss function also enables the model to be trained through back propagation and then learn better parameters, thereby improving the accuracy of small target detection in foreign object images.

[0151] Specific implementation method nine: A computer storage medium of this implementation method stores at least one instruction, and the at least one instruction is loaded and executed by a processor to implement the lightweight detection method for foreign objects at the bottom of a railway freight car based on deep learning.

[0152] It should be understood that the instructions include computer program products, software or computerized methods corresponding to any method described in the present invention; the instructions can be used to program a computer system, or other electronic device. Computer storage media may include readable media on which instructions are stored, which may include but are not limited to magnetic storage media, optical storage media; magneto-optical storage media include read-only memory, random access memory, erasable programmable memory such as, and and flash memory layers, or other types of media suitable for storing electronic instructions.

[0153] Specific embodiment 10: This embodiment is a lightweight detection device for foreign matter at the bottom of a railway freight car based on deep learning, the device includes a processor and a memory. It should be understood that including any device including a processor and a memory described in the present invention, the device may also include other units and modules that perform display, interaction, processing, control, etc. and other functions through signals or instructions;

[0154] At least one instruction is stored in the memory, and the at least one instruction is loaded and executed by the processor to implement the method for lightweight detection of foreign objects at the bottom of a railway freight car based on deep learning.

[0155] The above calculation examples of the present invention are only used to explain the calculation model and calculation process of the present invention in detail, and are not intended to limit the implementation methods of the present invention. For ordinary technicians in the relevant field, other different forms of changes or modifications can be made based on the above description. It is impossible to list all the implementation methods here. All obvious changes or modifications derived from the technical solution of the present invention are still within the scope of protection of the present invention.

Claims

1. A method for lightweight detection of foreign objects at the bottom of railway freight cars based on deep learning, characterized in that: The specific process of the method is: Step 1: Obtain a dataset of foreign object images on the bottom of railway freight cars, mark the foreign objects in the images on the bottom of railway freight cars, and obtain a training set of foreign object images on the bottom of railway freight cars; Step 2: Build a foreign object detection network and train the foreign object detection network using a training set of foreign object images at the bottom of a railway freight car; During the training of the foreign object detection network, the working process of the foreign object detection network is: In the foreign object detection network, the input image first passes through the first CBS module, and then the output of the first CBS module is used as the input of the second CBS module; The output of the second CBS module is used as the input of the first C2f module, and the output of the first C2f module is used as the input of the third CBS module; The output of the third CBS module is used as the input of the second C2f module, and the output of the second C2f module is used as the input of the fourth CBS module; The output of the fourth CBS module is used as the input of the third C2f module, and the output of the third C2f module is used as the input of the fifth CBS module; The output of the fifth CBS module is used as the input of the fourth C2f module, and the output of the fourth C2f module is used as the input of the SPPF module; The output of the SPPF module is used as the input of the first upsampling module, and the output of the first upsampling module and the output of the third C2f module are concatenated, and the concatenated result is used as the input of the fifth C2f module; The output of the fifth C2f module is used as the input of the second upsampling module, and the output of the second upsampling module and the output of the second C2f module are concatenated, and the concatenated result is used as the input of the sixth C2f module; Pass the output of the sixth C2f module through the first MCA attention module, use the output of the first channel of the first MCA attention module as the input of the sixth CBS module, and use the output of the second channel of the first MCA attention module as the input of the seventh CBS module; The output of the sixth C2f module is used as the input of the eighth CBS module, the output of the fifth C2f module is spliced ​​with the output of the eighth CBS module, and the spliced ​​result is used as the input of the seventh C2f module; Pass the output of the seventh C2f module through the second MCA attention module, use the output of the first channel of the second MCA attention module as the input of the ninth CBS module, and use the output of the second channel of the second MCA attention module as the input of the tenth CBS module; The output of the seventh C2f module is used as the input of the eleventh CBS module. After splicing the output of the eleventh CBS module and the output of the SPPF module, the splicing result passes through the eighth C2f module, and then the output of the eighth C2f module passes through the third MCA attention module. The output of the first channel of the third MCA attention module is used as the input of the twelfth CBS module, and the output of the second channel of the third MCA attention module is used as the input of the thirteenth CBS module; The output of the sixth CBS module passes through the first convolutional layer, and the predicted box of the feature map with a size of W1×H1×C1 is output through the first convolutional layer; where W1 is the width of the feature map, H1 is the height of the feature map, and C1 is the number of channels of the feature map; The output of the seventh CBS module passes through the second convolutional layer, and the second convolutional layer outputs the predicted category of the feature map with a size of W1×H1×C1; The output of the ninth CBS module passes through the third convolutional layer, and the third convolutional layer outputs the prediction box of the feature map with a size of W2×H2×C2; The output of the tenth CBS module passes through the fourth convolutional layer, and the fourth convolutional layer outputs the predicted category of the feature map with a size of W2×H2×C2; The output of the twelfth CBS module passes through the fifth convolutional layer, and the fifth convolutional layer outputs the prediction box of the feature map with a size of W3×H3×C3; The output of the thirteenth CBS module passes through the sixth convolutional layer, and the sixth convolutional layer outputs the predicted category of the feature map with a size of W3×H3×C3; The loss function L is calculated according to the outputs of the first convolutional layer to the sixth convolutional layer, and the training is stopped when the loss function converges to obtain a trained foreign body detection network; Step 3: Prune the trained foreign body detection network to obtain a pruned foreign body detection network; Step 4: Use the pruned foreign object detection network to identify the bottom image of the railway freight car to be inspected, and obtain the foreign object detection result of the bottom image of the railway freight car to be inspected.

2. According to claim 1, a method for detecting lightweight foreign objects at the bottom of railway freight cars based on deep learning is characterized in that: The specific process of obtaining the foreign body image dataset at the bottom of a railway freight car is as follows: The constructed imaging equipment is used to photograph the bottom of a passing railway freight car to obtain a railway freight car bottom image dataset, and then all railway freight car bottom foreign object images are selected from the obtained railway freight car bottom image dataset, and the railway freight car bottom foreign object image dataset is formed using all the selected railway freight car bottom foreign object images.

3. According to claim 2, a method for detecting lightweight foreign matter at the bottom of a railway freight car based on deep learning is characterized in that: The marking of foreign objects in the bottom image of the railway freight car is specifically as follows: Mark the minimum bounding rectangle of the foreign object in each railway truck bottom image.

4. According to claim 3, a method for detecting lightweight foreign matter at the bottom of a railway freight car based on deep learning is characterized in that: Before using the railway freight car bottom foreign object image training set to train the foreign object detection network, the railway freight car bottom foreign object image training set needs to be amplified, and then the amplified railway freight car bottom foreign object image training set is used to train the foreign object detection network.

5. According to claim 4, a method for detecting lightweight foreign objects at the bottom of railway freight cars based on deep learning is characterized in that: The first CBS module includes a convolutional layer, a BN layer and a SILU activation function layer; and the working process of the first CBS module is: In the first CBS module, the input image first passes through the convolution layer, and then the output of the convolution layer is used as the input of the BN layer; Use the output of the BN layer as the input of the SILU activation function layer; The output of the SILU activation function layer is used as the output of the first CBS module.

6. According to claim 5, a method for detecting lightweight foreign objects at the bottom of railway freight cars based on deep learning is characterized in that: The first C2f module includes the fourteenth CBS module, the fifteenth CBS module, the first Bottleneck unit to the nth Bottleneck unit; wherein: First, the input of the first C2f module is used as the input of the fourteenth CBS module, and then the output of the fourteenth CBS module is divided into two parts, wherein one part of the output passes through the first Bottleneck unit, the output of the first Bottleneck unit passes through the second Bottleneck unit, and the output of the second Bottleneck unit passes through the third Bottleneck unit; And so on, the output of the n-1th Bottleneck unit is used as the input of the nth Bottleneck unit; Then, another part of the output of the fourteenth CBS module is spliced ​​with the outputs of the first Bottleneck unit to the nth Bottleneck unit; Finally, the splicing result is used as the input of the fifteenth CBS module, and the output of the fifteenth CBS module is used as the output of the first C2f module; The first Bottleneck unit includes a sixteenth CBS module and a seventeenth CBS module. The input of the first Bottleneck unit is used as the input of the sixteenth CBS module, and the output of the sixteenth CBS module is used as the input of the seventeenth CBS module. The input of the first Bottleneck unit and the output of the seventeenth CBS module are added, and the addition result is used as the output of the first Bottleneck unit.

7. According to claim 6, a method for detecting lightweight foreign matter at the bottom of a railway freight car based on deep learning is characterized in that: The working process of the first MCA attention module is: In the first MCA attention module, the input F of the first MCA attention module is first used as the input of the first channel and the second channel in parallel; In the first channel, the input F first passes through the LN layer, and then the output of the LN layer passes through three parallel convolutional layers, and the outputs of the three parallel convolutional layers are spliced, and the spliced ​​result is used as the input of the convolutional layer to obtain the output F of the convolutional layer. x ; Then for F x Perform linear transformation to generate value vector V1, key vector K1 and query vector Q1; In the second channel, the input F first passes through the LN layer, and then the output of the LN layer passes through three parallel convolutional layers, and the outputs of the three parallel convolutional layers are spliced, and the spliced ​​result is used as the input of the convolutional layer to obtain the output F of the convolutional layer. y ; Then for F y Perform linear transformation to generate value vector V2, key vector K2 and query vector Q2; Pass the value vector V1, key vector K1 and query vector Q2 through the Reshape layer respectively, multiply the output of the key vector K1 through the Reshape layer and the output of the query vector Q2 through the Reshape layer to obtain the multiplication result a; then multiply a with the output of the value vector V1 through the Reshape layer to obtain the multiplication result b; Then the multiplication result b passes through the Reshape layer and the convolution layer in turn to obtain the output F of the first channel T ; Pass the value vector V2, key vector K2 and query vector Q1 through the Reshape layer respectively, multiply the output of the key vector K2 through the Reshape layer and the output of the query vector Q1 through the Reshape layer to obtain the multiplication result c; then multiply c with the output of the value vector V2 through the Reshape layer to obtain the multiplication result d; Then the multiplication result d is passed through the Reshape layer and the convolution layer in turn to obtain the output F of the second channel B .

8. The method for detecting lightweight foreign matter at the bottom of a railway freight car based on deep learning according to claim 7 is characterized in that: The calculation method of the loss function L is: Where λ represents the balancing factor, Γ represents the set of all BN layers in the foreign object detection network, γ is the scaling factor added by the BN layer, and L SIoU is the SIoU loss; After adding the scaling factor, the working process of the BN layer is: In the formula, z in Represents the input of the BN layer with the scaling factor γ added; represents an intermediate variable; z out represents the output of the BN layer with the scaling factor γ added; β represents the offset coefficient of the BN layer; ε represents the offset; μ B It represents the mean of all activation values ​​of the current BN layer after the training samples of the current batch pass through the current BN layer; It shows the standard deviation of all activation values ​​of the current BN layer after the training samples of the current batch pass through the current BN layer.

9. A computer storage medium, characterized in that: The storage medium stores at least one instruction, and the at least one instruction is loaded and executed by the processor to implement a lightweight detection method for foreign objects at the bottom of a railway freight car based on deep learning as described in any one of claims 1 to 8.

10. A lightweight detection device for foreign matter at the bottom of railway freight cars based on deep learning, characterized in that: The device includes a processor and a memory, wherein the memory stores at least one instruction, and the at least one instruction is loaded and executed by the processor to implement a method for lightweight detection of foreign objects at the bottom of a railway freight car based on deep learning as described in any one of claims 1 to 8.

Images