A multi-scale feature fusion method for detecting foreign object intrusions in railways and its backbone network

Abstract

This invention relates to a multi-scale feature fusion method and backbone network for detecting foreign object intrusions in railways. The multi-scale feature fusion backbone network comprises M layers, each layer including N feature fusion inputs and one convolutional input, N feature fusion outputs and one convolutional output. The N feature fusion outputs of the previous layer in the M layers serve as the N feature fusion inputs of the next layer, and the one convolutional output of the previous layer serves as the one convolutional input of the next layer. The convolutional output of each layer in the M layers also serves as a feature at that scale. The multi-scale feature fusion backbone network of this invention can improve the ability to extract features of railway targets, especially the extraction ability of small targets within the railway network; the features extracted by the method of this invention are more significant. Simultaneously, the use of depthwise separable convolutions can reduce the number of parameters and computational load of the backbone network, reducing the storage and computational pressure on the backbone network.

Description

Technical Field

[0001] This invention relates to the field of railway operation safety inspection technology, and in particular to a method and backbone network for detecting foreign object intrusion into railway lines. Background Technology

[0002] As the main artery of the national economy, a vital national infrastructure, and a popular mode of transportation, railways play a crucial role in my country's economic and social development. In recent years, my country's railways have developed rapidly. By the end of 2021, the total operating mileage of my country's railways exceeded 150,000 kilometers, including over 40,000 kilometers of high-speed rail. With the rapid increase in railway mileage, the task of ensuring the safe operation of trains has become increasingly challenging. Due to the long railway lines, significant climate differences along the railways, and insufficient protection and susceptibility to damage to railway infrastructure, foreign objects such as falling rocks, wild animals, and pedestrians can easily enter the railway clearance area, causing foreign object intrusion incidents and affecting train safety. Furthermore, due to inadequate monitoring methods for foreign objects, intrusion incidents are difficult to detect in the first instance, posing safety hazards to operating trains. Therefore, timely and accurate detection of foreign objects intruding into railway clearance is crucial for ensuring safe railway operation.

[0003] Video surveillance has become a commonly used information acquisition device in railway foreign object intrusion research due to its advantages such as rich information, low cost, intuitive results, no need for additional on-site equipment installation, and no need for personnel training. With the development of deep neural networks, the detection accuracy of target detection algorithms based on deep neural networks has been continuously improved. In the field of railway foreign object intrusion monitoring, neural network-based foreign object intrusion monitoring methods have gradually become the mainstream research approach. The target detection algorithm is the core of neural network-based foreign object intrusion monitoring methods, and the backbone network is an important component of the target detection algorithm. The backbone network undertakes the task of feature extraction in the algorithm, and its feature extraction capability directly affects the detection accuracy of the algorithm; its inference time also affects the detection time of the algorithm.

[0004] Currently, railway foreign object intrusion detection algorithms often use backbone networks designed based on public datasets, or improvements to general backbone networks. However, the target size distribution in railway datasets differs significantly from that in public datasets. In railway scenarios, the target size distribution is relatively concentrated, with over 90% being medium to small targets, while the target size distribution in public datasets is more uniform. Therefore, directly using a general backbone network as a feature extractor in railway scenarios has certain limitations. To address the specific characteristics of railway scenarios, many scholars have improved general backbone networks, including adding attention mechanisms, fusing high- and low-level features, and different forms of model pruning. Although the improved models show some improvement in detection accuracy, continuously adding modules makes the backbone network overly bloated, increasing computational load and time. While model pruning can slim down the model, this method results in a loss of accuracy.

[0005] To further enhance the target detection capability of neural networks in railway scenarios, a backbone network suitable for railway scenarios is needed. This backbone network can improve the feature extraction capability for medium and small-scale targets while ensuring the feature extraction capability for large-scale targets. Based on the designed backbone network, a brand-new railway foreign object intrusion monitoring system can be constructed. Summary of the Invention

[0006] The purpose of this invention is to design a backbone network suitable for feature extraction of intruding objects in railway scenarios, thereby enhancing the backbone network's feature extraction capability for targets in railway scenarios, improving the detection capability of intruding targets in railways, and eliminating safety hazards in a timely manner.

[0007] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0008] A multi-scale feature fusion backbone network includes M layers, each consisting of 2*N MIDO modules (multi-scale feature fusion modules). The multi-scale feature fusion module adopts a multi-input dual-output structure. In each MIDO module, x, x1, x2...xN are feature inputs at different scales, and y1 and y2 are feature outputs at two different scales. In the feature inputs, x1, x2...xN are called feature fusion inputs, and x is called a convolution input. In the feature outputs, y1 is called the feature fusion output, and y2 is called the convolution output. Each layer in the M layers includes N feature fusion inputs and 1 convolution input, as well as N feature fusion outputs and 1 convolution output. The N feature fusion outputs of the previous layer in the M layers serve as the N feature fusion inputs of the next layer, and the 1 convolution output of the previous layer serves as the 1 convolution input of the next layer. The convolution output of each layer in the M layers also serves as the feature k at that scale, where 1≤k≤M.

[0009] Each layer includes N+1 sets of inputs: layer_x, layer_x1, layer_x2, ..., layer_xN, and N+1 sets of outputs: layer_y, layer_y1, layer_y2, ..., layer_yN;

[0010] The inputs layer_x1, layer_x2, ..., layer_xN of each layer are used as the feature fusion inputs x1, x2, ..., xN of the N MIDO modules MIDO Block1_1, MIDO Block1_2, ..., MIDO Block1_N in that layer, respectively. The input layer_x is the convolution input x of the MIDO Block1_1 module.

[0011] For the convolutional input layer_x of this layer, after convolution by the MIDO Block1_1 module, the convolutional output block1_1y2 is obtained. This convolutional output is used as the convolutional input x of the MIDO Block1_2 module. The convolutional input passes through MIDO Block1_1, MIDO Block1_2, ..., MIDO Block1_N, MIDO Block2_1, MIDO Block2_2, ..., MIDO Block2_N in sequence, and then is downsampled to obtain the convolutional output layer_y of this layer.

[0012] For the feature fusion inputs layer_x1, layer_x2, ..., layer_xN of this layer, after passing through N MIDO modules (MIDOBlock1_1, MIDO Block1_2, ..., MIDO Block1_N), each MIDO module outputs one feature fusion output, resulting in a total of N feature fusion outputs from the N MIDO modules. These N feature fusion outputs will be used as feature fusion inputs for MIDO Block2_1, MIDO Block2_2, ..., MIDO Block2_N, respectively. These N modules will then produce N sets of feature fusion outputs. After passing through a downsampling module, these N sets of feature fusion outputs will be used as the feature fusion outputs layer_y1, layer_y2, ..., layer_yN of this layer.

[0013] A downsampling module, downsample, is set for the outputs of MIDO Block2_1, MIDO Block2_2, ..., MIDO Block2_N. The convolutional output and feature fusion output of each layer are output after passing through the downsampling module.

[0014] When different features are input into the MIDO module, the MIDO module assigns a learnable weighting value p0, p1, p2, ..., p3 to xN to the different inputs x, x1 to xN. N This weighting value is used to weight different inputs, and the final value is obtained by the network learning on its own.

[0015] The feature fusion output y1 of the MIDO module is obtained by weighted fusion of N feature inputs and 1 convolution input. This output y1 serves as the output of the MIDO module and is also fed into the MobileNetV2 module. After convolution, y1 yields another convolution output y2 of the MIDO module.

[0016] In the MobileNetV2 module, the number of channels is first increased by using a 1×1 pointwise convolution kernel. Then, a 3×3 convolution kernel is used to perform separable convolution on the features to obtain the corresponding features in the image. Finally, a 1×1 pointwise convolution is used to integrate the information between the channels, restore the number of channels, and output the extracted features.

[0017] A multi-scale feature fusion method for detecting foreign object intrusions in railway lines includes the following steps:

[0018] Step 1. An image is first scaled to H×H and input into the first layer (layer1) of the multi-scale feature fusion backbone network. Layer1 produces N feature fusion outputs and 1 convolution output, which is used as feature1.

[0019] Step 2. The N feature fusion outputs and 1 convolution output of layer1 are used as the feature fusion input and convolution input of the second layer layer2, respectively. The inputs are fed into layer2, and layer2 produces N feature fusion outputs and 1 convolution output. The convolution output is used as feature2.

[0020] Step 3. The N feature fusion outputs and 1 convolution output of layer 2 are used as the feature fusion input and convolution input of the third layer layer 3, respectively. The inputs are fed into layer 3, which produces N feature fusion outputs and 1 convolution output. The convolution output is used as feature3.

[0021] Step 4. By analogy, the N feature fusion outputs and 1 convolution output of layer M-1 are finally used as the feature fusion input and convolution input of layer M, respectively, and input into layer M. Layer M produces 1 convolution output, which is used as feature M.

[0022] Step 5. The features generated by layers 2 to 3M with different scales are upsampled and unified to the same scale as the feature feature 1 output by layer 1. Then, they are uniformly input into the channel weighting unit for feature enhancement. The enhanced features (i.e. the weighted features) are fused by the feature weighting unit and input into the detection module for target classification and detection, so as to realize the classification and recognition of intrusion targets in the railway scene.

[0023] The channel weighting unit adopts the SE attention mechanism, which can generate a weight for each channel of the feature. The SE attention mechanism first converts the information of each channel into a channel identifier through global average pooling, as shown in formula (1):

[0024]

[0025] X c (i,j) represents the value at position (i,j) in channel c of the input feature. The global average pooling layer transforms the C×H×W feature map into a C×1×1 vector. Then, the channel identifiers are passed through two fully connected layers to obtain the channel weights, as shown in formula (2):

[0026] C out =σ(L(δ(L(g) c ))))(2)

[0027] L represents a fully connected layer, σ represents the sigmoid activation function, and δ represents the ReLU activation function.

[0028] After obtaining the weights of each layer, the weights are multiplied by the original features of the corresponding input channels, and then added to the original features to obtain the weighted features, as shown in formula (3):

[0029]

[0030] X' represents the weighted feature, X represents the original feature, and C out Weights generated by the attention mechanism, Indicates that C out The one-dimensional weight in the matrix is ​​multiplied by the corresponding feature channel in the original feature X.

[0031] The beneficial effects of this invention are:

[0032] 1. The multi-scale feature fusion backbone network designed in this invention can improve the ability to extract features of railway targets, especially the ability to extract features of small and medium-sized targets in railways; compared with the detection effect of other methods, the features extracted by the method of this invention are more significant.

[0033] 2. Depthwise separable convolution can reduce the number of parameters and computational cost of the backbone network, thereby reducing the storage and computational pressure on the backbone network. Attached Figure Description

[0034] The present invention includes the following figures:

[0035] Figure 1 Schematic diagram of the multi-scale feature fusion module structure.

[0036] Figure 2 Schematic diagram of MobileNetV2 module structure.

[0037] Figure 3 A schematic diagram of the structure of a single layer in a multi-scale feature fusion backbone network.

[0038] Figure 4 Schematic diagram of the multi-scale feature fusion backbone network structure.

[0039] Figure 5 Schematic diagram of channel weighting unit and feature weighting unit.

[0040] Figure 6 In the diagram, Figures a and b are schematic diagrams of the detection effect and obtained heat map features of other methods, respectively; Figures c and d are schematic diagrams of the detection effect and obtained heat map features of the present invention, respectively. Detailed Implementation

[0041] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments.

[0042] 1. This invention relates to a multi-scale feature fusion method for detecting foreign object intrusions in railway lines, comprising the following (taking a 4-layer backbone network as an example):

[0043] Step 1. An image is first scaled to H×H (256×256 in this example) and then input into... Figure 4 The backbone network shown is layer 1, which produces three feature fusion outputs and one convolutional output, which is used as feature1.

[0044] Step 2. The three feature fusion outputs and one convolution output of layer 1 are used as the feature fusion input and convolution input of the second layer layer 2, respectively. Layer 2 produces three feature fusion outputs and one convolution output, which is used as feature2.

[0045] Step 3. The three feature fusion outputs and one convolution output of layer 2 are used as the feature fusion input and convolution input of the third layer layer 3, respectively. Layer 3 produces three feature fusion outputs and one convolution output, which is used as feature3.

[0046] Step 4. The three feature fusion outputs and one convolution output of layer 3 are used as the feature fusion input and convolution input of the fourth layer layer 4, respectively. The input is fed into layer 4, and layer 4 produces one convolution output, which is used as feature4.

[0047] Step 5. Upsample the features (features 2-4) generated by layers 2-4 to the same scale as the feature (feature 1) output by layer 1, and then input them into the same input. Figure 5 The channel weighting unit shown performs feature enhancement. The enhanced features are then fused by the feature weighting unit and input into the detection module for target classification and detection, thereby realizing the classification and recognition of intrusion targets in railway scenarios.

[0048] The channel weighting unit described in step 5 employs the SE attention mechanism. The SE attention mechanism generates a weight for each channel of the feature and multiplies it by the original feature channel to obtain the enhanced feature. The enhanced feature has a stronger expressive power for the target in the image, improving the model's average detection accuracy. The SE attention mechanism first converts the information of each channel into a channel identifier through global average pooling, as shown in formula (1):

[0049]

[0050] X here c (i,j) represents the value at position (i,j) in channel c of the input feature. The global average pooling layer transforms the CxHxW feature map into a Cx1x1 vector. Then, these channel identifiers are passed through two fully connected layers to obtain the channel weights, as shown in formula (2):

[0051] C out =σ(L(δ(L(g) c (2)

[0052] Here, L represents a fully connected layer, σ represents the sigmoid activation function, and δ represents the ReLU activation function. After obtaining the weights of each layer, the weights are multiplied by the original features of the corresponding input channels, and then added to the original features to obtain the final output, as shown in formula (3).

[0053]

[0054] X' represents the weighted feature, X represents the original feature, and C out Weights generated by the attention mechanism, Indicates that C out The one-dimensional weight in the matrix is ​​multiplied by the corresponding feature channel in the original feature X.

[0055] Channel weighting only enhances features across different channels of the same feature, without considering the differences between different features. Therefore, a feature weighting method is designed to measure the differences between different features. The algorithm initializes a set of learnable parameters t and inputs t into the softmax function. The output of the function yields the weighted values ​​p. The maximum output of the softmax function is 1, and it can make the sum of all weighted values ​​p equal to 1, thus limiting the range of weights and improving the model's generalization ability.

[0056] 2.a) The multi-scale feature fusion railway intrusion foreign object intrusion detection method of the present invention uses a multi-scale feature fusion backbone network to extract and fuse features at multiple scales in the image, and realizes the detection of intrusion targets through a target detection unit.

[0057] b) The multi-scale feature fusion backbone network described in a) comprises M layers, each layer including N feature fusion inputs and 1 convolutional input, N feature fusion outputs and 1 convolutional output; the N feature fusion outputs of the previous layer in the M layers serve as the N feature fusion inputs of the next layer, and the 1 convolutional output of the previous layer serves as the 1 convolutional input of the next layer; the convolutional output of each layer in the M layers also serves as the feature k at that scale, 1≤k≤M.

[0058] Design the entire MIDO backbone network based on the structure of each layer. Figure 4 The diagram shows a 4-layer multi-scale feature fusion backbone network structure. Each layer performs downsampling once, with a downsampling rate of 2.

[0059] c) For the M features (feature1~featureM) output in b), feature2~featureM are upsampled to the same scale as feature1, and then uniformly input into the channel weighting unit for feature enhancement. Figure 5 The diagram shows the channel weighting unit corresponding to M=4. Since channel weighting only enhances features between different channels of the same feature, and the differences between different features are not weighed, the features enhanced by the channel weighting unit are fused by the feature weighting unit and then input into the detection module for target classification and detection, thereby realizing the classification and recognition of intrusion targets in railway scenarios.

[0060] 3. Each layer in the M layers of the multi-scale feature fusion backbone network consists of 2*N MIDO modules (multi-scale feature fusion modules), containing N+1 sets of inputs: layer_x, layer_x1, layer_x2, ..., layer_xN, and N+1 sets of outputs: layer_y, layer_y1, layer_y2, ..., layer_yN. Multiple inputs provide the foundation for the backbone network to process multi-scale features, while multiple outputs provide the backbone network with rich feature outputs. Figure 3 The diagram shows the case where N=3.

[0061] The inputs layer_x1, layer_x2, ..., layer_xN of each layer serve as the feature fusion inputs x1, x2, ..., xN of the N MIDO modules MIDO Block1_1, MIDO Block1_2, ..., MIDO Block1_N in that layer, respectively. The input layer_x is the convolution input x of the MIDO Block1_1 module.

[0062] For the convolutional input layer_x of this layer, after convolution by the MIDO Block1_1 module, the convolutional output block1_1y2 is obtained. This convolutional output is used as the convolutional input x of the MIDO Block1_2 module. The convolutional input is then passed sequentially through MIDO Block1_1, MIDO Block1_2, ..., MIDO Block1_N, MIDO Block2_1, MIDO Block2_2, ..., MIDO Block2_N, and then downsampled to obtain the convolutional output layer_y of this layer. The convolutional output layer_y undergoes multiple convolutions, resulting in a larger receptive field and more abstract feature information.

[0063] For the feature fusion inputs layer_x1, layer_x2, ..., layer_xN of this layer, after passing through N MIDO modules (MIDOBlock1_1, MIDO Block1_2, ..., MIDO Block1_N), each MIDO module outputs one feature fusion output, resulting in a total of N feature fusion outputs from the N MIDO modules. These N feature fusion outputs will then serve as the feature fusion inputs for the MIDO Block2_1, MIDO Block2_2, ..., MIDO Block2_N modules, which will generate N sets of feature fusion outputs. These N sets of feature fusion outputs will then be processed by a downsampling module and used as the feature fusion outputs layer_y1, layer_y2, ..., layer_yN of this layer.

[0064] The feature fusion outputs layer_y1, layer_y2, ..., layer_yN have undergone extensive feature fusion without convolution, containing more detailed target information. A downsampling module, downsample, is implemented for the outputs of MIDO Block2_1, MIDO Block2_2, ..., MIDO Block2_N. This downsampling module uses a 3x3 convolutional kernel with a stride of 2 to achieve a 2x downsampling of the features. The convolutional outputs and feature fusion outputs of each layer are processed by the downsampling module before being output. The downsampling module reduces the feature size, thereby reducing the number of parameters and computational cost of the algorithm. Convolution is used for downsampling because the parameters in the convolutional kernel can retain feature information during the downsampling process, reducing information loss caused by downsampling.

[0065] 4. To enable the multi-scale feature fusion backbone network module to receive more feature information and generate features containing information at different levels for the image, the MIDO module is designed with a multi-input dual-output structure. The MIDO module can receive feature inputs at different scales and produce two different outputs: feature fusion output and convolutional output.

[0066] The overall structure of the MIDO module is as follows: Figure 1 As shown, x, x1, x2...xN are feature inputs at different scales, and y1 and y2 are feature outputs at two different scales. In the feature inputs, x1, x2...xN are called feature fusion inputs, and x is called the convolution input. In the feature outputs, y1 is called the feature fusion output, and y2 is called the convolution output.

[0067] Different inputs have varying degrees of influence on the final target prediction. To reflect this influence, different weighting values ​​are assigned to different inputs to the MIDO module. When different features are input into the MIDO module, the module assigns a learnable weighting value p0, p1, p2, ..., p3 to xN to different inputs x, x1 to xN. N This weighting value is used to weight different inputs, and the final value is obtained by the network learning on its own.

[0068] The MIDO module's feature fusion output y1 is obtained by weighted fusing N feature fusion inputs and 1 convolutional input. This output y1 stores the feature fusion results of the MIDO module. Since y1 incorporates different input information without undergoing convolutional operations, it possesses more detailed features and plays a crucial role in predicting small targets in railway systems.

[0069] The feature fusion output y1 serves as the module's output on one hand, and on the other hand, it is also fed into the MobileNetV2 module. The structure of the MobileNetV2 module is as follows: Figure 2 As shown, y1, after convolution, yields another convolutional output y2 from the MIDO module. In the MobileNetV2 module, a 1×1 pointwise convolution is first performed using a convolutional kernel to increase the number of channels. Then, a 3×3 convolutional kernel is used to perform separable convolution on the features to extract the corresponding features from the image. Finally, a 1×1 pointwise convolution is used to integrate the information between the channels, restore the number of channels, and output the extracted features. The convolutional output y2, having undergone feature extraction by the convolutional module, possesses a larger receptive field and more abstract feature information, playing a crucial role in the classification of railway targets. Figure 6 In the diagram, Figure a shows the detection results of other methods, where two pedestrians in the distance cannot be detected; Figure c shows the detection results of the multi-scale feature fusion detection method of the present invention, where all pedestrians are effectively detected. Figures b and d show the heatmap features obtained by other methods and the present invention, respectively, with the heatmap features obtained by the method of the present invention being more obvious.

[0070] The above embodiments are only used to illustrate the present invention and are not intended to limit the present invention. Those skilled in the art can make various changes and modifications without departing from the essence and scope of the present invention. Therefore, all equivalent technical solutions also fall within the protection scope of the present invention.

[0071] The contents not described in detail in this specification are existing technologies known to those skilled in the art.

Claims

1. A multi-scale feature fusion backbone network, characterized in that: Includes M layers, each layer consisting of 2 The system consists of N multi-scale feature fusion modules, each employing a multi-input dual-output structure. In each module, x, x1, x2…xN are feature inputs at different scales, and y1 and y2 are feature outputs at two different scales. Among the feature inputs, x1, x2…xN are called feature fusion inputs, and x is called the convolution input. Among the feature outputs, y1 is called the feature fusion output, and y2 is called the convolution output. Each of the M layers includes N feature fusion inputs and one convolution input, as well as N feature fusion outputs and one convolution output. The N feature fusion outputs of the previous layer in the M layers serve as the N feature fusion inputs of the next layer, and the one convolution output of the previous layer serves as the one convolution input of the next layer. The convolution output of each layer in the M layers also serves as the feature k at that scale, where 1 ≤ k ≤ M. Each layer includes N+1 sets of inputs: layer_x, layer_x1, layer_x2, ..., layer_xN, and N+1 sets of outputs: layer_y, layer_y1, layer_y2, ..., layer_yN; The inputs layer_x1, layer_x2, ..., layer_xN of each layer are used as the feature fusion inputs x1, x2, ..., xN of the N multi-scale feature fusion modules MIDO Block1_1, MIDO Block1_2, ..., MIDO Block1_N in that layer, respectively. The input layer_x is used as the convolution input x of the MIDO Block1_1 module. For each layer's convolutional input layer_x, after convolution by the MIDO Block1_1 module, a convolutional output block1_1y2 is obtained. This convolutional output is used as the convolutional input x of the MIDO Block1_2 module. The convolutional input passes sequentially through MIDO Block1_1, MIDO Block1_2, ..., MIDO Block1_N, MIDO Block2_1, MIDO Block2_2, ..., MIDO Block2_N, and is then downsampled to obtain the convolutional output layer_y of that layer. In each multi-scale feature fusion module, N feature fusion inputs and 1 convolution input are weighted and fused to obtain the feature fusion output y1 of the multi-scale feature fusion module. The feature fusion output y1 serves as the output of the multi-scale feature fusion module and is also fed into the MobileNetV2 module. After convolution, y1 is used to obtain another convolution output y2 of the multi-scale feature fusion module. The input is an image.

2. The multi-scale feature fusion backbone network as described in claim 1, characterized in that: For each layer's feature fusion input layer_x1, layer_x2, ..., layer_xN, after passing through N multi-scale feature fusion modules (MIDO Block1_1, MIDO Block1_2, ..., MIDOBlock1_N), each multi-scale feature fusion module outputs one feature fusion output, resulting in a total of N feature fusion outputs from the N multi-scale feature fusion modules. These N feature fusion outputs will be used as feature fusion inputs for modules MIDOBlock2_1, MIDO Block2_2, ..., MIDO Block2_N, respectively. These N modules will then generate N sets of feature fusion outputs, which will be further processed by downsampling modules to become the feature fusion outputs layer_y1, layer_y2, ..., layer_yN for that layer.

3. The multi-scale feature fusion backbone network as described in claim 2, characterized in that: A downsampling module, downsample, is set for the outputs of MIDO Block2_1, MIDO Block2_2, ..., MIDO Block2_N. The convolutional output and feature fusion output of each layer are output after passing through the downsampling module.

4. The multi-scale feature fusion backbone network as described in claim 1, characterized in that: When different features are input into the multi-scale feature fusion module, the multi-scale feature fusion module provides different inputs. x , x 1~ x N is assigned a learnable weighted value p0, p1~p N This weighting value is used to weight different inputs, and the final value is obtained by the network learning on its own.

5. The multi-scale feature fusion backbone network as described in claim 1, characterized in that: In the MobileNetV2 module, the number of channels is first increased by using a 1×1 pointwise convolution kernel. Then, a 3×3 convolution kernel is used to perform separable convolution on the features to obtain the corresponding features in the image. Finally, a 1×1 pointwise convolution is used to integrate the information between the channels, restore the number of channels, and output the extracted features.

6. A method for detecting foreign object intrusion in railway lines using multi-scale feature fusion, characterized in that: The multi-scale feature fusion backbone network according to any one of claims 1-5 includes the following steps: Step 1. An image is first scaled to H×H and input into the first layer 1 of the multi-scale feature fusion backbone network. Layer 1 produces N feature fusion outputs and 1 convolution output, which is used as feature1. Step 2. The N feature fusion outputs and 1 convolution output of layer 1 are used as the feature fusion input and convolution input of the second layer layer 2, respectively. The inputs are fed into layer 2, and layer 2 produces N feature fusion outputs and 1 convolution output. The convolution output is used as feature2. Step 3. The N feature fusion outputs and 1 convolution output of layer 2 are used as the feature fusion input and convolution input of the third layer layer 3, respectively. The inputs are fed into layer 3, which produces N feature fusion outputs and 1 convolution output. The convolution output is used as feature3. Step 4. By analogy, the N feature fusion outputs and 1 convolution output of layer M-1 are finally used as the feature fusion input and convolution input of layer M, respectively, and input into layer M. Layer M produces 1 convolution output, which is used as feature M. Step 5. The features generated by layers 2 to 3M with different scales are upsampled and unified to the same scale as the feature output by layer 1. Then, they are uniformly input into the channel weighting unit for feature enhancement. The enhanced features are fused by the feature weighting unit and input into the detection module for target classification and detection, so as to realize the classification and recognition of intrusion targets in the railway scene.

7. The railway foreign object intrusion detection method based on multi-scale feature fusion as described in claim 6, characterized in that: The channel weighting unit adopts the SE attention mechanism, which can generate a weight for each channel of the feature. The SE attention mechanism first converts the information of each channel into a channel identifier through global average pooling, as shown in formula (1): ; Representing input features c The location in the passage is The global average pooling layer transforms the C×H×W feature map into a C×1×1 vector; then, after the channel identifier passes through two fully connected layers, the channel weights are obtained, as shown in formula (2): ; L represents a fully connected layer. This represents the sigmoid activation function. Represents the ReLU activation function; After obtaining the weights of each layer, the weights are multiplied by the original features of the corresponding input channels, and then added to the original features to obtain the weighted features, as shown in formula (3): ; Represents the weighted features. Indicates original features, Weights generated by the attention mechanism, Indicates will One-dimensional weights and original features Multiply the corresponding feature channels.

Images