A railway scene uncommon intrusion target identification method based on small sample learning

Abstract

The present application relates to a kind of railway scene unusual intrusion target identification method based on small sample learning.The present application can acquire the railway scene image of railway along camera in real time, utilize improved metric meta-learning network model to identify unusual intrusion target in railway scene.The model includes pre-trained feature mapping network, self-learning class center fine-tuning network, distance measurement module three parts, and center correlation loss function is used to measure the training of meta-learning model.The present application can classify and identify the image of railway empty scene and scene, including but not limited to train, rockfall, pedestrian, debris flow and other categories.When train passes, no alarm is sent, and when rockfall, pedestrian, debris flow and other foreign matters invade railway gauge in railway scene, alarm information is sent, and relevant personnel is reminded by wired or wireless mode.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of railway operation safety monitoring, in particular to a railway scene uncommon intrusion target identification method based on small sample learning. BACKGROUND

[0002] Ensuring train operation safety is the top priority of the railway safety guarantee system. Line dangers such as personnel and foreign matter intrusion caused by natural disasters such as mudslides and human factors are one of the important reasons for major accidents on railways. Mudslides and rockfalls and other disasters have a low occurrence probability but are extremely dangerous. Because the braking distance of a high-speed train is much greater than the driver's visual distance, when the driver discovers the disaster, it is too late to take emergency braking measures. At the same time, it is difficult to collect enough samples for traditional deep learning recognition model network training from such images. How to accurately identify mudslides, rockfalls and other intrusion foreign matters in the railway scene based on only a small number of image samples is of great significance to ensuring railway operation safety.

[0003] In recent years, with the rapid development of artificial intelligence, methods based on intelligent video analysis have been increasingly applied in railway foreign matter intrusion detection systems due to their simple installation and cost advantages. However, traditional deep learning methods often require the collection of a large number of training samples to enable the network to have better generalization performance. In practice, on the one hand, line rockfalls, mudslides and other railway dangers often cannot collect enough samples, and on the other hand, with the continuous development of society, new foreign matter categories will continue to appear, and traditional deep learning cannot identify these new categories of intrusion foreign matters. To solve the above problems, the present application designs a small sample learning method suitable for the railway scene by improving the metric meta-learning model and designs a corresponding system for identifying mudslides, rockfalls, trains, pedestrians and other targets in the railway scene.

[0004] Metric meta-learning is a relatively effective method among many small sample learning methods. This method focuses on two parts: first, designing a good feature mapping module to make samples of the same category closer and samples of different categories farther apart, and second, learning a good metric model to better describe the differences between samples of different categories.

[0005] The performance of the feature mapping module directly determines the final detection accuracy of the model. Most current feature mapping modules use convolutional blocks to extract features, and all features contribute equally to the final classification decision. However, in reality, the features to be detected are the most important content that the network needs to focus on and should have the greatest contribution to classification. To solve this problem, the present application proposes an improved channel attention module that uses the extracted features to generate feature weights on each channel for distance measurement in subsequent classification.

[0006] In the initialization of the category center, SNELL J et al. adopt the mean value of all sample features in the support set as the center of all categories, and Finn C et al. directly fuse the sample features in the support set, but sometimes the noise samples may be distributed in a position far away from the feature space, and are not suitable for initializing the category center. Therefore, a category center fine-tuning algorithm is proposed in this paper, which enables the network to learn the center of each category by itself.

[0007] The metric meta-learning algorithm generally directly adopts meta-training to train the network, and adopts cross-entropy loss function to optimize the network parameters; meta-training enables the network to learn quickly, but the feature extraction capability is weak due to the lack of prior knowledge; at the same time, the cross-entropy loss only considers whether the classification is correct, and does not consider the distribution relationship of the features of the same category samples in the feature space. In order to improve the feature extraction capability of the network and make the features of the same category more compact in the sample space, this paper introduces the model pre-training method and the center-related loss function to improve the detection effect. SUMMARY

[0008] In view of the defects in the prior art, the purpose of the present application is to provide a railway scene unusual intrusion target recognition method based on small sample learning, which can realize real-time detection and classification identification of the intrusion foreign matter only by using a few training samples.

[0009] To achieve the above purpose, the technical scheme adopted by the present application is:

[0010] A railway scene unusual intrusion target recognition method based on small sample learning, characterized in that:

[0011] An improved metric meta-learning network model is used for recognizing the unusual intrusion target in the railway scene, and the improved metric meta-learning model includes a pre-trained feature mapping network, a self-learning class center fine-tuning network and a distance metric module; and a center-related loss function is used for training the feature mapping network.

[0012] On the basis of the above scheme, a railway scene unusual intrusion target recognition method based on small sample learning includes the following steps:

[0013] Step 1, first input a small sample public data set similar to the railway image feature into a randomly initialized feature mapping network for pre-training to obtain a pre-trained feature mapping network and its parameters;

[0014] Step 2, a meta-training set is established by using a small sample public data set, and is input into the pre-trained feature mapping network obtained in step 1 for metric meta-training to obtain a meta-trained feature mapping network and its parameters;

[0015] Step 3, the railway empty scene image and the intrusion target sample image are used to establish a small sample support set, which is input into the meta-trained feature mapping network obtained in step 2, and the meta-trained feature mapping network is used to map the above intrusion target sample to the same feature space;

[0016] Step 4, the image features of the support set obtained in step 3 are input into the self-learning class center fine-tuning network, and the center representation of each image class is learned;

[0017] Step 5, the image to be tested is input into the meta-trained feature mapping network obtained in step 2 to obtain the image features of the image to be tested, and then the distance measurement module is used to calculate the similarity between the image features to be tested and the center of each image class obtained in step 4, to obtain the intrusion target recognition result.

[0018] On the basis of the above scheme, the feature mapping network in step 1 is a feature mapping network based on a channel attention mechanism, which is composed of two parts:

[0019] The first part is composed of four residual blocks connected in turn, and each residual block is followed by a maximum pooling layer;

[0020] The second part is an improved channel attention module.

[0021] On the basis of the above scheme, the improved channel attention module comprises:

[0022] The information compression part compresses the features after convolution in the spatial dimension by using a parallel global maximum pooling layer and a global average pooling layer ;

[0023] The information excitation part fuses the features of the global maximum pooling layer and the global average pooling layer by adding them, and uses an activation function to normalize the fused features to 0-1 to obtain the channel attention weight value; the activation function can be a Sigmod activation function.

[0024] Channel feature enhancement , the obtained channel attention weight value is multiplied by the original feature value F, so as to realize attention enhancement of the key channel.

[0025] On the basis of the above scheme, the initial network parameters of the feature mapping network are obtained by using a model pre-training method, and then meta-training is performed, and the specific training steps are as follows:

[0026] Step 1-1: Train a classifier on the feature mapping network based on the channel attention mechanism using standard cross-entropy loss. During the training process, adopt the method of training and meta-validation simultaneously. Save the feature mapping network with the highest meta-validation accuracy under the N-way K-shot setting as the pre-trained model.

[0027] Steps 1-2: Extract the network parameters of the pre-trained model, transfer them to metric learning, and continue training the network using meta-training.

[0028] Based on the above scheme, the steps of the self-learning class center fine-tuning network processing algorithm are as follows:

[0029] Step 2-1, for the support set S The middle category is k All samples Features are obtained after passing through a feature mapping network. The mean obtained is used as the initial value of the central prototype of the learnable parameter class k in the feature space, as shown in equation (1):

[0030] ;

[0031] in, N For category k The total number of samples, For is the category k The i Sample Features obtained after passing through a feature mapping network;

[0032] Step 2-2: According to equation (1), the initial values ​​of all class center prototypes in the support set are obtained sequentially. Using the softmax function, the distance from the sample in the support set to each initial class center is normalized to obtain the values ​​belonging to the class. k The probability is given by equation (2):

[0033] ;

[0034] in d Let be the Euclidean distance between two eigenvectors in the feature space;

[0035] Steps 2-3: Calculate the cross-entropy loss according to equation (3):

[0036] .

[0037] Based on the above scheme, the use of the center correlation loss function for feature mapping network training is specifically as follows:

[0038] Central loss L cThe expression is specifically as follows formula (4):

[0039] ;

[0040] In the above formula Is a hyperparameter, which can be used to control the distance within the class during training; Indicates the feature value of a certain intrusion target sample in the feature space, c k Indicates the class center of each image category.

[0041] At the same time, during the training of the feature mapping network, the support set feature mean is used to reinitialize the class center each time iteration;

[0042] The support set feature mean is determined by referring to the method described in step 2-1, and specifically:

[0043] For all samples of class k in the support set S , the feature obtained after the feature mapping network is According to the obtained mean value as the initial value of the center prototype of the learnable parameter class k in the feature space, as follows:

[0044] ;

[0045] Finally, the cross-entropy loss and the center loss are combined to construct the center-related loss function represented by the following formula (5) for feature mapping network training:

[0046] .

[0047] The railway scene uncommon intrusion target identification method based on small sample learning has the beneficial effects that:

[0048] By using the above method, the railway scene images obtained by the cameras along the railway can be collected in real time, and the above small sample learning method is used to classify and identify the images of the railway scene, including but not limited to trains, falling rocks, pedestrians, mudslides and other categories. When the train passes, no alarm is given, but when the railway scene has falling rocks, pedestrians, mudslides and other foreign objects invading the railway clearance, an alarm information is given to remind the relevant personnel through wired or wireless mode. BRIEF DESCRIPTION OF DRAWINGS

[0049] The present application has the following drawings:

[0050] Figure 1 Metric meta-learning model structure diagram;

[0051] Figure 2 Feature mapping network structure diagram;

[0052] Figure 3 Improved channel attention module;

[0053] Figure 4 Railway dataset heatmap: (a) Train entry, (b) Debris flow intrusion, (c) Pedestrian intrusion, (d) Rockfall intrusion;

[0054] Figure 5 Sample distribution map without using model pre-training and center correlation loss;

[0055] Figure 6 Using model pre-training and center correlation loss sample distribution diagram. Detailed Implementation

[0056] The present invention will be further described in detail below with reference to the accompanying drawings.

[0057] The following detailed description of specific embodiments of the present invention is provided in conjunction with examples. These embodiments are used to explain the present invention, but are not intended to limit the scope of protection of the present invention.

[0058] Example 1

[0059] The following describes an embodiment of the present invention, which can be used to detect and identify foreign object intrusion scenarios such as mudslides and rockfalls in railway settings. A sample dataset of railway pedestrian intrusion images, railway empty scene images, railway train images, railway rockfall images, and mudslide images was established using images acquired at the railway site, as shown in Table 1. Each sample class contains 40 images.

[0060] Table 1. Composition of the railway dataset

[0061] Number Scene Category Number / Sheet 1 Empty Scene 40 2 Pedestrian Invasion 40 3 Debris Flow Invasion 40 4 Rockfall Invasion 40 5 Train Passing 40 ;

[0062] like Figure 1 As shown, the network takes images from the support set (including empty railway scenes, as well as intrusion target samples such as pedestrian intrusion, debris flow intrusion, rockfall intrusion, and train passing) and the query set (i.e., the image to be tested) as input. First, a feature mapping network maps them to the same feature space. Then, the image features of the support set are input into the class center fine-tuning network to learn the center representation of each class. Finally, the distance metric module is used to calculate the similarity between the image features of the query set and the centers of each class to predict the classification result.

[0063] According to the training mode of the small sample learning learning algorithm, the model is first trained on the public dataset miniImageNet, and then migrated to the railway dataset for meta testing. The experimental parameter settings for meta training on the public dataset are as follows. First, the meta learning algorithm designed in this paper is built on the PyTorch 0.4.0 platform of Ubuntu, the model uses the Adam (Adaptive Moment Estimation) optimization algorithm to update the model parameters, the initial learning rate is 0.001, the learning rate is updated every 20 epochs using the equal interval adjustment method, the multiplication factor gamma = 0.5, each epoch contains 100 meta tasks, a total of 200 epochs, each meta task support set (support set) has 5 categories, and each category has 5 pictures, that is, 5-way 5-shot setting. In the test stage, the classifier used for fine-tuning uses the SGD optimization algorithm to fine-tune the center, the initial learning rate is set to 0.1, the momentum factor momentum = 0.9, and the momentum damping factor dampening = 0.9. The above settings are only to illustrate the process, and the parameters are not a constraint on the present application.

[0064] In order to further analyze the effect of the model on the railway dataset, according to the related literature, the model migrated to the railway dataset for meta testing after meta training on the public dataset miniImageNet.

[0065] The experimental detection results of the algorithm of the present application and the classic networks ProtoNet, RelationNet, MAML and MatchNet under 5-way 1-shot and 5-way 5-shot settings are compared, and the experimental results are shown in Table 2. Under the railway small sample dataset, the algorithm of the present application has a 3.61% improvement in detection accuracy compared with the classic network ProtoNet with the best effect on the railway dataset under 5-way 5-shot setting, and a 3.51% improvement under 5-way 1-shot setting.

[0066] Table 2 Experimental results on the railway dataset

[0067] ;

[0068] To more intuitively see the effect of network joining the channel attention mechanism, the GradCAM visualization technology is used to extract the feature map after the last convolution of the image after the feature mapping network. Starting from the node of the maximum probability classification category, the gradient mean of the feature map is calculated by back propagation as the weight of each feature map. Each feature map is multiplied by the weight to obtain a weighted feature map. The mean value is calculated in the third dimension. After ReLU activation and normalization processing, the processed heat map is scaled to the size of the image. Finally, it is superimposed on the original image. The effect visualization is shown in Figure 4 .

[0069] In Figure 4 (a)-(d), the left picture represents the original image input into the network, and the right picture is the contribution of different features in the image to classification when we use GradCAM visualization technology to output the final classification. The color change from blue to red in the figure represents the gradually increasing contribution. Blue has low contribution, and red area has high contribution. Overall, the feature extraction network can focus on different intruding objects.

[0070] To more intuitively observe the comparison of the metric space before and after using model pre-training and center-related loss, under the 5-way 5-shot setting, the feature map of the last layer of the convolutional neural network after the model is trained is extracted, and TSNE analysis method is used to reduce it to two-dimensional plane. The effect is shown in Figure 5 and Figure 6 .

[0071] Figure 5 In the above, the empty scene and pedestrian category samples are distributed in an interleaved manner, which makes it difficult for the distance measurement module to distinguish between the two; at the same time, the rockfall is distributed relatively dispersed in the feature space, resulting in a high classification error rate. In Figure 6 , after using model pre-training and center-related loss function, the pedestrian and empty scene distribution tends to be far away and the rockfall class target feature distribution tends to be dense, indicating that the center-related loss function can effectively reduce the distance between samples of the same class. After model pre-training, the distribution quality of each class of samples in the feature space is significantly improved. Experiments prove that using model pre-training and center-related loss function can achieve good results in railway small sample dataset classification.

[0072] To verify the effectiveness of the channel attention module, class center fine-tuning algorithm, model pre-training and center correlation loss function designed in this algorithm on the railway dataset, the model constructed in this paper is taken as the benchmark, and the four modules are added respectively in the network for ablation experiments. In order to ensure the fairness of the experiment, 20-way 5-shot small sample learning task setting is adopted in the training process, and Manhattan distance is used as the distance function in the test process. The experimental results are shown in Table 3. In the 5-way 5-shot setting, the improved attention mechanism CAM module has a significant effect on the improvement of accuracy, which improves the detection accuracy by 0.86%; the class center fine-tuning algorithm can reduce the influence of noise samples on the initialization of class center, which improves the detection accuracy by 3.05%; the center correlation loss function can make the features of the same class more compact, which improves the detection accuracy by 1.91%; the model pre-training improves the detection accuracy by 6.70%; the combination of the four improves the detection accuracy by 7.90%. In the 5-way 1-shot setting, that is, when there is only one picture of each class in the support set, the center fine-tuning algorithm is no longer effective, so it is not discussed here; when the CAM module is introduced alone in the network, the detection accuracy can be improved by 3.75%; after adding the center correlation loss function alone, the detection accuracy can be improved by 5.89%; after adding the model pre-training method, the detection accuracy can be improved by 5.15%. Compared with the backbone network ResNet12, when the area occupied by the target to be tested is small, the improved attention mechanism CAM module can effectively focus on important features, making them occupy a larger proportion in classification; in the process of class center initialization, class center fine-tuning can effectively filter out samples whose features are far away from the features of the same class in the feature space, and also effectively filter out the interference of noise samples; while the center correlation loss function can make the features of the same class more compact in the feature space, which is more conducive to subsequent classification; introducing model pre-training before meta-training can make the feature mapping network obtain more prior knowledge, and the feature extraction capability is stronger. Therefore, the attention module CAM, class center fine-tuning algorithm, center correlation loss function and model pre-training can effectively improve the classification detection accuracy of the network.

[0073] Table 3 Ablation experiment on railway dataset

[0074] ;

[0075] To verify the influence of the number of samples on the small sample learning model, the number of samples in the railway support set is sequentially increased for related experiments, and the experimental results under the settings of 5-way 5-shot, 5-way 10-shot, 5-way 15-shot and 5-way 20-shot are shown in Table 4. It can be seen from the table that with the increase of the number of samples in the support set, the accuracy of the small sample learning model designed in this paper will continue to improve, and under the setting of 5-way 20-shot, the accuracy can reach 93.51±0.21%. Since it is operable to collect dozens of samples of each type of intrusion target in actual railway scene application, the recognition mode of 5-way 20-shots has wide practical application value and can be directly used for classification and recognition of rare types of intrusion targets in railway scenes.

[0076] Table 4 Model accuracy under different number of samples in support set

[0077] N-way K-shot Accuracy (%) 5-way 5-shot 85.44±0.25 5-way 10-shot 88.16±0.22 5-way 15-shot 91.21±0.21 5-way 20-shot 93.51±0.21

[0078] The contents not described in detail in the specification belong to the prior art known to those skilled in the art.

Claims

1. A railway scene unusual intrusion target identification method based on small sample learning, characterized in that: an improved metric meta-learning network model is used to identify unusual intrusion targets in a railway scene, and the improved metric meta-learning model includes a pre-trained feature mapping network, a self-learning class center fine-tuning network, and a distance metric module; and a center correlation loss function is used to train the feature mapping network; Specifically, the method comprises the following steps: Step 1, first input a small sample public data set with similar features to railway image features into a randomly initialized feature mapping network for pre-training to obtain a pre-trained feature mapping network and its parameters; Step 2, use the small sample public data set to establish a meta-training set, input it into the pre-trained feature mapping network obtained in step 1 for metric meta-training to obtain a meta-trained feature mapping network and its parameters; Step 3, establish a small sample support set S from railway scene images and intrusion target sample images, input it into the meta-trained feature mapping network obtained in step 2, and use the meta-trained feature mapping network to map the above intrusion target samples to the same feature space; Step 4, input the image features of the support set obtained in step 3 into the self-learning class center fine-tuning network to learn the center representation of each image class; Step 5, input the test image into the meta-trained feature mapping network obtained in step 2 to obtain the image features of the test image, and then use the distance metric module to calculate the similarity between the test image features and the center of each image class obtained in step 4 to obtain the intrusion target identification result; The self-learning class center fine-tuning network processing algorithm steps are as follows: Step 2-1, for the support set S The middle category is k All samples Features are obtained after passing through a feature mapping network. The mean value is used as a learnable parameter class. k The initial value of the central prototype in the feature space is as follows (1): ； wherein, N the total number of samples of the class k , is the k th sample of the class i , feature obtained after the feature mapping network Step 2-2, get the support set according to formula (1) in turn S all the initial values of the class center prototypes, normalize the distance from the samples in the support set to each initial class center using the softmax function, and get the probability of belonging to the class k , as formula (2): ； wherein d is the Euclidean distance of the two feature vectors in the feature space; Step 2-3, calculate the cross-entropy loss according to formula (3): 。 2. The method of claim 1, wherein the method is based on small sample learning. The feature mapping network in step 1 is a feature mapping network based on a channel attention mechanism, which consists of two parts: The first part is composed of four residual blocks connected in turn, and each residual block is followed by a max pooling layer; The second part is an improved channel attention module.

3. The method of claim 2, wherein the method is based on small sample learning. The improved channel attention module includes: The information compression part compresses the features after convolution through a parallel global max-pooling layer and a global average-pooling layer in the spatial dimension; Information incentive part The features of the global maximum pooling layer and the global average pooling layer are added and fused, and the fused features are normalized to 0-1 by using an activation function to obtain a channel attention weight value. Channel feature enhancement The obtained channel attention weight value is multiplied with the original feature value F, so as to realize attention enhancement on the key channel.

4. The method of claim 2, wherein the method is based on small sample learning of railway scene unusual intrusion target recognition. The initial network parameters of the feature mapping network are obtained by using a model pre-training method, and then meta-training is performed, and the specific training steps are as follows: Step 1-1, train a classifier on the feature mapping network based on the channel attention mechanism using the standard cross-entropy loss, and use the meta-validation method during training to save the feature mapping network with the highest meta-validation accuracy under the N-way K-shot setting as the pre-trained model; Step 1-2, extract the network parameters of the pre-trained model and migrate them to the metric meta-learning, and continue to train the network using meta-training.

5. The method of claim 1, wherein the method is based on small sample learning. The feature mapping network is trained using a center correlation loss function, specifically: center loss L c The expression is specifically as follows formula (4): ； In the above formula λ is a hyper-parameter that can be used to control the distance within the class during training; represents the feature value of a certain intrusion target sample in the feature space, c k represents the class center of each image class; At the same time, during the training of the feature mapping network, the class centers are reinitialized using the mean value of the support set features at each iteration; Finally, the cross-entropy loss and the center loss are combined to construct the center correlation loss function represented by formula (5) for feature mapping network training: 。

Images