Training method of railway intrusion target classification model, electronic equipment and storage medium

Abstract

The invention provides a training method of a railway intrusion target classification model, electronic equipment and a storage medium, and aims at each railway intrusion target category in a training sample set. Iterating the following steps for a specified round to train and generate a railway intrusion target classification model: extracting a first feature extraction result of the query sample image and a second feature extraction result of each support sample image through a feature extraction layer of the to-be-trained model; correcting the second feature extraction result to obtain a third feature extraction result; determining a probability value that the support sample image belongs to an actual railway intrusion target category; and performing back propagation based on the probability value and a cross entropy loss function to update a class center matrix formed by class centers corresponding to all railway intrusion target classes, so as to train and generate a railway intrusion target classification model with accurate recognition under the condition that the open total number of classification classes is not fixed with rare samples. And the module has good expansibility for new classification categories.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a training method, electronic device and storage medium for a railway intrusion target classification model. Background Art

[0002] With the rapid development of the railway industry, perimeter intrusions such as natural disasters, illegal entry of personnel, and animal activities occur from time to time, which may lead to serious consequences such as derailment and overturning of trains, posing a serious threat to people's life and property safety. Therefore, accurate perception of railway perimeter intrusion target categories is particularly important. It can provide semantic prior knowledge for subsequent more refined pixel-level segmentation, which helps staff to take corresponding preventive measures in time for different intrusion categories, thereby reducing the accident rate.

[0003] However, in actual railway scenarios, there are many types of intrusion targets and their distribution is uneven, and the number of samples in some categories is relatively small. In this case, traditional deep learning methods are difficult to deal with effectively because they rely on a large number of samples, which makes the accurate identification of specific intrusion targets in railway scenarios an urgent problem to be solved. Summary of the invention

[0004] The purpose of the embodiments of the present application is to provide a training method, electronic device and storage medium for a railway intrusion target classification model, which is used to train and generate an accurate railway intrusion target classification model under the condition that the total number of open classification categories is not fixed and the samples are scarce, and the module has good scalability to new classification categories.

[0005] In a first aspect, the present invention provides a method for training a railway intrusion target classification model, the method comprising:

[0006] For each railway intrusion target category in the training sample set, the following steps are iterated for a specified number of rounds to train and generate a railway intrusion target classification model: for a sample group after labeling, the sample group is input into the input layer of the model to be trained, and the first feature extraction result of the query sample image and the second feature extraction result of each support sample image are extracted through the feature extraction layer of the model to be trained. The sample group includes 1 query sample image and k support sample images corresponding to the nth railway intrusion target category; based on the similarity between all the second feature extraction results, the second feature extraction result is corrected to obtain the third feature extraction result; based on the distance value between the third feature extraction result and the class center corresponding to each railway intrusion target category, the probability value of the support sample image belonging to the actual railway intrusion target category is determined; back propagation is performed based on the probability value and the cross entropy loss function to update the class center matrix composed of the class centers corresponding to all railway intrusion target categories; it is determined whether all the support sample images corresponding to the actual railway intrusion target category in the support sample set participate in the training; if not, the sample group is re-determined, and the step of inputting the sample group into the input layer of the model to be trained is returned; if yes, it is determined that this round of training is completed.

[0007] In an optional embodiment, the input layer includes a large convolution unit, a normalization unit, an activation unit and a pooling unit connected in sequence, the feature extraction layer includes a first fusion residual module, a second fusion residual module, a third fusion residual module, and a fourth fusion residual module connected in sequence, and each fusion residual module includes a residual unit and a CBAM unit connected in sequence.

[0008] In an optional embodiment, the first feature extraction result includes the query feature output by the fourth fusion residual module, and the second feature extraction result includes the first supporting feature output by the second fusion residual module, the second supporting feature output by the third fusion residual module, and the third supporting feature output by the fourth fusion residual module.

[0009] In an optional implementation manner, the third feature extraction result corresponding to each supporting sample image is obtained in the following manner:

[0010] Based on the similarity between the first feature extraction result and the feature extraction mean result corresponding to the current support sample image, a similarity fusion matrix is ​​constructed; for the similarity value at each position in the similarity fusion matrix, it is determined whether the similarity value at the position is greater than a preset value; if it is less than the preset value, the feature value at the corresponding position in the third support feature corresponding to the current support sample image is multiplied by the preset coefficient to obtain the third feature extraction result corresponding to the current support sample image.

[0011] In an optional implementation, the similarity fusion matrix is ​​constructed in the following manner:

[0012] Calculate the means of the first support feature, the second support feature and the third support feature corresponding to the remaining sample images except the current support sample image in the K support sample images respectively to obtain the first average feature, the second average feature and the third average feature; calculate the similarity value at the same position between the first average feature and the first support feature by the cosine similarity algorithm to construct a first similarity submatrix; calculate the similarity value at the same position between the second average feature and the second support feature by the first similarity algorithm to construct a second similarity submatrix; calculate the similarity value at the same position between the third average feature and the third support feature by the second similarity algorithm to construct a third similarity submatrix; input the first similarity submatrix, the second similarity submatrix and the third similarity submatrix into the fully connected module to obtain the similarity fusion matrix output by the fully connected module.

[0013] In an optional implementation, the network parameters of the module to be trained are optimized and updated based on the main loss function, where the main loss function L is:

[0014] L=L con +L cyc ;

[0015]

[0016] L cyc =L 1 (F A ,cyc(F Q ))+L 2 (C j ,cyc(C j ))+L 3 (F Q ,f BA (C j ))+

[0017] L 4 (C j ,f AB (F Q ))+L d (d S ,C j ,f AB (F Q ))+L d (d Q , F Q ,f BA (C j );

[0018] cyc(F Q )=f BA (f AB (F Q ));

[0019] Among them, L con is the distance loss function, L cyc is the cycle-consistent loss function, F Q To query the feature extraction results, c i is the class center corresponding to the query sample image in the test sample set, f AB 、f BA To map the network, d S d Q is the similarity detection neural network model, and sim(·) is the similarity calculation.

[0020] In an optional embodiment, the method further includes performing a standardized calculation on the similarity between the predicted value and the true value before the step of calculating the main loss function.

[0021] In an optional embodiment, it also includes:

[0022] Based on the collected railway monitoring images, a training sample set and a test sample set are constructed, wherein the training sample set includes one query sample image and K support sample images corresponding to multiple first railway intrusion targets;

[0023] One query sample image and K support sample images corresponding to multiple second railway intrusion targets in the test sample set;

[0024] Among them, the first railway invasion target and the second railway invasion target are of different categories.

[0025] In a second aspect, the present invention provides an electronic device, comprising: a processor, a memory and a bus, the memory storing machine-readable instructions executable by the processor, and when the electronic device is running, the processor and the memory communicate through the bus, and the processor executes the machine-readable instructions to perform the steps of the training method of the railway intrusion target classification model as described in any of the aforementioned implementations.

[0026] In a third aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of the method for training a railway intrusion target classification model in any of the aforementioned implementations are executed.

[0027] The present application provides a training method, electronic device and storage medium for a railway intrusion target classification model. The method includes iterating the following steps to specify rounds for each railway intrusion target category in a training sample set to train and generate a railway intrusion target classification model: for each railway intrusion target category in a training sample set, iterating the following steps to specify rounds for training and generate a railway intrusion target classification model: for a labeled sample group, inputting the sample group into the input layer of the model to be trained, and extracting the first feature extraction result of the query sample image and the second feature extraction result of each supporting sample image through the feature extraction layer of the model to be trained, the sample group includes 1 query sample image corresponding to the nth railway intrusion target category and k supporting sample images image; based on the similarity between all the second feature extraction results, the second feature extraction result is corrected to obtain the third feature extraction result; based on the distance value between the third feature extraction result and the class center corresponding to each railway intrusion target category, the probability value of the support sample image belonging to the actual railway intrusion target category is determined; based on the probability value and the cross entropy loss function, back propagation is performed to update the class center matrix composed of the class centers corresponding to all railway intrusion target categories; determine whether all the support sample images corresponding to the actual railway intrusion target category in the support sample set are involved in the training; if not, the sample group is re-determined, and the step of inputting the sample group into the input layer of the model to be trained is returned; if so, it is determined that this round of training is completed. It is used to train and generate a railway intrusion target classification model that can accurately identify the classification of samples with sparse samples and an unfixed total number of open classification categories, and the module has good scalability to new classification categories. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments of the present application will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0029] Figure 1 A feature extraction neural network structure diagram provided in an embodiment of the present application;

[0030] Figure 2 A structural diagram of a CBAM unit provided in an embodiment of the present application;

[0031] Figure 3 A flowchart of a simulation task provided in an embodiment of the present application;

[0032] Figure 4 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0033] With the rapid development of the railway industry, the total operating mileage of railways has exceeded 160,000 kilometers, and the operating mileage of urban rail has exceeded 10,000 kilometers. In the vast railway area, perimeter intrusions such as natural disasters, illegal entry of personnel, and animal activities often occur, which may cause serious consequences such as derailment and overturning of trains, posing a serious threat to people's life and property safety. Therefore, the precise perception of railway perimeter intrusion target categories can be further used to provide semantic prior knowledge for subsequent more refined pixel-level segmentation, which can assist staff to take corresponding preventive measures in time for different intrusion categories, eliminate potential dangers, and reduce the probability of such accidents.

[0034] Machine vision technology has developed rapidly in recent years and has also performed well in the field of railway intrusion target perception. At present, most of the work is divided into three directions: target classification, target detection, and semantic segmentation. Target classification identifies and classifies targets in images or videos, extracts and analyzes features of input images or videos through machine learning models or deep learning models, and predicts which category the image belongs to.

[0035] At present, the main common methods of target classification focus on relatively common categories such as pedestrians. However, in actual railway scenarios, there are many types of intrusion targets, and their distribution is uneven. The number of samples of some categories is very scarce compared to common categories. In this case, the traditional deep learning method that originally used a large number of samples for training is no longer applicable.

[0036] That is, the railway intrusion target classification model training mainly faces the following two problems:

[0037] (1) The problem of difficulty in detecting the long-tail distribution of railway intrusion targets. Due to the large regional span of my country's railway network, railway intrusion targets show the characteristics of long-tail distribution, which is specifically reflected in the following two aspects: On the one hand, the number of targets is unevenly distributed. Most of the existing machine vision-based methods are aimed at common types such as railway workers and pedestrians crossing the road, ignoring relatively rare types such as equally dangerous fallen rocks and trackside dumping. On the other hand, it is difficult to obtain samples of railway intrusion targets under real operating conditions. This is the case for relatively common illegal pedestrians and animals, and other samples at the tail end of the distribution, such as natural disaster relics, are even more difficult to obtain. Existing methods mostly use simulated intrusions to alleviate "data hunger", but simulations can only be performed during the non-operating period of the railway, and cannot cover a wide range of railway intrusion target sample types, making it difficult to meet the large-scale data requirements of deep learning models.

[0038] (2) The problem of open categories of railway intrusion targets and an unfixed total number. In the complex environment along the railway, the types of intrusion targets are open, and new and unprecedented target categories may appear at any time, such as new animal species, accidental falling objects, equipment failures, or obstacles caused by natural disasters. Existing railway intrusion detection methods mostly use fixed category training methods, which makes it difficult to quickly identify and respond to new target types. Once a new type is added, on the one hand, a large amount of resources will be consumed to adjust the model additionally, and on the other hand, it is difficult to cope with the unlimited increase of new types. This results in a significant decrease in detection effectiveness when the system faces new categories or complex scenarios in actual operation, making it difficult to cope with the diverse security threats along the railway in a real operating environment.

[0039] Based on this, the present application provides a training method, electronic device and storage medium for a railway intrusion target classification model.

[0040] The technical solutions in the embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application.

[0041] Embodiment 1

[0042] In one embodiment of the present application, a small sample classification method is used to improve the generalization ability of the model through meta-learning. In this embodiment, the railway intrusion target classification model can be trained and generated through the following steps:

[0043] 1. Sample Collection and Division

[0044] Constructing a training sample set and a test sample set based on the collected railway monitoring images, wherein the training sample set includes one query sample image and K support sample images corresponding to a plurality of first railway intrusion targets;

[0045] One query sample image and K support sample images corresponding to multiple second railway intrusion targets in the test sample set;

[0046] Among them, the first railway invasion target and the second railway invasion target are of different categories.

[0047] Based on the collected railway monitoring images, the training set D is divided train and the test set D test , which respectively contain the category set C used for training train and the set of categories C used for testing test , where C train With C test The categories of railroad intrusion targets are completely disjoint In D train The trained model is directly used in D rest Test on.

[0048] 2. Model Framework Construction

[0049] The framework of the model to be trained here may include an input layer, a feature extraction layer, and a classification layer.

[0050] Among them, Figure 1 As shown, the input layer conv1 includes a large convolution unit, a normalization unit (batch normalization unit), an activation unit (ReLU activation unit), and a pooling unit (max pooling layer) connected in sequence.

[0051] In order to enable the model to adaptively adjust its focus on different channels and spatial positions of the features, so as to better capture the local and global information of the samples, the feature extraction layer includes the first fused residual module (ResCBAM1), the second fused residual module (ResCBAM2), the third fused residual module (ResCBAM3), and the fourth fused residual module (ResCBAM4) connected in sequence. Each fused residual module includes residual units (conv2_x, conv3_x, conv4_x, conv5_x) and CBAM units connected in sequence. Among them, the bottleneck block structure of each residual unit is different, but they are all reduced in dimension by 1×1 convolution, followed by general convolution to process features, and finally implemented by 1×1 convolution to increase in dimension. The structure of each CBAM unit can be as follows Figure 2 As shown in the figure. The CBAM module applies channel and spatial attention at the same time, first calculating the channel attention, and then weighted summing it in the spatial dimension, which has a more refined effect. For image data, different channels may capture different information such as color, texture, or edge. Through channel attention, the model can selectively amplify more critical channel signals and reduce redundant information. Spatial attention assigns different weights to different spatial positions on each channel, allowing the model to focus more on more distinguishable areas in the image, such as edges and corners with rich features.

[0052] Through such a feature extraction layer, key feature areas and channels can be enhanced.

[0053] The classification layer is used to determine the railway intrusion target category with the greatest similarity as the railway intrusion target category label of the output query sample according to the similarity between the query features of the query sample image and the class center corresponding to each railway intrusion target category.

[0054] 3. Model training and optimization

[0055] For the training set D train For each railway intrusion target category in , a corresponding training task can be created: the sample group of each simulated training task consists of k supporting sample images {I s} and 1 query sample image, both of which contain objects of the same type, expressed as:

[0056] one k-shotepisode={{I s} k ,I q}

[0057] Among them, k is smaller than the training set D train The total number of sample images supported by this category.

[0058] In this way, when the number of simulated training tasks for the training set and the test set is N train 、N test When , the training set and test set can be expressed as:

[0059]

[0060] During training, the categories of both support sample images and query sample images can be viewed to calculate losses and optimize model parameters, while during testing, only the categories of support sample images can be viewed.

[0061] Figure 3 A flowchart of a simulation task provided in an embodiment of the present application. Figure 3 As shown, for the training set D train For each railway invasion target category, each round of simulation tasks can be completed by the following steps:

[0062] S100, determine the current sample group, including k randomly selected supporting sample images {I s} and 1 query sample image, where the support sample image and the query sample image contain the same type of railway intrusion targets. The railway intrusion targets in the sample group are annotated with railway intrusion target labels.

[0063] S101, for a sample group after annotation, input the sample group into the input layer of the model to be trained, and extract the first feature extraction result of the query sample image and the second feature extraction result of each supporting sample image through the feature extraction layer of the model to be trained. The sample group includes 1 query sample image and k supporting sample images corresponding to the nth railway intrusion target category.

[0064] In the process of feature extraction, the intermediate features of the feature extraction network also contain rich information. Using only the features of the final output layer will cause a lot of information loss. For example, the underlying features usually capture the underlying information of the image, such as edges, textures, and colors; high-level features focus more on the overall shape and semantic information of the object. All of this information may be useful for classification. Therefore, the first feature extraction result includes the third query feature F output by the fourth fusion residual module. Q , the second feature extraction result includes the first supporting feature output by the second fusion residual module The second support feature output by the third fusion residual module The third supporting feature output by the fourth fusion residual module

[0065] S102: based on the similarities between all the second feature extraction results, modify the second feature extraction results to obtain a third feature extraction result.

[0066] In step S102, since the number of samples in each category is small, abnormal samples will have a great impact on the subsequent determination of the prototype center, so the extracted third supporting features need to be Correction is performed to obtain F s ′ .

[0067] S103: Based on the distance value between the third feature extraction result and the class center corresponding to each railway intrusion target class, determine the probability value that the supporting sample image belongs to the actual railway intrusion target class.

[0068] In step S103, assuming that the space corresponding to each category is around a certain class center, the initial value of the corresponding class center can be obtained by obtaining the mean of the support sample set in the feature space, and then the class center is fine-tuned, so that the classification problem is transformed into a problem of obtaining the closest distance in the feature space. The idea of ​​fine-tuning the class center is to treat the class center as a set of learnable parameters (class center matrix, composed of class centers corresponding to all railway intrusion target categories) and use an optimizer to learn the class center matrix.

[0069] Specifically, for N railway intrusion target categories, each support sample set contains K images. Assume that the support set S of the nth category is n for where x i refers to the i-th support sample image, x i ∈R D ,y i is the corresponding railway intrusion target label. Using the embedding function Map the D-dimensional support sample image to the M-dimensional feature space. At this time, the support sample feature can be expressed as The mean of all support sample feature vectors is used as the class center c of the learnable parameter class n n The initial class center in the feature space is expressed as:

[0070]

[0071] According to the above formula, the initial class centers of all railway intrusion target categories in the support set are obtained in turn.

[0072] The third support feature F after the support sample image is corrected by normalizing it using the Softmax function s ′ The distance to the center of each class is used to obtain the probability that it belongs to the real railway intrusion target label n. Right now:

[0073]

[0074] Among them, d is the distance calculation, c n is the class center of the current railway intrusion target category, c j is the current class center corresponding to the j-th railway intrusion target.

[0075] S104. Back propagation is performed based on the probability value and the cross entropy loss function to update a class center matrix composed of class centers corresponding to all railway intrusion target categories.

[0076] The cross entropy loss function here can be expressed as:

[0077]

[0078] Here, we can use small batch stochastic gradient descent to back-propagate the cross entropy loss function to complete the update of the class center matrix.

[0079] S105: Determine whether all support sample images in the support sample set corresponding to the actual railway intrusion target label participate in the training.

[0080] S106: If not, redetermine the sample group and return to the step of inputting the sample group into the input layer of the model to be trained.

[0081] The support set S of the nth railway intrusion target category n There are K support sample images in total, and k of them are used in the first class center fine-tuning. In the second class center fine-tuning, k images need to be reselected until all support sample images of the nth category are involved in the fine-tuning.

[0082] S107: If yes, determine that this round of training is completed.

[0083] In this way, a training task is completed for the nth railway intrusion target category, and each railway intrusion target category can be cyclically executed 100 times. After all railway intrusion target categories are traversed, the final railway intrusion target classification model can be obtained.

[0084] In a specific application scenario, the railway intrusion target classification model can take as input a railway monitoring image and output the labels of all railway intrusion targets contained in the railway monitoring image.

[0085] The training method of the railway intrusion target classification model provided in this embodiment of the present application has significant advantages in dealing with the long-tail distribution detection and category openness problems of railway intrusion targets. First, the task-based small sample learning training mode is adopted to improve the detection ability of the model on long-tail distribution targets, and can effectively identify low-frequency targets including fallen rocks and trackside dumping, making up for the existing methods' neglect of tail data. Only a small amount of category annotation is required to adjust the model, which greatly saves the cost of annotation and model adjustment. This not only improves the model's recognition accuracy for rare dangerous targets, but also enhances the robustness of the system when sample data is limited, and reduces dependence on simulated intrusion samples.

[0086] In addition, the algorithm is designed for the open characteristics of railway intrusion target categories, so that the system has dynamic expansion capabilities. When a new railway intrusion target sample appears, it can be judged whether it is a new category based on the similarity gap with the known class center. There is no need to retrain each new category. It only needs to collect a small number of supporting sample images corresponding to the new category and determine the corresponding class center. This advantage significantly enhances the adaptability and generalization ability of the system, enabling it to effectively deal with complex and changeable intrusion targets in the actual operating environment, providing more flexible and efficient technical support for railway security.

[0087] Embodiment 2

[0088] In one embodiment of the present application, a multi-level similarity feature correction step is proposed for the multi-level feature extraction in step S101.

[0089] Specifically, in step S102, the third feature extraction result corresponding to each supporting sample image can be obtained in the following manner:

[0090] First, based on the similarity between the first feature extraction result and the feature extraction mean result corresponding to the current support sample image, a similarity fusion matrix is ​​constructed. Specifically, it includes:

[0091] The means of the first support feature, the second support feature and the third support feature corresponding to the remaining sample images in the K support sample images except the current support sample image are respectively calculated to obtain the first average feature, the second average feature and the third average feature.

[0092] The similarity value at the same position between the first average feature and the first supporting feature is calculated by the cosine similarity algorithm to construct a first similarity submatrix. The similarity value at the same position between the second average feature and the second supporting feature is calculated by the first similarity algorithm to construct a second similarity submatrix.

[0093] For low-level features, simple metrics such as cosine similarity can be used to calculate similarity.

[0094] The similarity value at the same position between the third average feature and the third supporting feature is calculated by using the second similarity algorithm to construct a third similarity sub-matrix.

[0095] For high-level features, weighted calculation combined with attention mechanism can be used for measurement.

[0096] The first similarity submatrix, the second similarity submatrix, and the third similarity submatrix are input into a fully connected module to obtain a similarity fusion matrix output by the fully connected module.

[0097] Finally, a small fully connected network can be used to adaptively fuse the similarities at different levels to obtain the fused similarity matrix D∈R WH×WH , each eigenvalue in the matrix is ​​used to indicate the similarity between the current support sample image and the remaining support sample images at the corresponding position.

[0098] For the similarity value at each position in the similarity fusion matrix, determine whether the similarity value at the position is greater than a preset value. If less than the preset value, multiply the feature value at the corresponding position in the third support feature corresponding to the current support sample image by a preset coefficient to obtain the third feature extraction result corresponding to the current support sample image.

[0099] If the eigenvalue D at position (i, j) in D ij is greater than or equal to the preset value, then the feature value at the position (i, j) in the third support feature corresponding to the current support sample image remains unchanged. ij If it is less than the preset value, the feature value at the position (i, j) in the third support feature corresponding to the current support sample image is multiplied by the coefficient α, α∈(0,1). Then, all images in the support set are traversed to adjust the third support features corresponding to all support sample images in turn.

[0100] By making unsupervised corrections to the third supporting features, the impact of abnormal samples on subsequent training can be reduced.

[0101] Embodiment 3

[0102] In one embodiment of the present application, for a sample group in a prediction sample set, the network parameters may be optimized and updated according to a loss function between a prediction result of a query sample and a true value.

[0103] Specifically, the network parameters of the module to be trained are optimized and updated based on the main loss function, wherein the main loss function L is:

[0104] L=L con +Lcyc ;

[0105]

[0106] Among them, L con is the distance loss function, L cyc is the cycle-consistent loss function, F Q To query the feature extraction results, c i is the class center corresponding to the query sample image in the test sample set, f AB 、f BA To map the network, d s d Q is the similarity detection neural network model, and sim(·) is the similarity calculation.

[0107] Here, at the beginning of training, the standard deviation between the features is large, which may cause the problem of gradient explosion. In order to make the training process more stable, the similarity can be standardized before calculating the loss. The specific operation can be expressed as:

[0108]

[0109] Among them, L cyc is a generative cycle-consistent loss. Q By mapping the network f AB Mapped to the prototype feature space B to obtain the intermediate generated feature f AB (F Q ), and then through the mapping network f BA Mapping back to the query feature space A, we get the cyclically generated feature cyc(F Q ) = f BA (f AB (F Q )), then cyc(F Q ) should be consistent with the actual F Q Be consistent.

[0110] Similarly, the class center C j After mapping network f BA and f AB After the mapping, the cyclically generated feature cyc(C j ) = f AB (f BA (C j )) should be consistent with the actual C j Be consistent.

[0111] In addition, the generated features belonging to the same space should also be consistent with the original features of the space, such as f AB (F q ) and C j. Therefore, the proposed generative cycle-consistent loss function is:

[0112] L cyc =L 1 (F Q ,cyc(F Q ))+L 2 (C j ,cyc(C j ))+L 3 (F Q ,f BA (C j ))+

[0113] L 4 (C j ,f AB (F Q ))+L d (d S ,C j ,f AB (F Q ))+L d (d Q , F Q ,f BA (C j );

[0114] cyc(F Q ) = f BA (f AB (F Q ));

[0115] Among them, L 1 , L 2 It is used to indicate the loss of original features and cycle-generated features, L 3 , L 4 It is used to indicate the loss of original features and intermediate generated features. d For the constraint d S d Q Learning function.

[0116] Embodiment 4

[0117] See also Figure 4 , Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Figure 4 As shown in , the electronic device 400 includes a processor 410 , a memory 420 and a bus 430 .

[0118] The memory 420 stores machine-readable instructions executable by the processor 410. When the electronic device 400 is running, the processor 410 communicates with the memory 420 via the bus 430. When the machine-readable instructions are executed by the processor 410, the steps of a training method for a railway intrusion target classification model in the above method embodiment can be executed. The specific implementation method can be found in the method embodiment, which will not be repeated here.

[0119] An embodiment of the present application also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of a training method for a railway intrusion target classification model in the above method embodiment can be executed. The specific implementation method can be found in the method embodiment, which will not be repeated here.

[0120] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0121] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0122] In addition, the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0123] Furthermore, the functional modules in the various embodiments of the present application may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.

[0124] It should be noted that if the function is implemented in the form of a software function module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can essentially be embodied in the form of a software product, or the part that contributes to the prior art or the part of the technical solution. The computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM) random access memory (RAM), disk or optical disk, and other media that can store program codes.

[0125] In this document, relational terms such as first and second, etc. are used merely to distinguish one entity or operation from another entity or operation, but do not necessarily require or imply any such actual relationship or order between these entities or operations.

[0126] The above description is only an embodiment of the present application and is not intended to limit the protection scope of the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A training method for a railway intrusion target classification model, characterized in that: The method comprises: For each railway intrusion target category in the training sample set, iterate the following steps to specify the rounds to train and generate a railway intrusion target classification model: For a sample group after annotation, the sample group is input into the input layer of the model to be trained, and the first feature extraction result of the query sample image and the second feature extraction result of each supporting sample image are extracted through the feature extraction layer of the model to be trained, wherein the sample group includes 1 query sample image and k supporting sample images corresponding to the nth railway intrusion target category; Based on the similarities between all the second feature extraction results, the second feature extraction results are modified to obtain a third feature extraction result; Based on the distance value between the third feature extraction result and the class center corresponding to each railway intrusion target class, determining the probability value of the supporting sample image belonging to the actual railway intrusion target class; Back propagation is performed based on the probability value and the cross entropy loss function to update a class center matrix composed of class centers corresponding to all railway intrusion target categories; Determine whether all support sample images in the support sample set corresponding to the actual railway intrusion target category are involved in training; If not, the sample group is re-determined, and the step of inputting the sample group into the input layer of the model to be trained is returned to be executed; If yes, then this round of training is completed.

2. The method according to claim 1, characterized in that The input layer includes a large convolution unit, a normalization unit, an activation unit and a pooling unit connected in sequence, the feature extraction layer includes a first fusion residual module, a second fusion residual module, a third fusion residual module, and a fourth fusion residual module connected in sequence, and each fusion residual module includes a residual unit and a CBAM unit connected in sequence.

3. The method according to claim 2, characterized in that The first feature extraction result includes the query feature output by the fourth fusion residual module, The second feature extraction result includes the first supporting feature output by the second fusion residual module, the second supporting feature output by the third fusion residual module, and the third supporting feature output by the fourth fusion residual module.

4. The method according to claim 3, characterized in that The third feature extraction result corresponding to each supporting sample image is obtained in the following way: Based on the similarity between the first feature extraction result and the feature extraction mean result corresponding to the current support sample image, construct a similarity fusion matrix; For the similarity value at each position in the similarity fusion matrix, determining whether the similarity value at the position is greater than a preset value; If it is less than the preset value, the feature value at the corresponding position in the third supporting feature corresponding to the current supporting sample image is multiplied by the preset coefficient to obtain the third feature extraction result corresponding to the current supporting sample image.

5. The method according to claim 4, characterized in that The similarity fusion matrix is ​​constructed in the following way: Respectively calculate the means of the first support feature, the second support feature and the third support feature corresponding to the remaining sample images in the K support sample images except the current support sample image to obtain the first average feature, the second average feature and the third average feature; Calculate the similarity value at the same position between the first average feature and the first supporting feature by using a cosine similarity algorithm to construct a first similarity submatrix; Calculating the similarity value at the same position between the second average feature and the second supporting feature by using the first similarity algorithm to construct a second similarity submatrix; Calculating the similarity value at the same position between the third average feature and the third supporting feature by using the second similarity algorithm to construct a third similarity submatrix; The first similarity submatrix, the second similarity submatrix, and the third similarity submatrix are input into a fully connected module to obtain a similarity fusion matrix output by the fully connected module.

6. The method according to claim 1, characterized in that The network parameters of the module to be trained are updated based on the optimization of the main loss function, wherein the main loss function L is: L=L con +L cyc ; L cyc =L1(F Q ,cyc(F Q ))+L2(C j ,cyc(C j ))+L3(F Q ,f BA (C j ))+ L4(C j ,favorite AB (FAVORITE Q ))+L d (d) S ,C j ,favorite AB (FAVORITE Q ))+L d (d) Q ,FAVORITE Q ,favorite BA (C) j ); cyc(F Q )=f BA (f AB (F Q )); Among them, L con is the distance loss function, L cyc is the cycle-consistent loss function, F Q To query the feature extraction results, c i is the class center corresponding to the query sample image in the test sample set, f AB 、f BA To map the network, d S d Q is the similarity detection neural network model, and sim(·) is the similarity calculation.

7. The method according to claim 6, characterized in that It also includes a step of normalizing the similarity between the predicted value and the true value before calculating the main loss function.

8. The method according to claim 1, characterized in that: Also includes: Constructing a training sample set and a test sample set based on the collected railway monitoring images, wherein the training sample set includes one query sample image and K support sample images corresponding to a plurality of first railway intrusion targets; One query sample image and K support sample images corresponding to multiple second railway intrusion targets in the test sample set; Among them, the first railway invasion target and the second railway invasion target are of different categories.

9. An electronic device, characterized in that: include: A processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor communicates with the memory via the bus, and the processor executes the machine-readable instructions to perform the steps of the training method of the railway intrusion target classification model as described in any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method for training a railway intrusion target classification model as claimed in any one of claims 1 to 8 are executed.

Images