A railway scene target detection method based on few-shot learning

Abstract

The application provides a railway scene target detection method based on few-shot learning. The method comprises the following steps: acquiring railway scene image samples, constructing a basic dataset, a generated dataset and a fine-tuned dataset; constructing a convolutional neural network model for railway scene target detection, training the convolutional neural network model using the basic dataset, the generated dataset and the fine-tuned dataset in sequence, and obtaining a trained convolutional neural network model; inputting a railway scene image to be identified into the trained convolutional neural network prediction model, and outputting a target identification result of the railway scene image to be identified by the trained convolutional neural network prediction model. The application proposes a training framework combining sample generation and image style transfer for secondary fine-tuning, improves the accuracy of railway few-shot target detection from the perspective of data enhancement and transfer learning, avoids the condition that a complex model occupies many computer resources, and has the advantages of small running memory, fast detection speed and the like.

Description

Technical Field

[0001] This invention relates to the field of few-shot target detection technology, and in particular to a railway scene target detection method based on few-shot learning. Background Technology

[0002] With the large-scale speed increase and continuous increase in operating mileage of railways, ensuring train operation safety has become the top priority of the railway system. Foreign object intrusion is one of the important factors threatening railway operation safety. Due to the high speed and long braking distance of trains, once a foreign object intrudes into the clearance gauge during operation, it will seriously affect normal operation and even cause tragic loss of life and property.

[0003] Railway target detection is a crucial step in solving the problem of foreign object intrusion. Compared to traditional manual inspection methods that rely on a large workforce, computer vision-based target detection methods are characterized by low cost and high efficiency. Through real-time monitoring and timely warnings, train accidents caused by foreign object intrusion can be effectively avoided, further ensuring railway transportation order and passenger safety.

[0004] In recent years, with the rapid development of computer vision, railway target detection technology has gradually matured, achieving a high level of detection for common target categories such as people, vehicles, and large rocks. However, existing deep learning-based railway target detection technologies are mostly based on learning from a large number of samples, while research on the detection of railway targets with fewer samples, such as kites, balloons, and bird nests, is scarce.

[0005] Currently, one existing method for target detection in railway scenes with limited sample sizes involves extracting features from scarce sample resources using a more complex network structure. However, this method suffers from drawbacks such as excessive computer resource consumption, slow detection speed, and low efficiency, failing to meet the real-time detection and timely warning requirements of railway scenarios. Summary of the Invention

[0006] Embodiments of the present invention provide a railway scene target detection method based on few-shot learning, so as to effectively detect targets in railway scene images.

[0007] To achieve the above objectives, the present invention adopts the following technical solution.

[0008] A railway scene target detection method based on few-shot learning includes:

[0009] Acquire railway scene image samples, construct a basic dataset, generate a dataset, and fine-tune the dataset;

[0010] A convolutional neural network model for target detection in railway scenes is constructed. The convolutional neural network model is trained sequentially using the base dataset, the generated dataset, and the fine-tuning dataset to obtain a trained convolutional neural network model.

[0011] The railway scene image to be identified is input into the trained convolutional neural network prediction model, and the trained convolutional neural network prediction model outputs the target recognition result of the railway scene image to be identified.

[0012] Preferably, the acquisition of railway scene image samples, construction of a basic dataset, generation dataset, and fine-tuning dataset includes:

[0013] Obtain a public dataset containing a certain number of railway scene image samples, and use the public dataset as the base dataset;

[0014] The generated dataset is obtained by processing railway scene images using a sample generation method and an image style transfer method. The generated dataset consists of a portion of the data in the base dataset and the generated data. The sample generation method refers to adding a small number of target objects to railway scene images without targets. The image style transfer refers to performing weather transfer and seasonal transfer processing on the images. By adding rain, fog, and snow to the images, railway scene images under severe weather conditions are generated. By changing the season of the images, railway scene images under different seasons are generated.

[0015] The fine-tuning dataset consists of a small amount of data from the base dataset and a small sample of real railway target data.

[0016] Preferably, the public dataset is the VOC2007 dataset containing samples of 20 categories. The generated data includes four categories: kites, bird nests, balloons, and plastic film. The generated data and a portion of the VOC2007 data together constitute the generated dataset. A small number of target data samples of kites, bird nests, balloons, and plastic film in real railway scenes are collected and together with a small amount of data from a portion of the VOC2007 dataset constitute the fine-tuning dataset.

[0017] Preferably, the construction of the convolutional neural network model for target detection in railway scenes includes:

[0018] A convolutional neural network model for target detection in railway scenes is constructed. This convolutional neural network model is Faster R-CNN. The Faster R-CNN network model consists of four parts: a feature extraction network, a region candidate network (RPN), a region of interest pooling layer, and a fully connected layer. The training process is as follows: the feature extraction network extracts features from the input image to obtain a feature map; the region candidate network generates multiple regions of interest; the region of interest output by the region candidate network and the feature map output by the feature extraction network are combined by the region of interest pooling layer to obtain a region feature map of a fixed size; finally, classification and regression are performed by the fully connected layer.

[0019] Preferably, the step of training the convolutional neural network model sequentially using the base dataset, the generated dataset, and the fine-tuning dataset to obtain the trained convolutional neural network model includes:

[0020] The convolutional neural network model is trained using the aforementioned basic dataset to obtain basic training weights, enabling the convolutional neural network model to learn generalized feature representations of railway scene image samples. The number of categories corresponding to the basic training weight prediction head is m.

[0021] Modify the number of categories corresponding to the basic training weight prediction head to m+n, where n is the few sample categories corresponding to the generated data. The m-class weight parameters corresponding to the basic data remain unchanged, and the n-class weight parameters corresponding to the generated data are randomly initialized. Use the generated dataset containing m+n classes of data to train the convolutional neural network model to obtain the generated training weights.

[0022] The convolutional neural network prediction model is trained using the fine-tuned dataset. The backbone network parameters of the convolutional neural network model with training weights are frozen. The head parameters of the convolutional neural network prediction model are adjusted to obtain fine-tuned training weights suitable for railway target detection with few samples. The head parameters and backbone network parameters of the trained convolutional neural network prediction model are combined to obtain the trained convolutional neural network prediction model.

[0023] As can be seen from the technical solutions provided by the embodiments of the present invention above, the railway scene target detection method based on few-shot learning proposed by the embodiments of the present invention proposes a training framework that combines sample generation and image style transfer for secondary fine-tuning, thereby improving the accuracy of railway few-shot target detection from the perspectives of data augmentation and transfer learning. This method does not improve the accuracy of few-shot target detection by constructing complex networks, but can improve the performance on the basis of lightweight networks, avoiding the situation where complex models consume a lot of computer resources, and has the advantages of small memory usage and fast detection speed.

[0024] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and will become apparent from the description or may be learned by practice of the invention. Attached Figure Description

[0025] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0026] Figure 1 This is a schematic diagram illustrating the implementation principle of a railway scene target detection method based on few-shot learning provided in an embodiment of the present invention.

[0027] Figure 2 This is a flowchart of a railway scene target detection method based on few-shot learning provided by an embodiment of the present invention. Detailed Implementation

[0028] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention.

[0029] Those skilled in the art will understand that, unless specifically stated otherwise, the singular forms “a,” “an,” “the,” and “the” used herein may also include the plural forms. It should be further understood that the term “comprising” as used in this specification means the presence of the stated features, integers, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. It should be understood that when we say an element is “connected” or “coupled” to another element, it can be directly connected or coupled to the other element, or there may be intermediate elements. Furthermore, “connected” or “coupled” as used herein can include wireless connections or couplings. The term “and / or” as used herein includes any and all combinations of one or more of the associated listed items.

[0030] It will be understood by those skilled in the art that, unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. It should also be understood that terms such as those defined in general dictionaries should be understood to have the same meaning as in the context of the prior art, and should not be interpreted in an idealized or overly formal sense unless defined as herein.

[0031] To facilitate understanding of the embodiments of the present invention, the following will provide further explanation and description with reference to the accompanying drawings and several specific embodiments. These embodiments do not constitute a limitation on the embodiments of the present invention.

[0032] This invention provides a railway scene target detection method based on few-shot learning. The implementation principle of this method is illustrated in the following diagram. Figure 1 As shown, the specific processing flow is as follows: Figure 2 As shown, it includes the following steps:

[0033] Step S21: Construct three datasets: a basic dataset, a generated dataset, and a fine-tuning dataset. Use sample generation and image style transfer methods to process railway scene images to obtain the generated dataset.

[0034] The aforementioned base dataset is a publicly available dataset containing a large number of railway scene image samples. This publicly available dataset can be the VOC2007 dataset, which contains samples of 20 categories.

[0035] The aforementioned generated dataset consists of a portion of the data from the base dataset and generated data obtained by processing railway scene images using sample generation and image style transfer methods.

[0036] The aforementioned sample generation refers to generating a target with few samples from a railway scene image without a target.

[0037] The image style transfer mentioned refers to performing weather and seasonal transfer processing on images. Rain, fog, and snow are added to the images to generate railway scene images under severe weather conditions, and seasonal changes are applied to generate railway scene images under different seasons. The generated data includes four categories: kites, bird nests, balloons, and plastic film. This generated data, along with a portion of the VOC2007 data, constitutes the generated dataset.

[0038] The aforementioned fine-tuning dataset consists of a small amount of data from the base dataset and a small number of target data from real railways. Small sample target data from real railway scenes, including kites, bird nests, balloons, and plastic film, were collected and, together with a small amount of data from part of the VOC2007 dataset, constitute the fine-tuning dataset.

[0039] Step S22: Construct a convolutional neural network model for target detection in railway scenes. This convolutional neural network model can be a Faster R-CNN model. The Faster R-CNN network model consists of four parts: a feature extraction network, a region candidate network (RPN), a region of interest pooling layer (RoIPooling layer), and a fully connected layer. The training process is as follows: First, the feature extraction network extracts features from the input image to obtain a feature map. Then, the RPN generates multiple regions of interest. Next, the RoIPooling layer combines the regions of interest output by the RPN network and the feature map output by the feature extraction network to obtain a fixed-size region feature map. Finally, the fully connected layer performs classification and regression.

[0040] The convolutional neural network model is trained using the aforementioned basic dataset to obtain basic training weights, enabling the convolutional neural network model to learn generalized feature representations of railway scene image samples. The prediction category corresponding to the basic training weights is m. For example, m can be 20, including 20 categories from the VOC2007 dataset: airplane, bicycle, bird, boat, bottle, bus, car, cat, chair, cow, dining table, dog, horse, motorcycle, person, flower pot, sheep, sofa, train, and remote control.

[0041] Step S23: Modify the number of categories corresponding to the basic training weight prediction head to m+n, where n is the few-sample category. Use the generated dataset to train the convolutional neural network model to obtain the generated training weights, further improving the generalization of the convolutional neural network model.

[0042] The number of prediction head categories for the basic training weights is adjusted. The 20 weight parameters corresponding to the basic data remain unchanged, while the 4 weight parameters corresponding to the generated data are randomly initialized, increasing the number of prediction head categories for the basic training weights to 24. Based on this, a convolutional neural network model is trained using the generated dataset containing 24 categories of data to obtain the generated training weights.

[0043] Step S24: Train the convolutional neural network prediction model using the aforementioned fine-tuned dataset. Freeze the backbone network parameters of the convolutional neural network model that generated the training weights. Adjust the head parameters of the convolutional neural network prediction model to obtain fine-tuned training weights suitable for railway target detection with few samples. Combine the head parameters and backbone network parameters of the trained convolutional neural network prediction model to obtain the trained convolutional neural network prediction model.

[0044] Step S25: Input the railway scene image to be identified into the above-trained convolutional neural network prediction model, and the above-trained convolutional neural network prediction model outputs the target recognition result of the railway scene image to be identified.

[0045] To verify the effectiveness of the present invention, railway scene images containing few-sample targets such as kites, bird nests, balloons, and mulch were used to verify the target detection performance of the model. The average detection accuracy for the above four types of targets with few samples was 79%, which achieved a good detection effect.

[0046] In summary, the railway scene target detection method based on few-shot learning proposed in this invention presents a training framework that combines sample generation and image style transfer for secondary fine-tuning. From the perspectives of data augmentation and transfer learning, this helps the model better learn the features of few-shot targets, thereby improving the accuracy of few-shot railway target detection. This method does not improve the accuracy of few-shot target detection by constructing complex networks; instead, it enhances performance on a lightweight network basis, offering advantages such as small memory footprint and fast detection speed.

[0047] Those skilled in the art will understand that the accompanying drawings are merely schematic diagrams of one embodiment, and the modules or processes shown in the drawings are not necessarily essential for implementing the present invention.

[0048] As can be seen from the above description of the embodiments, those skilled in the art can clearly understand that the present invention can be implemented by means of software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in various embodiments or some parts of the embodiments of the present invention.

[0049] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, for apparatus or system embodiments, since they are basically similar to method embodiments, the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments. The apparatus and system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0050] The above description is merely a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A railway scene target detection method based on few-shot learning, characterized in that, include: Acquire railway scene image samples, construct a basic dataset, generate a dataset, and fine-tune the dataset; A convolutional neural network model for target detection in railway scenes is constructed. The convolutional neural network model is trained sequentially using the base dataset, the generated dataset, and the fine-tuning dataset to obtain a trained convolutional neural network model. The railway scene image to be identified is input into the trained convolutional neural network prediction model, and the trained convolutional neural network prediction model outputs the target recognition result of the railway scene image to be identified. The process of acquiring railway scene image samples, constructing a basic dataset, generating a dataset, and fine-tuning a dataset includes: Obtain a public dataset containing a certain number of railway scene image samples, and use the public dataset as the base dataset; The generated dataset is obtained by processing railway scene images using a sample generation method and an image style transfer method. The generated dataset consists of a portion of the data in the base dataset and the generated data. The sample generation method refers to adding a small number of target objects to railway scene images without targets. The image style transfer refers to performing weather transfer and seasonal transfer processing on the images. By adding rain, fog, and snow to the images, railway scene images under severe weather conditions are generated. By changing the season of the images, railway scene images under different seasons are generated. The fine-tuning dataset consists of a small amount of data from the base dataset and a small sample of real railway target data; The process of training the convolutional neural network model sequentially using the base dataset, the generated dataset, and the fine-tuning dataset to obtain a trained convolutional neural network model includes: The convolutional neural network model is trained using the aforementioned basic dataset to obtain basic training weights, enabling the convolutional neural network model to learn generalized feature representations of railway scene image samples. The number of categories corresponding to the basic training weight prediction head is m. Modify the number of categories corresponding to the basic training weight prediction head to m+n, where n is the few sample categories corresponding to the generated data. The m-class weight parameters corresponding to the basic data remain unchanged, and the n-class weight parameters corresponding to the generated data are randomly initialized. Use the generated dataset containing m+n classes of data to train the convolutional neural network model to obtain the generated training weights. The convolutional neural network prediction model is trained using the fine-tuned dataset. The backbone network parameters of the convolutional neural network model with training weights are frozen. The head parameters of the convolutional neural network prediction model are adjusted to obtain fine-tuned training weights suitable for railway target detection with few samples. The head parameters and backbone network parameters of the trained convolutional neural network prediction model are combined to obtain the trained convolutional neural network prediction model.

2. The method according to claim 1, characterized in that, The publicly available dataset is the VOC2007 dataset containing samples of 20 categories. The generated data includes four categories: kites, bird nests, balloons, and plastic film. The generated data and a portion of the VOC2007 data together form the generated dataset. A small number of target data samples of kites, bird nests, balloons, and plastic film in real railway scenes are collected and together with a small amount of data from a portion of the VOC2007 dataset, they form the fine-tuning dataset.

3. The method according to claim 1 or 2, characterized in that, The construction of the convolutional neural network model for target detection in railway scenes includes: A convolutional neural network model for target detection in railway scenes is constructed. This convolutional neural network model is the Faster R-CNN model, which consists of four parts: a feature extraction network, a region candidate network (RPN), a region of interest pooling layer, and a fully connected layer. The training process is as follows: the feature extraction network extracts features from the input image to obtain a feature map; the region candidate network generates multiple regions of interest; the region of interest output by the region candidate network and the feature map output by the feature extraction network are combined by the region of interest pooling layer to obtain a region feature map of a fixed size; and finally, classification and regression are performed by the fully connected layer.

Images