A train track transportation forward foreign matter detection method and system based on unsupervised learning

Abstract

The application provides a train track transportation forward foreign matter detection method and system based on unsupervised learning. The method comprises the following steps: acquiring a video stream of an operation environment in front of a train track transportation through an industrial camera to obtain to-be-recognized image data; processing the to-be-recognized image data through a cross-scale feature extraction module, extracting local detail features and global semantic features of the to-be-recognized image data in parallel, and generating a group of multi-scale features; performing parallel probability density estimation on the multi-scale features through a double-path fusion flow module, and mapping the multi-scale features into two groups of mutually complementary hidden variables under local paths and global paths respectively; integrating the two groups of hidden variable information through a target segmentation positioning module, performing adaptive fusion, and generating a pixel-level positioning heat map indicating the position and contour of foreign matters. The method can break away from the dependence on massive artificial annotation data, and can accurately detect and position unknown categories and various forms of foreign matters in a track environment.

Description

Technical Field

[0001] This invention relates to the field of rail transit safety technology, and in particular to a method and system for detecting forward foreign objects in train rail transit based on unsupervised learning. Background Technology

[0002] With the rapid development of rail transit networks, ensuring train operation safety has become a top priority. Intrusion of foreign objects onto the tracks, such as falling rocks, fallen trees, pedestrians, animals, or obstacles caused by severe weather, poses a serious threat to high-speed trains. Therefore, developing a system capable of real-time, accurate, and all-weather automatic detection of forward-moving foreign objects on the tracks is of paramount importance for accident prevention and the protection of life and property.

[0003] Currently, existing orbital foreign object detection technologies are mainly divided into traditional methods and deep learning-based methods. Traditional methods rely on hardware such as radar, lidar, or infrared sensors. Although effective under specific conditions, these methods are costly, complex to deploy, and susceptible to electromagnetic interference or specific environmental factors.

[0004] In recent years, deep learning-based computer vision methods have shown great potential in the field of foreign object detection due to their advantages of low cost and high flexibility. Among them, supervised learning methods, represented by YOLO and Mask R-CNN, are currently the mainstream. These methods, trained on large datasets with labeled foreign object locations and categories, can achieve good detection results for foreign objects of known categories. However, in the open and complex real-world scenario of rail transit, the aforementioned deep learning-based computer vision methods have the following drawbacks:

[0005] (1) Strong dependence on data annotation: The types, shapes, sizes and locations of orbital foreign objects are extremely random and diverse, making it impossible to exhaustively enumerate them using a limited dataset. For unknown categories of foreign objects that have never appeared in the training set, the detection performance of supervised models will drop sharply, posing a serious risk of missed detections.

[0006] (2) Limited generalization ability: The performance of the model is highly dependent on the quality and coverage of the training data. When the actual operating environment (such as sudden changes in lighting, severe weather such as rain, snow and fog) differs from the training scenario, the model's generalization ability is insufficient, which can easily lead to false positives and false negatives.

[0007] (3) High labeling cost: Acquiring and accurately labeling massive amounts of orbital foreign object image data requires huge human and time costs, which greatly limits the rapid iteration and widespread application of the algorithm.

[0008] Therefore, how to design a high-precision, high-efficiency unsupervised foreign object detection method that can effectively detect unknown types of foreign objects without manual labeling and can adapt to complex and ever-changing track environments is a technical problem that urgently needs to be solved in the field of rail transit safety. Summary of the Invention

[0009] The embodiments of the present invention provide a method and system for detecting forward foreign objects in train track traffic based on unsupervised learning, so as to effectively detect forward foreign objects in track traffic.

[0010] To achieve the above objectives, the present invention adopts the following technical solution.

[0011] According to one aspect of the present invention, a forward foreign object detection method for train track traffic based on unsupervised learning is provided, comprising:

[0012] The video stream of the operating environment in front of the train track is captured by an industrial camera, and the captured video stream is decoded to obtain the image data to be identified.

[0013] The image data to be identified is processed by the cross-scale feature extraction module, which extracts local detail features and global semantic features of the image data to be identified in parallel to generate a set of multi-scale features;

[0014] The multi-scale features are estimated in parallel using a dual-path fusion flow module, which maps the multi-scale features into two sets of complementary latent variables under local and global paths, respectively.

[0015] The target segmentation and localization module integrates the two sets of latent variable information and performs adaptive fusion to generate a pixel-level localization heatmap that indicates the location and outline of the foreign object.

[0016] Preferably, the process of acquiring video streams of the operating environment ahead of the train track using an industrial camera, and decoding the acquired video streams to obtain image data to be identified includes:

[0017] A high-resolution video stream of the operating environment in front of the train is acquired by an industrial camera installed at the front of the train. The acquired video stream is decoded to obtain continuous image frames. The size of each frame is uniformly adjusted to 256×256 pixels. All the adjusted frame images are combined to obtain the image data to be identified.

[0018] Preferably, the process of processing the image data to be identified through the cross-scale feature extraction module extracts local detail features and global semantic features of the image data to be identified in parallel, generating a set of multi-scale features, including:

[0019] The cross-scale feature extraction module includes a local feature extraction branch, a semantic feature extraction branch, and a context-aware fusion module. The local feature extraction branch utilizes a cascaded residual network structure and downsampling operations to capture local detail information in the image data to be identified and outputs multi-scale local feature maps. ;

[0020] The semantic feature extraction branch utilizes a stacked Swing Transformer structure to capture global semantic information in the image data to be recognized and outputs multi-scale global semantic feature maps. ;

[0021] The multi-scale local feature map and the global semantic feature map The input is fed into the context-aware fusion module, which processes the multi-scale local feature map using a 1×1 2D convolutional module. and the global semantic feature map Channel alignment is performed, and dynamic weights are learned through gated convolution to optimize local feature maps. and global semantic feature map Adaptive weighted fusion is performed at each spatial location to generate a set of information-enhanced multi-scale features using formula (1). ( );

[0022] (1)

[0023] In the formula: Indicates the gating weight value. This represents a 1×1 2D convolutional layer. This indicates element-wise multiplication. This indicates element-wise addition.

[0024] Preferably, the method of performing parallel probability density estimation of the multi-scale features through a dual-path fusion flow module, mapping the multi-scale features into two sets of complementary latent variables under local and global paths respectively, includes:

[0025] In the local path, for the multi-scale features ( For each scale feature in the process, a multi-scale decoupling fusion module is applied, which performs a multi-scale decoupling fusion on each scale feature. ( Each channel is transformed using a separate 1×1 2D convolutional module, and then connected with the original features through an identity mapping. Add them together and perform preprocessing to obtain the features. ( );

[0026] In the global path, the global scale interaction module first performs channel alignment using a 1×1 2D convolutional module. After downsampling to the minimum scale using bilinear interpolation, the channels are concatenated. Then, a channel weighting module consisting of a 2D average pooling layer, a 3×3 2D convolutional module, a ReLU activation function, a 1×1 2D convolutional module, and a Softmax activation function generates weights for the feature maps at each scale. ( ), and the multi-scale features ( The initial fusion features are integrated using formula (2). ;

[0027] (2)

[0028] In the formula: Indicates the initial fusion features. Represents feature weights, The features are those obtained after channel transformation and downsampling.

[0029] For the initial fusion features Enhanced features are obtained by performing an affine nonlinear transformation using formula (3).

[0030] (3)

[0031] In the formula: This is the enhanced feature after nonlinear transformation. and Initial fusion features Features obtained through channel segmentation and The subnetwork predicts that it uses a structure of 3×3 2D convolutional modules, a ReLU activation function, and 3×3 2D convolutional modules. Indicates a splicing operation;

[0032] For the enhanced features ( Each feature in the model is subjected to a reversible transformation using a normalized flow model to generate local latent variables. ( ) and local Jacobian matrix ( ), for the enhanced features By applying a normalized flow model to perform a reversible transformation, two sets of complementary global latent variables are generated. and the global Jacobian matrix .

[0033] Preferably, the step of integrating the two sets of latent variable information through the target segmentation and localization module and performing adaptive fusion to generate a pixel-level localization heatmap indicating the location and contour of the foreign object includes:

[0034] Calculate the local latent variables respectively ( and global hidden variables The negative log-likelihood value under the standard normal distribution is obtained by using an exponential function and formula (4) to express the local latent variable. ( and global hidden variables Convert to a local initial anomaly probability map ( ) and global initial anomaly probability map ;

[0035] (4)

[0036] In the formula: ( )and This is a local and global initial anomaly probability map. Indicates the first A local latent variable, This represents a global implicit variable. Indicates the first The first local latent variable The square of the number of channels; The first global hidden variable represents the... The square of the number of channels; Indicates the number of channels. Indicate the base;

[0037] The local initial anomaly probability map ( ) and global initial anomaly probability map Upsampling to the original image resolution via bilinear interpolation, and then generating local path target localization heatmaps by weighted averaging of probability maps and spatial neighborhood max pooling using a 3×3 2D convolutional module. and global path target localization heatmap ;

[0038] Through learnable adaptive weight parameters The local path target location heatmap is obtained using formula (5). and global path target localization heatmap Weighted fusion was performed to generate a pixel-level localization heatmap that indicated the location and outline of the foreign object. ;

[0039] (5)

[0040] In the formula: Indicates adaptive weights, This represents a global path target location heatmap. ( () represents a local path target location heatmap.

[0041] According to another aspect of the present invention, a forward foreign object detection system for train track traffic based on unsupervised learning is provided, characterized in that it includes: an image data acquisition module, a cross-scale feature extraction module, a dual-path fusion flow module, and a target segmentation and localization module;

[0042] The image data acquisition module is used to acquire video streams of the operating environment in front of the train track traffic through an industrial camera, and to decode the acquired video streams to obtain image data to be identified.

[0043] The cross-scale feature extraction module is used to process the image data to be identified, extract local detail features and global semantic features of the image data to be identified in parallel, and generate a set of multi-scale features.

[0044] The dual-path fusion flow module is used to perform parallel probability density estimation on the multi-scale features, mapping the multi-scale features into two sets of complementary latent variables under the local path and the global path respectively.

[0045] The target segmentation and localization module is used to integrate the two sets of latent variable information and perform adaptive fusion to generate a pixel-level localization heatmap that indicates the location and contour of the foreign object.

[0046] Preferably, the image data acquisition module is used to acquire a high-resolution video stream of the operating environment in front of the train through an industrial camera installed at the front of the train, decode the acquired video stream to obtain continuous image frames, adjust the size of each frame to 256×256 pixels, and combine all the adjusted frame images to obtain the image data to be identified.

[0047] Preferably, the cross-scale feature extraction module includes a local feature extraction branch, a semantic feature extraction branch, and a context-aware fusion module. The local feature extraction branch utilizes a cascaded residual network structure and downsampling operations to capture local detail information in the image data to be identified and outputs multi-scale local feature maps. ;

[0048] The semantic feature extraction branch utilizes a stacked Swing Transformer structure to capture global semantic information in the image data to be recognized and outputs multi-scale global semantic feature maps. ;

[0049] The multi-scale local feature map and the global semantic feature map The input is fed into the context-aware fusion module, which processes the multi-scale local feature map using a 1×1 2D convolutional module. and the global semantic feature map Channel alignment is performed, and dynamic weights are learned through gated convolution to optimize local feature maps. and global semantic feature map Adaptive weighted fusion is performed at each spatial location to generate a set of information-enhanced multi-scale features using formula (1). ( );

[0050] (1)

[0051] In the formula: Indicates the gating weight value. This represents a 1×1 2D convolutional layer. This indicates element-wise multiplication. This indicates element-wise addition.

[0052] Preferably, the dual-path fusion flow module is used to process the multi-scale features in a local path. ( For each scale feature in the process, a multi-scale decoupling fusion module is applied, which performs a multi-scale decoupling fusion on each scale feature. ( Each channel is transformed using a separate 1×1 2D convolutional module, and then connected with the original features via an identity mapping. Add them together and perform preprocessing to obtain the features. ( );

[0053] In the global path, the global scale interaction module first performs channel alignment using a 1×1 2D convolutional module. After downsampling to the minimum scale using bilinear interpolation, the channels are concatenated. Then, a channel weighting module consisting of a 2D average pooling layer, a 3×3 2D convolutional module, a ReLU activation function, a 1×1 2D convolutional module, and a Softmax activation function generates weights for the feature maps at each scale. ( ), and the multi-scale features ( The initial fusion features are integrated using formula (2). ;

[0054] (2)

[0055] In the formula: Indicates the initial fusion features. Represents feature weights, The features are those obtained after channel transformation and downsampling.

[0056] For the initial fusion features Enhanced features are obtained by performing an affine nonlinear transformation using formula (3).

[0057] (3)

[0058] In the formula: This is the enhanced feature after nonlinear transformation. and Initial fusion features Features obtained through channel segmentation and The subnetwork predicts that it uses a structure of 3×3 2D convolutional modules, a ReLU activation function, and 3×3 2D convolutional modules. Indicates a splicing operation;

[0059] For the enhanced features ( Each feature in the model is subjected to a reversible transformation using a normalized flow model to generate local latent variables. ( ) and local Jacobian matrix ( ), for the enhanced features By applying a normalized flow model to perform a reversible transformation, two sets of complementary global latent variables are generated. and the global Jacobian matrix .

[0060] Preferably, the target segmentation and localization module is used to calculate the local latent variables respectively. ( and global hidden variables The negative log-likelihood value under the standard normal distribution is obtained by using an exponential function and formula (4) to express the local latent variable. ( and global hidden variables Convert to a local initial anomaly probability map ( ) and global initial anomaly probability map ;

[0061] (4)

[0062] In the formula: ( )and This is a local and global initial anomaly probability map. Indicates the first A local latent variable, This represents a global implicit variable. Indicates the first The first local latent variable The square of the number of channels; The first global hidden variable represents the... The square of the number of channels; Indicates the number of channels. Indicate the base;

[0063] The local initial anomaly probability map ( ) and global initial anomaly probability map Upsampling to the original image resolution via bilinear interpolation, and then generating local path target localization heatmaps by weighted averaging of probability maps and spatial neighborhood max pooling using a 3×3 2D convolutional module. and global path target localization heatmap ;

[0064] Through learnable adaptive weight parameters The local path target location heatmap is obtained using formula (5). and global path target localization heatmap Weighted fusion was performed to generate a pixel-level localization heatmap that indicated the location and outline of the foreign object. ;

[0065] (5)

[0066] In the formula: Indicates adaptive weights, This represents a global path target location heatmap;

[0067] ( () represents a local path target location heatmap.

[0068] As can be seen from the technical solutions provided by the embodiments of the present invention above, the method of the present invention can get rid of the dependence on massive manual annotation data, and can accurately and robustly detect and locate foreign objects of unknown type and different shapes in the orbital environment, while having real-time processing capabilities to meet actual operational needs.

[0069] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and will become apparent from the description or may be learned by practice of the invention. Attached Figure Description

[0070] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0071] Figure 1 A flowchart illustrating a forward foreign object detection method for train track traffic based on unsupervised learning, provided in an embodiment of the present invention.

[0072] Figure 2 This is a schematic diagram illustrating the implementation principle of a global scale switching module provided in an embodiment of the present invention.

[0073] Figure 3 This is a schematic diagram illustrating the implementation principle of a normalized flow model provided in an embodiment of the present invention.

[0074] Figure 4 This is a schematic diagram of a detection result provided in an embodiment of the present invention;

[0075] Figure 5 This is a structural diagram of a train track traffic forward foreign object detection system based on unsupervised learning, provided in an embodiment of the present invention. Detailed Implementation

[0076] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention.

[0077] Those skilled in the art will understand that, unless specifically stated otherwise, the singular forms “a,” “an,” “the,” and “the” used herein may also include the plural forms. It should be further understood that the term “comprising” as used in this specification means the presence of the stated features, integers, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. It should be understood that when we say an element is “connected” or “coupled” to another element, it can be directly connected or coupled to the other element, or there may be intermediate elements. Furthermore, “connected” or “coupled” as used herein can include wireless connections or couplings. The term “and / or” as used herein includes any and all combinations of one or more of the associated listed items.

[0078] It will be understood by those skilled in the art that, unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. It should also be understood that terms such as those defined in general dictionaries should be understood to have the same meaning as in the context of the prior art, and should not be interpreted in an idealized or overly formal sense unless defined as herein.

[0079] To facilitate understanding of the embodiments of the present invention, several specific embodiments will be further explained below with reference to the accompanying drawings, and these embodiments do not constitute a limitation on the embodiments of the present invention.

[0080] Example 1

[0081] The processing flow of a forward foreign object detection method for train track traffic based on unsupervised learning provided in this embodiment of the invention is as follows: Figure 1 As shown, the processing steps include the following:

[0082] Step S1: A high-resolution video stream of the operating environment in front of the train is acquired using an industrial camera installed at the front of the train. The acquired video stream is decoded to obtain continuous image frames. The size of each frame is uniformly adjusted to 256×256 pixels. Before inputting the data into the network, all the resized frame images are combined to obtain the image data to be recognized.

[0083] Step S2: The image data to be identified is processed by the cross-scale feature extraction module, which extracts local detail features and global semantic features of the image data in parallel. The local detail features and global semantic features are then fused to generate a set of multi-scale features. The cross-scale feature extraction module includes a Local Feature Extraction (LFE) branch, a Semantic Feature Extraction (SFE) branch, and a context-aware fusion module. The local feature extraction branch is used to capture fine-grained local details such as the edges of foreign objects; the semantic feature extraction branch is used to capture high-level global semantic information such as the orbital environment; the context-aware fusion module is used to adaptively weight and fuse the features output by the local feature extraction branch and the semantic feature extraction branch to generate the information-enhanced multi-scale features. ( ).

[0084] Step S3: Parallel probability density estimation of the multi-scale features is performed using a dual-path fusion flow module, mapping the multi-scale features into two sets of complementary latent variables under both local and global paths. The dual-path fusion flow module includes local path processing, global path processing, and normalized flow transformation. Specifically, the local path performs multi-scale feature... ( The feature is obtained by preprocessing using a multi-scale decoupling and fusion module. ( This is used to maintain the independence of features at each scale to accurately locate small targets and foreign objects; global path, through multi-scale features... ( Enhanced features are obtained by preprocessing using a global-scale interaction module. This is used to establish dependencies between cross-scale features to understand global contextual information; wherein, the local path features ( ) and global path enhancement features All methods employ a series of reversible normalized flow transformations to accurately fit the probability distribution of normal samples, mapping the input features to corresponding local latent variables. ( Local Jacobian matrix ( and global hidden variables Global Jacobian matrix .

[0085] Step S4: The target segmentation and localization module integrates the two sets of latent variable information and performs adaptive fusion to generate a pixel-level localization heatmap that can mark the location and contour of the foreign object. The target segmentation and localization module first converts the local and global latent variables into corresponding initial anomaly probability maps based on the standard normal distribution assumption. Then, it enhances the saliency of the target region through spatial upsampling and neighborhood enhancement operations. Finally, it uses learnable adaptive weights to perform weighted fusion of the local and global localization results, outputting the final foreign object detection result.

[0086] Furthermore, step S2 specifically includes the following steps:

[0087] S21: By utilizing a cascaded residual network structure and downsampling operations through the local feature extraction branch, fine-grained local details such as foreign object edges and textures in the image data to be identified are captured, and multi-scale local feature maps are output. ;

[0088] S22: By utilizing a stacked Swing Transformer structure through the semantic feature extraction branch, high-level global semantic information such as the track environment and background layout in the image data to be recognized is captured, and multi-scale global semantic feature maps are output. ;

[0089] S23: Extract the multi-scale local feature maps from steps S21 and S22. and global semantic feature map The input is fed into the context-aware fusion module. This module first aligns the two sets of feature maps using a 1×1 2D convolutional module, and then learns dynamic weights through a gated convolution mechanism to optimize local feature maps. and global semantic feature map Adaptive weighted fusion is performed at each spatial location to generate information-enhanced multi-scale features using formula (1). ( );

[0090] (1)

[0091] In the formula: Indicates the gating weight value. This represents a 1×1 2D convolutional layer. This indicates element-wise multiplication. This indicates element-wise addition.

[0092] Furthermore, step S3 specifically includes the following steps:

[0093] S31: In the local path, for the multi-scale features ( For each scale feature in the process, a multi-scale decoupling fusion module is applied, which performs a multi-scale decoupling fusion on each scale feature. ( Each channel is transformed using a separate 1×1 2D convolutional module, and then connected with the original features through an identity mapping. Add them together and perform preprocessing to obtain the features. ( );

[0094] S32: Figure 2 This is a schematic diagram illustrating the implementation principle of a global scale exchange module provided in an embodiment of the present invention. In the global path, the global scale interaction module first performs channel alignment using a 1×1 2D convolutional module. After downsampling to the minimum scale using bilinear interpolation, channel concatenation is performed. A channel weighting module, consisting of a 2D average pooling layer, a 3×3 2D convolutional module, a ReLU activation function, a 1×1 2D convolutional module, and a Softmax activation function, generates weights for the feature maps at each scale. ( ), and the multi-scale features ( The initial fusion features are integrated using formula (2). ;

[0095] (2)

[0096] In the formula: Indicates the initial fusion features. Represents feature weights, The features are those obtained after channel transformation and downsampling.

[0097] S33: Regarding the initial fusion features Enhanced features are obtained by performing an affine nonlinear transformation using formula (3).

[0098] (3)

[0099] In the formula: This is the enhanced feature after nonlinear transformation. and Initial fusion features Features obtained through channel segmentation and Based on subnetwork predictions, the network employs a structure consisting of 3×3 2D convolutional modules, a ReLU activation function, and 3×3 2D convolutional modules. Indicates a splicing operation;

[0100] S34: Figure 3This is a schematic diagram illustrating the implementation principle of a normalized flow model provided in an embodiment of the present invention. Regarding the local features... ( Each feature is processed by an independent normalized flow model for reversible transformation to maintain the independence of features at each scale, thereby accurately modeling local anomalies and generating local latent variables. ( Local Jacobian matrix ( ), for the enhanced features A unified normalized flow model is applied for reversible transformation to establish cross-scale dependencies between features and generate global latent variables. Global Jacobian matrix .

[0101] Furthermore, step S34 specifically includes the following steps:

[0102] S341: Features of the input module ( After random rearrangement along the channel dimension, the features are evenly divided into two independent groups. ;

[0103] S342: Features ( The input is fed into a subnetwork consisting of 3×3 2D convolutional modules, a LeakyReLU activation function, and a 3×3 2D convolutional module, which is used to learn the scaling factor. Translation factor Perform nonlinear transformation to transform features ( The features are then merged with the transformed features to obtain the new features. ( At the same time, the scaling factor The logarithmic summation is accumulated to the corresponding Jacobian determinant. middle;

[0104] S343: Features ( The input is fed into a subnetwork consisting of 3×3 2D convolutional modules, a LeakyReLU activation function, and a 3×3 2D convolutional module, which is used to learn the scaling factor. Translation factor Perform nonlinear transformation to transform features ( The features are then merged with the transformed features to obtain the new features. ( At the same time, the scaling factor The logarithmic summation is accumulated to the corresponding Jacobian determinant. middle;

[0105] S344: Merged features and characteristics Features are obtained by splicing channels. The features generated after concatenation no longer contain spatially correlated image semantic information, but are instead mapped to random variables following a standard normal distribution. That is, the local hidden variable. The logarithmic determinant values ​​in steps S342 and S343 and The accumulated output is the corresponding local Jacobian matrix. ;

[0106] S345: Enhanced features Similarly, steps S341-S344 are performed to obtain the global hidden variables. and the global Jacobian matrix .

[0107] Furthermore, step S4 specifically includes the following steps:

[0108] S41: Calculate the local latent variables respectively. ( and global hidden variables The negative log-likelihood value under the standard normal distribution is used to convert it into a local initial anomaly probability map using the exponential function and formula (4). ( ) and global initial anomaly probability map ;

[0109] (4)

[0110] In the formula: ( )and This is a local and global initial anomaly probability map. Indicates the first A local latent variable, This represents a global implicit variable. Indicates the first The first local latent variable The square of the number of channels; The first global hidden variable represents the... The square of the number of channels; Indicates the number of channels. Indicate the base;

[0111] S42: The local initial anomaly probability map ( ) and global initial anomaly probability map Upsampling to the original image resolution via bilinear interpolation, followed by enhancement processing using a weighted average of probability maps and spatial neighborhood max pooling with a 3×3 2D convolutional module, generates local path target localization heatmaps. and global path target localization heatmap ;

[0112] S43: Through a learnable adaptive weight parameter Formula (5) is used to perform weighted fusion of the target location heatmaps of the local path and the global path to generate a pixel-level location heatmap that indicates the location and contour of the foreign object. .

[0113] (5)

[0114] In the formula: Indicates adaptive weights, This represents a global path target location heatmap. ( () represents a local path target location heatmap. Figure 4 This is a schematic diagram of a detection result provided in an embodiment of the present invention.

[0115] Example 2

[0116] The structure of a forward foreign object detection system for train track traffic based on unsupervised learning provided in this embodiment of the invention is as follows: Figure 5 As shown, it includes: an image data acquisition module, a cross-scale feature extraction module, a dual-path fusion flow module, and a target segmentation and localization module;

[0117] The image data acquisition module is used to acquire video streams of the operating environment in front of the train track traffic through an industrial camera, and to decode the acquired video streams to obtain image data to be identified.

[0118] The cross-scale feature extraction module is used to process the image data to be identified, extract local detail features and global semantic features of the image data to be identified in parallel, and generate a set of multi-scale features.

[0119] The dual-path fusion flow module is used to perform parallel probability density estimation on the multi-scale features, mapping the multi-scale features into two sets of complementary latent variables under the local path and the global path respectively.

[0120] The target segmentation and localization module is used to integrate the two sets of latent variable information and perform adaptive fusion to generate a pixel-level localization heatmap that indicates the location and contour of the foreign object.

[0121] Preferably, the image data acquisition module is used to acquire a high-resolution video stream of the operating environment in front of the train through an industrial camera installed at the front of the train, decode the acquired video stream to obtain continuous image frames, adjust the size of each frame to 256×256 pixels, and combine all the adjusted frame images to obtain the image data to be identified.

[0122] Preferably, the cross-scale feature extraction module includes a local feature extraction branch, a semantic feature extraction branch, and a context-aware fusion module. The local feature extraction branch utilizes a cascaded residual network structure and downsampling operations to capture local detail information in the image data to be identified and outputs multi-scale local feature maps. ;

[0123] The semantic feature extraction branch utilizes a stacked Swing Transformer structure to capture global semantic information in the image data to be recognized and outputs multi-scale global semantic feature maps. ;

[0124] The multi-scale local feature map and the global semantic feature map The input is fed into the context-aware fusion module, which processes the multi-scale local feature map using a 1×1 2D convolutional module. and the global semantic feature map Channel alignment is performed, and dynamic weights are learned through gated convolution to optimize local feature maps. and global semantic feature map Adaptive weighted fusion is performed at each spatial location to generate a set of information-enhanced multi-scale features using formula (1). ( );

[0125] (1)

[0126] In the formula: Indicates the gating weight value. This represents a 1×1 2D convolutional layer. This indicates element-wise multiplication. This indicates element-wise addition.

[0127] Preferably, the dual-path fusion flow module is used to process the multi-scale features in a local path. ( For each scale feature in the process, a multi-scale decoupling fusion module is applied, which performs a multi-scale decoupling fusion on each scale feature. ( Each channel is transformed using a separate 1×1 2D convolutional module, and then connected with the original features through an identity mapping. Add them together and perform preprocessing to obtain the features. ( );

[0128] In the global path, the global scale interaction module first performs channel alignment using a 1×1 2D convolutional module. After downsampling to the minimum scale using bilinear interpolation, the channels are concatenated. Then, a channel weighting module consisting of a 2D average pooling layer, a 3×3 2D convolutional module, a ReLU activation function, a 1×1 2D convolutional module, and a Softmax activation function generates weights for the feature maps at each scale. ( ), and the multi-scale features ( The initial fusion features are integrated using formula (2). ;

[0129] (2)

[0130] In the formula: Indicates the initial fusion features. Represents feature weights, The features are those obtained after channel transformation and downsampling.

[0131] For the initial fusion features Enhanced features are obtained by performing an affine nonlinear transformation using formula (3).

[0132] (3)

[0133] In the formula: This is the enhanced feature after nonlinear transformation. and Initial fusion features Features obtained through channel segmentation and The subnetwork predicts that it uses a structure of 3×3 2D convolutional modules, a ReLU activation function, and 3×3 2D convolutional modules. Indicates a splicing operation;

[0134] For the enhanced features ( Each feature in the model is subjected to a reversible transformation using a normalized flow model to generate local latent variables. ( ) and local Jacobian matrix ( ), for the enhanced features By applying a normalized flow model to perform a reversible transformation, two sets of complementary global latent variables are generated. and the global Jacobian matrix

[0135] Preferably, the target segmentation and localization module is used to calculate the local latent variables respectively. ( and global hidden variables The negative log-likelihood value under the standard normal distribution is obtained by using an exponential function and formula (4) to express the local latent variable. ( and global hidden variables Convert to a local initial anomaly probability map ( ) and global initial anomaly probability map ;

[0136] (4)

[0137] In the formula: ( )and This is a local and global initial anomaly probability map. Indicates the first A local latent variable, This represents a global implicit variable. Indicates the first The first local latent variable The square of the number of channels; The first global hidden variable represents the... The square of the number of channels; Indicates the number of channels. Indicate the base;

[0138] The local initial anomaly probability map ( ) and global initial anomaly probability map Upsampling to the original image resolution via bilinear interpolation, and then generating local path target localization heatmaps by weighted averaging of probability maps and spatial neighborhood max pooling using a 3×3 2D convolutional module. and global path target localization heatmap ;

[0139] Through learnable adaptive weight parameters The local path target location heatmap is obtained using formula (5). and global path target localization heatmap Weighted fusion was performed to generate a pixel-level localization heatmap that indicated the location and outline of the foreign object. ;

[0140] (5)

[0141] In the formula: Indicates adaptive weights, This represents a global path target location heatmap. ( () represents a local path target location heatmap.

[0142] The specific process of performing forward foreign object detection in train track traffic based on unsupervised learning using the system of this invention is similar to the aforementioned method embodiments, and will not be repeated here.

[0143] The proposed unsupervised learning-based forward foreign object detection method for train track traffic achieves a detection accuracy of 99.34%. The innovation of this method lies primarily in addressing the diversity of unknown foreign objects and the interference from complex backgrounds in the track environment. By constructing and utilizing four core mechanisms—cross-scale parallel feature extraction, dual-path complementary probability density estimation, affine transformation feature enhancement within the global path, and adaptive fusion of dual-path localization results—and employing end-to-end unsupervised deep learning, it effectively solves the problems of missed and false detections of multi-scale foreign objects (especially small targets) in complex backgrounds. This significantly improves the accuracy of unknown foreign object detection and localization, filling the gap in the performance of existing unsupervised methods in complex track scenarios and providing advanced and practical technical support for the operational safety of rail transit.

[0144] In summary, the method of this invention adopts a completely unsupervised learning paradigm, requiring only track images under normal driving conditions for training, without any manual annotation. This enables the model to detect foreign objects of any unknown category, fundamentally solving the problem of insufficient generalization ability of supervised methods and greatly improving robustness in open environments. Through the collaborative design of a cross-scale feature extraction module and a dual-path fusion flow module, this invention can simultaneously focus on the fine boundaries of foreign objects and their complex environment. This design allows the model to maintain extremely high detection accuracy and localization accuracy even when facing small-sized foreign objects, disguised foreign objects, or complex track backgrounds, significantly outperforming existing single-view unsupervised methods. The parallel and complementary design of local and global paths allows the model to model objects without foreign objects from different dimensions, forming a more comprehensive discrimination criterion. This complementary mechanism effectively improves the ability to capture the boundary contours of foreign objects while reducing false alarms caused by complex backgrounds, achieving more accurate target localization. The entire detection process is integrated into a high-efficiency deep learning network, realizing end-to-end processing from raw image input to foreign object localization heatmap output. Experimental results show that this method, while ensuring high precision, can meet the real-time requirements of rail transit in terms of processing speed, providing a practical and advanced technical solution for train safety systems.

[0145] Those skilled in the art will understand that the accompanying drawings are merely schematic diagrams of one embodiment, and the modules or processes shown in the drawings are not necessarily essential for implementing the present invention.

[0146] As can be seen from the above description of the embodiments, those skilled in the art can clearly understand that the present invention can be implemented by means of software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in various embodiments or some parts of the embodiments of the present invention.

[0147] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, for apparatus or system embodiments, since they are basically similar to method embodiments, the description is relatively simple, and relevant parts can be referred to the description of the method embodiments. The apparatus and system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0148] The above description is merely a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for forward foreign object detection in train track traffic based on unsupervised learning, characterized in that, include: The video stream of the operating environment in front of the train track is captured by an industrial camera, and the captured video stream is decoded to obtain the image data to be identified. The image data to be identified is processed by the cross-scale feature extraction module, which extracts local detail features and global semantic features of the image data to be identified in parallel to generate a set of multi-scale features; The multi-scale features are estimated in parallel using a dual-path fusion flow module, which maps the multi-scale features into two sets of complementary latent variables under local and global paths, respectively. The target segmentation and localization module integrates two sets of latent variable information and performs adaptive fusion to generate a pixel-level localization heatmap that indicates the location and outline of the foreign object. The process involves using a cross-scale feature extraction module to process the image data to be identified, extracting local detail features and global semantic features in parallel to generate a set of multi-scale features, including: The cross-scale feature extraction module includes a local feature extraction branch, a semantic feature extraction branch, and a context-aware fusion module. The local feature extraction branch utilizes a cascaded residual network structure and downsampling operations to capture local detail information in the image data to be identified and outputs multi-scale local feature maps. ; The semantic feature extraction branch utilizes a stacked Swing Transformer structure to capture global semantic information in the image data to be recognized and outputs a multi-scale global semantic feature map. ; The multi-scale local feature map and the global semantic feature map The input is fed into the context-aware fusion module, which processes the multi-scale local feature map using a 1×1 2D convolutional module. and the global semantic feature map Channel alignment is performed, and dynamic weights are learned through gated convolution to optimize local feature maps. and global semantic feature map Adaptive weighted fusion is performed at each spatial location to generate a set of information-enhanced multi-scale features using formula (1). , ; （1） In the formula: Indicates the gating weight value. This represents a 1×1 2D convolutional layer. This indicates element-wise multiplication. This indicates element-wise addition; The method described above involves performing parallel probability density estimation of the multi-scale features using a dual-path fusion flow module, mapping the multi-scale features into two sets of complementary latent variables under both local and global paths, including: In the local path, for the multi-scale features Each scale feature in the process is processed by a multi-scale decoupling and fusion module. Each channel transformation is performed using a separate 1×1 2D convolutional module, and the transformed features are connected to the original features through an identity mapping. Add them together and perform preprocessing to obtain the features. ; In the global path, the global scale interaction module first performs channel alignment using a 1×1 2D convolutional module. After downsampling to the minimum scale using bilinear interpolation, the channels are concatenated. Then, a channel weighting module consisting of a 2D average pooling layer, a 3×3 2D convolutional module, a ReLU activation function, a 1×1 2D convolutional module, and a Softmax activation function generates weights for the feature maps at each scale. The multi-scale features The initial fusion features are integrated using formula (2). ; （2） In the formula: Indicates the initial fusion features. Represents feature weights, The features are those obtained after channel transformation and downsampling. For the initial fusion features Enhanced features are obtained by performing an affine nonlinear transformation using formula (3). （3） In the formula: This is the enhanced feature after nonlinear transformation. and Initial fusion features Features obtained through channel segmentation and The subnetwork is predicted to employ a structure consisting of 3×3 2D convolutional modules, a ReLU activation function, and 3×3 2D convolutional modules. Indicates a splicing operation; For the enhanced features Each feature in the model is subjected to a reversible transformation using a normalized flow model to generate local latent variables. and local Jacobian matrix For the enhanced features By applying a normalized flow model to perform a reversible transformation, two sets of complementary global latent variables are generated. and the global Jacobian matrix .

2. The method according to claim 1, characterized in that, The process of acquiring video streams of the operating environment ahead of train tracks using industrial cameras, decoding the acquired video streams to obtain image data to be identified, includes: A high-resolution video stream of the operating environment in front of the train is acquired by an industrial camera installed at the front of the train. The acquired video stream is decoded to obtain continuous image frames. The size of each frame is uniformly adjusted to 256×256 pixels. All the adjusted frame images are combined to obtain the image data to be identified.

3. The method according to claim 1, characterized in that, The process of integrating the two sets of latent variable information through the target segmentation and localization module, and performing adaptive fusion to generate a pixel-level localization heatmap indicating the location and contour of the foreign object, includes: Calculate the local latent variables respectively and global hidden variables The negative log-likelihood value under the standard normal distribution is obtained by using an exponential function and formula (4) to express the local latent variable. and global hidden variables Convert to a local initial anomaly probability map and global initial anomaly probability map ; （4） In the formula: (and This is a local and global initial anomaly probability map. Indicates the first A local latent variable, This represents a global implicit variable. Indicates the first The first local latent variable The square of the number of channels; The first global hidden variable represents the... The square of the number of channels; Indicates the number of channels. Indicate the base; The local initial anomaly probability map and global initial anomaly probability map Upsampling to the original image resolution via bilinear interpolation, and then generating local path target localization heatmaps by weighted averaging of probability maps and spatial neighborhood max pooling using a 3×3 2D convolutional module. and global path target localization heatmap ; Through learnable adaptive weight parameters The local path target location heatmap is obtained using formula (5). and global path target localization heatmap Weighted fusion was performed to generate a pixel-level localization heatmap that indicated the location and outline of the foreign object. ; （5） In the formula: Indicates adaptive weights, This represents a global path target location heatmap. This represents a heatmap showing the local path target location.

4. A forward foreign object detection system for train track traffic based on unsupervised learning, characterized in that, include: Image data acquisition module, cross-scale feature extraction module, dual-path fusion stream module, and target segmentation and localization module; The image data acquisition module is used to acquire video streams of the operating environment in front of the train track traffic through an industrial camera, and to decode the acquired video streams to obtain image data to be identified. The cross-scale feature extraction module is used to process the image data to be identified, extract local detail features and global semantic features of the image data to be identified in parallel, and generate a set of multi-scale features. The dual-path fusion flow module is used to perform parallel probability density estimation on the multi-scale features, mapping the multi-scale features into two sets of complementary latent variables under the local path and the global path respectively. The target segmentation and localization module is used to integrate two sets of latent variable information and perform adaptive fusion to generate a pixel-level localization heatmap that indicates the location and contour of the foreign object. The aforementioned cross-scale feature extraction module includes a local feature extraction branch, a semantic feature extraction branch, and a context-aware fusion module. Through the local feature extraction branch, it utilizes a cascaded residual network structure and downsampling operations to capture local detail information in the image data to be recognized and outputs multi-scale local feature maps. ; The semantic feature extraction branch utilizes a stacked Swing Transformer structure to capture global semantic information in the image data to be recognized and outputs a multi-scale global semantic feature map. ; The multi-scale local feature map and the global semantic feature map The input is fed into the context-aware fusion module, which processes the multi-scale local feature map using a 1×1 2D convolutional module. and the global semantic feature map Channel alignment is performed, and dynamic weights are learned through gated convolution to optimize local feature maps. and global semantic feature map Adaptive weighted fusion is performed at each spatial location to generate a set of information-enhanced multi-scale features using formula (1). , ; （1） In the formula: Indicates the gating weight value. This represents a 1×1 2D convolutional layer. This indicates element-wise multiplication. This indicates element-wise addition; The dual-path fusion flow module is used to process the multi-scale features in a local path. , Each scale feature in the process is processed by a multi-scale decoupling and fusion module. Each channel transformation is performed using a separate 1×1 2D convolutional module, and the transformed features are connected to the original features through an identity mapping. Add them together and perform preprocessing to obtain the features. ; In the global path, the global scale interaction module first performs channel alignment using a 1×1 2D convolutional module. After downsampling to the minimum scale using bilinear interpolation, the channels are concatenated. Then, a channel weighting module consisting of a 2D average pooling layer, a 3×3 2D convolutional module, a ReLU activation function, a 1×1 2D convolutional module, and a Softmax activation function generates weights for the feature maps at each scale. The multi-scale features The initial fusion features are integrated using formula (2). ; （2） In the formula: Indicates the initial fusion features. Represents feature weights, The features are those obtained after channel transformation and downsampling. For the initial fusion features Enhanced features are obtained by performing an affine nonlinear transformation using formula (3). （3） In the formula: This is the enhanced feature after nonlinear transformation. and Initial fusion features Features obtained through channel segmentation and The subnetwork is predicted to employ a structure consisting of 3×3 2D convolutional modules, a ReLU activation function, and 3×3 2D convolutional modules. Indicates a splicing operation; For the enhanced features Each feature in the model is subjected to a reversible transformation using a normalized flow model to generate local latent variables. and local Jacobian matrix For the enhanced features By applying a normalized flow model to perform a reversible transformation, two sets of complementary global latent variables are generated. and the global Jacobian matrix .

5. The system according to claim 4, characterized in that, The image data acquisition module is used to acquire high-resolution video streams of the operating environment in front of the train through an industrial camera installed at the front of the train. The acquired video streams are decoded to obtain continuous image frames. The size of each frame is uniformly adjusted to 256×256 pixels. All the adjusted frame images are combined to obtain the image data to be identified.

6. The system according to claim 5, characterized in that, The target segmentation and localization module is used to calculate the local latent variables respectively. and global hidden variables The negative log-likelihood value under the standard normal distribution is obtained by using an exponential function and formula (4) to express the local latent variable. and global hidden variables Convert to a local initial anomaly probability map and global initial anomaly probability map ; （4） In the formula: and This is a local and global initial anomaly probability map. Indicates the first A local latent variable, This represents a global implicit variable. Indicates the first The first local latent variable The square of the number of channels; The first global hidden variable represents the... The square of the number of channels; Indicates the number of channels. Indicate the base; The local initial anomaly probability map and global initial anomaly probability map Upsampling to the original image resolution via bilinear interpolation, and then generating local path target localization heatmaps by weighted averaging of probability maps and spatial neighborhood max pooling using a 3×3 2D convolutional module. and global path target localization heatmap ; Through learnable adaptive weight parameters The local path target location heatmap is obtained using formula (5). and global path target localization heatmap Weighted fusion was performed to generate a pixel-level localization heatmap that indicated the location and outline of the foreign object. ; （5） In the formula: Indicates adaptive weights, This represents a global path target location heatmap. This represents a heatmap showing the local path target location.

Images