Foreign matter invasion detection method and system for high-speed railway
By performing frame extraction processing and feature fusion on high-speed railway monitoring videos, and combining the time domain filter module to integrate multi-frame alarm information, the error and missed detection of high-speed railway detection system in complex environments is solved, and efficient foreign object intrusion detection and timely early warning are achieved.
Patent Information
- Application Number
- CN202510434969.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-04-08
- Publication Date
- 2025-07-11
AI Technical Summary
High-speed railway detection systems are prone to false detection and missed detection in complex environments, especially in light changes, sand and dust occlusion, tree shadow changes and high speeds, which may lead to accidents that are not promptly alerted.
The change detection model is used to extract frames for monitoring videos, and the feature extraction module, feature comparison module, feature fusion module and classification head are used for feature processing and prediction. The time domain filter module combines multi-frame alarm information to determine the foreign object invasion limit results, reduce the error detection rate and improve the recognition accuracy.
It effectively reduces the false detection rate, improves the accuracy and speed of foreign object intrusion detection, and ensures the safe operation of high-speed railways.
Smart Images
Figure CN120299158A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the technical field of high-speed railway foreign object intrusion detection, and particularly relates to a high-speed railway foreign object intrusion detection method and system. Background Art
[0002] The total mileage of high-speed railways is long, the network scale is large, the coverage area is large, and the involved environments are numerous (such as plateaus, deserts, forests, debris flow-prone areas, bird protection areas, etc.), making detection complex. Firstly, it is necessary to conduct continuous detection for 7*24 hours. There are large changes in light and shadow during the day and night, which easily lead to false subtraction and missed detection. Secondly, passing through desert windy areas, sand and dust are easy to block the camera, and strong winds are likely to cause the camera to shake violently, resulting in unclear and distorted captured images, leading to false detection. Thirdly, passing through forest areas and bird protection areas, changes in tree shadows and shadows projected by birds in the air are also likely to cause false detection. Fourthly, the running speed of high-speed trains is fast, reaching 200 - 350 kilometers per hour. In the event of an intrusion, it may occur within a few minutes. If the detection speed is slow and the early warning is not timely, accidents will be triggered.
[0003] Some researchers use the inter-frame difference algorithm to perform difference operations on the digital images corresponding to two adjacent frames of the images captured by railway cameras to obtain the algorithm for the foreign object contour. This algorithm is simple and highly sensitive to light changes, but there are still many false detections and the recognition accuracy is not high. Some researchers use the convolutional neural network algorithm for high-speed railway foreign object intrusion detection. The convolutional neural network algorithm has better robustness to the environment, speeds up the calculation speed to a certain extent, and improves the recognition rate of small-size foreign object intrusion. However, due to the problem that the false detection and missed detection rates of new data and new scenarios in the training data are still relatively high. Summary of the Invention
[0004] The embodiments of this application provide a high-speed railway foreign object intrusion detection method and system for quickly detecting and identifying the problem of high-speed railway foreign object intrusion.
[0005] The embodiments of this application provide a high-speed railway foreign object intrusion detection method, including:
[0006] Performing frame extraction on the high-speed railway monitoring video to obtain frame-by-frame detection images;
[0007] Inputting the reference image and the detection image into a change detection model to obtain a change detection result, where the reference image is an image at the same position as the detection image but at a different time;
[0008] Inputting the change detection result into a result input alarm module to generate an alarm message;
[0009] The time-domain filtering module comprehensively judges the foreign object intrusion result based on multi-frame alarm information.
[0010] In a feasible implementation manner, the change detection model includes a feature extraction module, a feature comparison module, a feature fusion module, and a classification head;
[0011] The reference image and the detection image are respectively sent to the feature extraction module with shared weights for feature processing to obtain two sets of multi-scale feature maps;
[0012] Then, the feature comparison module performs comparison processing on the two sets of multi-scale feature maps to obtain a set of multi-scale feature difference maps;
[0013] Next, the feature fusion module fuses a set of multi-scale feature difference maps into a feature difference map of the same scale;
[0014] Finally, classification prediction is performed through the classification head to obtain the change detection result.
[0015] In a feasible implementation manner, the classification head is configured as a 3×3 convolution.
[0016] In a feasible implementation manner, when using the feature fusion module to fuse a set of multi-scale feature difference maps into a feature difference map of the same scale,
[0017] First, use 1×1 convolution to unify the features of each layer to m channels;
[0018] Next, perform 1×1 convolution on the features of the i-th layer and the i-1-th layer respectively, then perform upsampling on the i-th layer to make its size the same as that of the i-1-th layer features, and then add the features of the i-th and i-1-th layers to obtain the new i-1-th layer features. Then, perform batch normalization and ReLU operations on the fused feature difference map. After multiple processes, a feature difference map of the same scale with multiple levels of information is obtained.
[0019] In a feasible implementation manner, when the upsampling uses linear interpolation, swap the adjacent upsampling operation and 1×1 convolution without changing the output of the model.
[0020] In a feasible implementation manner, the result input alarm module determines whether to perform alarm processing according to the number of changed pixels detected in the change detection result;
[0021] If the number of changed pixels is greater than or equal to the alarm threshold, the result input alarm module generates an alarm message.
[0022] In a feasible implementation manner, the time domain filtering module comprehensively judges the foreign object intrusion result based on multiple frames of alarm information, including:
[0023] Assume that the length of the filtering window is l, and the final prediction result is determined by the l-frame time series information,
[0024]
[0025] Among them, Yi represents the final predicted result after filtering for the i-th frame; Yraw,i represents the original predicted result of the i-th frame without filtering.
[0026] In a second aspect, an embodiment of the present application provides a high-speed railway foreign object intrusion detection system, which uses the high-speed railway foreign object intrusion detection method described in any item of the first aspect to detect foreign objects on the high-speed railway for intrusion.
[0027] An embodiment of the present application provides a high-speed railway foreign object intrusion detection method, which includes performing frame extraction on high-speed railway monitoring videos to obtain frame-by-frame detection images; inputting a reference image and a detection image into a change detection model to obtain a change detection result, where the reference image is an image at the same position as the detection image but at a different time; inputting the change detection result into a result input alarm module to generate an alarm message; and a time-domain filtering module comprehensively judges the foreign object intrusion result based on multiple frames of alarm messages, thereby completing the detection of whether there are foreign objects intruding on the high-speed railway.
[0028] An embodiment of the present application also provides a high-speed railway foreign object intrusion detection system, which uses the high-speed railway foreign object intrusion detection method described in any item of the first aspect to detect foreign objects on the high-speed railway for intrusion. Therefore, it has all the beneficial effects of the high-speed railway foreign object intrusion detection method of any of the above technical solutions, which will not be elaborated here. BRIEF DESCRIPTION OF THE DRAWINGS
[0029] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of the present invention. The schematic embodiments of the present invention and their descriptions are used to explain the present application and do not constitute an improper limitation to the present invention.
[0030] In the drawings:
[0031] Figure 1 is a flowchart of the high-speed railway foreign object intrusion detection method provided by an embodiment of the present application;
[0032] Figure 2 is Figure 1 a schematic diagram of the high-speed railway foreign object intrusion detection method in DETAILED DESCRIPTION OF THE EMBODIMENTS
[0033] In order to enable those skilled in the art to better understand the technical solutions in the present application, the following will clearly and completely describe the technical solutions in the embodiments of the present application with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.
[0034] In the description of the embodiments of the present application, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the quantity of the indicated technical features. Thus, features defined with "first" and "second" may explicitly or implicitly include at least one such feature. In the description of the present application, "a plurality of" means at least two, such as two, three, etc., unless otherwise specifically defined.
[0035] In the present application, unless otherwise clearly specified and defined, terms such as "installed", "connected", "joined", "fixed", etc. shall be understood in a broad sense. For example, it may be a fixed connection, a detachable connection, or integrated; it may be a mechanical connection; it may be directly connected, or indirectly connected through an intermediate medium, and it may be the communication inside two elements or the interaction relationship between two elements, unless otherwise clearly defined. For those of ordinary skill in the art, the specific meanings of the above terms in the present application can be understood according to specific circumstances.
[0036] In the present application, unless otherwise clearly specified and defined, the first feature being "on" or "under" the second feature may be that the first and second features are in direct contact, or the first and second features are in indirect contact through an intermediate medium. Moreover, the first feature being "above", "over" and "on top of" the second feature may be that the first feature is directly above or obliquely above the second feature, or merely indicates that the first feature has a higher horizontal height than the second feature. The first feature being "under", "below" and "beneath" the second feature may be that the first feature is directly below or obliquely below the second feature, or merely indicates that the first feature has a lower horizontal height than the second feature.
[0037] The total mileage of high-speed railways is long, the network scale is large, the coverage area is large, and there are many involved environments (such as plateaus, deserts, forests, debris flow-prone areas, bird protection areas, etc.), making the detection complex. First, it is necessary to conduct uninterrupted detection for 7*24 hours. There are large changes in light and shadow during the day and night, which easily lead to false subtraction and missed detection. Second, in the desert windy area, sand and dust are likely to block the camera, and strong winds can easily cause the camera to shake violently, resulting in unclear and distorted captured images, leading to false detection. Third, in forest areas and bird protection areas, changes in tree shadows and shadows projected by birds in the air are also likely to cause false detection. Fourth, the running speed of high-speed trains is fast, reaching 200-350 kilometers per hour. An intrusion may occur within a few minutes. If the detection speed is slow and the early warning is not timely, accidents will be triggered.
[0038] Some researchers use the inter-frame difference algorithm to perform differential operations on the digital images corresponding to two adjacent frames of the images captured by the railway cameras, obtaining an algorithm for the foreign object contour. This algorithm is simple and highly sensitive to light changes, but there are still many false detections and the recognition accuracy is not high. Some researchers use the convolutional neural network algorithm for detecting foreign object intrusion in high-speed railways. The convolutional neural network algorithm has better robustness to the environment, speeds up the calculation to a certain extent, and improves the recognition rate of small-size foreign object intrusion. However, there are still problems with a relatively high false detection and missed detection rates for new data and new scenarios in the training data.
[0039] To solve the above problems, the embodiments of the present application provide a method and system for detecting foreign object intrusion in high-speed railways. The following will detail the solution provided by the embodiments of the present application in conjunction with the accompanying drawings of the specification.
[0040] Figure 1 It is a schematic flowchart of the method for detecting foreign object intrusion in high-speed railways provided by an embodiment of the present application;
[0041] Figure 2 is Figure 1 a schematic diagram of the method for detecting foreign object intrusion in high-speed railways in
[0042] Referring to Figure 1 and Figure 2 shown, the embodiments of the present application provide a method for detecting foreign object intrusion in high-speed railways, including:
[0043] S100: Perform frame extraction on the high-speed railway monitoring video to obtain frame-by-frame detection images.
[0044] In some examples, there is no time interval for frame extraction, that is, each frame of the high-speed railway monitoring video will be detected. Specifically, take the first frame of the high-speed railway monitoring video as the reference frame, that is, the reference image. All frames other than the reference frame are used as detection frames, that is, detection images. In some other examples, a certain time interval can be determined to determine a certain frame as the reference frame. Preferably, an algorithm is used to detect whether there is a foreign object in the frame image.
[0045] S200: Input the reference image and the detection image into a change detection model to obtain a change detection result, where the reference image is an image at the same position but different times from the detection image.
[0046] Exemplarily, the change detection model includes a feature extraction module, a feature comparison module, a feature fusion module, and a classification head.
[0047] The reference image and the detection image are respectively sent to a feature extraction module with shared weights for feature processing to obtain two sets of multi-scale feature maps.
[0048] Among them, the feature extraction module is a Siamese convolutional network with ResNet-18 as the backbone. Two modifications are made to ResNet-18 in the feature extraction module. First, the classifier of ResNet-18 is removed; second, the size of each layer of feature maps is made 1 / 2 of the previous layer, thereby obtaining a set of feature maps with lengths and widths of 1 / 2, 1 / 4, 1 / 8, 1 / 16, and 1 / 32 of the input respectively, which are named MsResNet (Multiscale ResNet). The input S0, S1, S2, S3, and S4 are multiscale feature maps extracted using MsResNet.
[0049] The feature comparison module performs comparison processing on two sets of multiscale feature maps to obtain a set of multiscale feature difference maps;
[0050] In the feature comparison module, the absolute difference operation is used to calculate the difference between two sets of multi-sized feature maps from dual-temporal images. Let S1 i and S2 i be the feature maps of dual-temporal images extracted using MsResNet, and i = 0, 1, 2, 3, 4. Then the difference feature map Di calculated by the feature comparison module is:
[0051] Di = |S1 i - S2 i |
[0052] For the multiscale feature difference maps, the low-level feature difference maps contain more fine-grained local information, while the high-level ones contain coarser-grained semantic information. The fine-grained local information is beneficial to the detection of small objects, while the coarser-grained semantic information is beneficial to the detection of large objects. Reasonably combining the multiscale feature difference maps to obtain a feature map that contains both local information and semantic information is very helpful for obtaining correct prediction outputs.
[0053] Next, the feature fusion module fuses a set of multiscale feature difference maps into a feature difference map of the same scale. Usually, a bottom-up fusion method is adopted. However, since the feature extraction network designed for the classification task gradually increases the number of channels from the shallow layer to the deep layer, during the top-down fusion process, the number of channels gradually decreases. The process is as follows: First, the number of channels of the feature difference map is unified to that of the i-th layer, and Di-1 of the i-1-th layer is used for convolution operation. Then, the width and height of the feature map are unified to the width and height of the i-1-th layer using the upsampling operation. Finally, the new Di and Di-1 are concatenated to obtain a feature difference map fused into the same scale. However, this method will not only cause information loss in the fusion process of the multiscale feature difference maps, but also make the contributions of features at different scales to the detection results unbalanced.
[0054] When using the feature fusion module to fuse a set of multi-scale feature difference maps into a feature difference map of the same scale, first, use a 1×1 convolution to unify the features of each layer to m channels; then, perform 1×1 convolutions on the features of the i-th layer and the (i-1)-th layer respectively, then perform upsampling on the i-th layer to make its size the same as that of the (i-1)-th layer features, and then add the features of the i-th and (i-1)-th layers to obtain the new (i-1)-th layer features. After that, perform batch normalization and ReLU operations on the fused feature difference map. After multiple processes, a feature difference map of the same scale with multiple levels of information is obtained. When the upsampling uses linear interpolation, swap the adjacent upsampling operation and the 1×1 convolution, and the output of the model remains unchanged.
[0055] When the feature fusion module in this application performs fusion processing on multi-scale feature difference maps, the two tensors are first concatenated in channels and then a 1×1 convolution is performed, which is equivalent to performing 1×1 convolutions on them respectively and then adding them. The mathematical proof is as follows: Let the two tensors be A and B, with sizes h×w×m1 and h×w×m2 respectively, and the number of output channels be m3. Then the weight matrix W is m3×(m1 + m2), and A i,j , B i,j are all column vectors, then there is:
[0056]
[0057] In addition, when the upsampling uses linear interpolation, swapping the adjacent upsampling operation and the 1×1 convolution does not change the output of the model. The mathematical proof is as follows:
[0058] Let the input be X(h1×w1×c1), which is upsampled to obtain the intermediate result T(h2×w2×c1), and then passes through a 1×1 convolution module to obtain the output Y(h2×w2×c2). Then the interpolation function is:
[0059]
[0060] where g is the coefficient determined by the interpolation method.
[0061]
[0062] That is, the upsampling operation and the 1×1 can be swapped.
[0063] Finally, classification prediction is performed through the classification head to obtain the change detection result. Exemplarily, the classification head is configured as a 3×3 convolution.
[0064] S300: Input the change detection result into the result input alarm module to generate an alarm message.
[0065] It is understandable that subtle changes such as light and dust may cause noise points to appear in the detection results, resulting in false detections. To reduce the interference of these noise points, the result input alarm module determines whether to perform alarm processing based on the number of changed pixels detected in the change detection results; if the number of changed pixels is greater than or equal to the alarm threshold t, the result input alarm module generates an alarm message. If it does not exceed the alarm threshold t, it will be considered noise.
[0066] S400: The time-domain filtering module comprehensively judges the foreign object intrusion result based on multi-frame alarm information.
[0067] Exemplarily, in some examples, based on the above noise point tolerance mechanism, the time-domain filtering module is set to comprehensively judge the foreign object intrusion result based on multi-frame alarm information, including:
[0068] Let the length of the filtering window be l, and the final prediction result is determined by the l-frame time series information.
[0069]
[0070] Among them, Yi represents the final prediction result after filtering for the i-th frame; Yraw,i represents the original prediction result of the i-th frame without filtering.
[0071] To verify the feasibility of the high-speed railway foreign object intrusion detection method, the change detection image pairs are used as the training set, and the railway monitoring video is used as the test set. The training set contains a total of 11,810 pairs of images. The training image format is a three-channel png image of 1920×1080, and there is already pixel-level change detection annotation information. The test set is 37,712 frames, which is 1,885.6 seconds of railway monitoring video. Among them, 383 frames are three-channel videos of 1920×1088, and 37,329 frames are black-and-white videos of 1920×1080. Manual annotation is performed on the sorted test data to obtain the labels of whether there are foreign objects in each frame. To better evaluate the generalization ability of the model, each scene that appears in the test video has never appeared in the training set; for the convenience of testing and selecting reference frames, each video in the test set will contain at least 1 segment with foreign objects, and the first frame of each video will not contain foreign objects.
[0072] Through experiments, the effectiveness of the proposed noise point tolerance mechanism is verified, and the optimal alarm threshold t of the alarm module is selected. The experimental results with different alarm thresholds t are compared with the experimental results without using the noise point tolerance mechanism. The following table shows the comparison results of the per-frame missed detection rate and false detection rate under different alarm thresholds.
[0073]
[0074] It can be seen from the experimental results that the result of the noise point tolerance mechanism can well reduce the per-frame false detection rate, and at the same time has little impact on the per-frame missed detection rate.
[0075] In a second aspect, the embodiments of the present application provide a high-speed railway foreign object intrusion detection system, which uses the high-speed railway foreign object intrusion detection method described in any one of the first aspects to detect foreign object intrusion in high-speed railways.
[0076] It is easy to understand that those skilled in the art can combine, split, and reorganize the embodiments of the present application based on several embodiments provided in the present application to obtain other embodiments, and none of these embodiments exceeds the protection scope of the present application.
[0077] The above specific implementation manners further elaborate on the purpose, technical solutions, and beneficial effects of the embodiments of the present application. It should be understood that the above are only the specific implementation manners of the embodiments of the present application and are not used to limit the protection scope of the embodiments of the present application. Any modifications, equivalent replacements, improvements, etc. made on the basis of the technical solutions of the embodiments of the present application shall be included in the protection scope of the embodiments of the present application.
Claims
1. A method for detecting intrusion of foreign objects on high-speed railways, characterized in that, Including: Performing frame extraction on high-speed railway surveillance videos to obtain frame-by-frame detection images; Inputting the reference image and the detection image into a change detection model to obtain a change detection result, where the reference image is an image at the same position but at a different time from the detection image; Inputting the change detection result into a result input alarm module to generate an alarm message; The time-domain filtering module comprehensively judges the foreign object intrusion result based on multiple frames of alarm information.
2. The method for detecting foreign object intrusion in high-speed railway according to claim 1, characterized in that, The change detection model includes a feature extraction module, a feature comparison module, a feature fusion module, and a classification head; The reference image and the detection image are respectively sent to the feature extraction module with shared weights for feature processing to obtain two sets of multi-scale feature maps; Then, the feature comparison module performs comparison processing on the two sets of multi-scale feature maps to obtain a set of multi-scale feature difference maps; Next, the feature fusion module fuses a set of multi-scale feature difference maps into a feature difference map of the same scale; Finally, classification prediction is performed through the classification head to obtain a change detection result.
3. The high-speed railway intrusion detection method according to claim 2, wherein, The classification head is configured as a 3×3 convolution.
4. The method for detecting intrusion of foreign objects in high-speed railways according to claim 2, characterized in that, When using the feature fusion module to fuse a set of multi-scale feature difference maps into a feature difference map of the same scale, First, use 1×1 convolution to unify the features of each layer to m channels; Next, perform 1×1 convolution on the features of the i-th layer and the (i-1)-th layer respectively, then perform upsampling on the i-th layer to make its size the same as the features of the (i-1)-th layer, then add the features of the i-th and (i-1)-th layers to obtain the new (i-1)-th layer features, and then perform batch normalization and ReLU operations on the fused feature difference map. After multiple processes, a feature difference map of the same scale with multiple levels of information is obtained.
5. The method for detecting intrusion of foreign objects on high-speed railways according to claim 1, characterized in that, When the upsampling uses linear interpolation, swap the adjacent upsampling operation and 1×1 convolution without changing the output of the model.
6. The high-speed railway foreign object intrusion detection method according to claim 1, characterized in that, The result input alarm module judges whether to perform alarm processing according to the number of changed pixels detected in the change detection result; If the number of changed pixels is greater than or equal to the alarm threshold, the result input alarm module generates an alarm message.
7. The method for detecting intrusion of foreign objects on high-speed railways according to claim 1, characterized in that The time-domain filtering module comprehensively judges the foreign object intrusion result based on multiple frames of alarm information, including: Let the filter window length be l, and the final prediction result is determined by the l-frame time series information, where Yi represents the final prediction result of the i-th frame after filtering; Yraw,i represents the original prediction result of the i-th frame without filtering.
8. A high-speed railway foreign object intrusion detection system, characterized in that, Applying the high-speed railway foreign object intrusion detection method according to any one of claims 1-7 to detect foreign object intrusion on high-speed railways.