A method for detecting abnormality of roadbed cover of railway track

Abstract

The present invention relates to the field of image processing technology, and in particular to a method for detecting abnormalities in the roadbed cover of a railway track. The method comprises the following steps: using at least two cameras to synchronously capture images of the track area from different angles, calculating the depth value of the roadbed cover based on the parallax of the same target track area in the imaging of different cameras, and constructing a three-dimensional model of the roadbed cover; performing real-time compensation processing on the image during high-speed travel of the train; setting a periodic action sequence including different light intensities, focal length adjustments, and shooting frame rate changes, setting the duration of each action according to the train speed and camera shooting frequency, and collecting multiple sets of track images during the duration of each action. The present invention provides a method for detecting abnormalities in the roadbed cover of a railway track, which obtains the depth value of the roadbed cover, identifies the abnormality of the cover tilting, performs real-time compensation processing on the image, and improves the accuracy and efficiency of detection.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a method for detecting abnormalities in a roadbed cover of a railway track. Background Art

[0002] With the rapid development of railway transportation, the safe operation of railway tracks is of paramount importance. As a crucial component of railway tracks, the condition of track covers directly impacts the safety and stability of train operations. During high-speed train travel, track covers are prone to abnormalities such as missing or warping. Failure to promptly detect and repair these issues can easily lead to safety accidents.

[0003] Currently, roadbed cover inspection primarily relies on manual inspections and traditional detection methods. Manual inspections are inefficient, labor-intensive, and subject to significant human influence, making it difficult to ensure accurate and timely inspections. Traditional inspection methods, such as visual inspection based on a single camera, cannot capture depth information and can only detect obvious changes on the cover surface. However, subtle anomalies with depth variations, such as warping, are difficult to accurately detect. Furthermore, images can be blurred and distorted during high-speed train travel, further compromising inspection accuracy.

[0004] For example, the Chinese patent publication number CN119624945A, entitled "Line Feature Area Identification Method, System and Electronic Equipment for Track Geometry Detection", includes the following steps: determining the track line feature image to be identified; inputting the track line feature image to be identified into the line feature area recognition model to obtain the line feature area recognition result output by the line feature area recognition model; wherein the line feature area recognition model is obtained after training based on the sample data of the line feature image and the regional image annotation of the sample data; the line feature area recognition model is used to perform multiple feature judgments on the fusion data of the track line feature image to be identified and the track geometry detection waveform data after spatiotemporal synchronization to obtain the line feature area recognition result. The disadvantage is that only some obvious changes on the cover surface can be observed, and it is difficult to accurately detect subtle anomalies with depth changes such as warping. Summary of the Invention

[0005] In response to the problem that railway track anomaly detection in the existing technology cannot identify abnormal cover plate lift, the present invention provides a method for detecting abnormalities in the roadbed cover of a railway track, which obtains depth information of the roadbed cover, identifies abnormal cover plate lift, and performs real-time compensation processing on the image to improve the accuracy and efficiency of detection.

[0006] To achieve the above technical objectives, the present invention provides a technical solution, which is a method for detecting abnormalities in a roadbed cover of a railway track, comprising the following steps:

[0007] S1, using at least two cameras to synchronously capture images of the track area from different angles, calculating the depth value of the track bed cover based on the parallax of the same target track area in the images of different cameras, and constructing a three-dimensional model of the track bed cover;

[0008] S2, performs real-time compensation processing on the image while the train is running at high speed;

[0009] S3, setting a periodic action sequence including different lighting intensities, focus adjustments, and shooting frame rate changes. The duration of each action is set according to the train speed and camera shooting frequency, and multiple sets of track images are collected during each action.

[0010] S4 extracts the feature points of the image under each action, performs feature matching and depth value fusion, and determines whether the abnormality is caused by the roadbed cover itself, camera shooting parameters, or changes in train speed;

[0011] S5 builds a residual image regularity model based on the abnormal judgment results, performs real-time correction on subsequent images, generates abnormal candidate regions through the region proposal network, classifies and regresses the candidate regions, and determines whether the roadbed cover is missing or lifted, and locates it.

[0012] In this technical solution, at least two cameras simultaneously capture images of the track area from different angles, simulating binocular vision principles. This allows accurate calculation of the depth of the track cover and the construction of a detailed three-dimensional model. This provides rich spatial data for subsequent anomaly detection and facilitates a more comprehensive understanding of the track cover's condition. During high-speed train travel, real-time image compensation is performed on the images, effectively eliminating motion blur and enhancing contrast, ensuring clear, high-quality images even under high-speed conditions. A periodic action sequence is established, incorporating varying lighting intensities, focal length adjustments, and frame rate variations. The duration of each action is set based on the train's speed and camera capture frequency, enabling the detection method to adapt to diverse environmental conditions and improving its robustness and applicability. Feature points are extracted from the image for each action, and feature matching and depth value fusion are performed. This allows for accurate determination of whether anomalies originate from the track cover itself, camera capture parameters, or changes in train speed. This comprehensive consideration of multiple factors improves the accuracy and comprehensiveness of anomaly detection. Based on the abnormal results, a residual image regularity model is constructed, and the deep learning network is used to perform real-time correction and analysis on subsequent images. The abnormal candidate areas are generated through the region proposal network, and classification and position regression are performed. It can achieve accurate positioning and classification of abnormalities such as missing and lifted roadbed covers, which not only improves the detection accuracy, but also significantly improves the detection efficiency.

[0013] The present invention is further configured as follows: the method further includes camera setting optimization, and the camera setting optimization is:

[0014] A camera is set up to shoot at three different positions. The positions are selected according to the layout of the roadbed cover and the candidate areas of abnormalities. The camera is controlled to switch between different positions by a mechanical device. After each switch, the calibration device is used to calibrate the camera's internal and external parameters by shooting images of a calibration plate of known size. The differences between images at different positions are compared, and the images are stitched together using an image stitching algorithm. The differences between images at different positions in the stitched images are analyzed to assist in determining the cause of the abnormality.

[0015] Alternatively, three cameras are set up at the same location to capture images from different angles or using different parameters. When shooting at different angles, the angle selection ensures that information on different sides of the track area cover is obtained. When shooting with different parameters, the parameters include exposure time, focal length, and aperture. Automatic control and synchronous shooting technology are used to ensure that the image time base is the same. When comparing images, image registration and fusion technology are used to decompose the image into sub-bands of different scales. According to the characteristics of each sub-band of each scale of images from different cameras, the weighted average method is used for fusion. A corresponding relationship model between image differences and abnormal factors is established, and the cause of the abnormality is determined based on the difference points.

[0016] In this technical solution, a single camera is mechanically controlled to switch between three different positions, forming a multi-position camera switching strategy. This ensures comprehensive coverage of all areas of the roadbed cover, particularly those prone to anomalies. After each switch, a calibration device captures an image of a calibration plate of known dimensions to precisely calibrate the camera's intrinsic and extrinsic parameters. This ensures consistent geometric relationships between images captured at different positions, laying a solid foundation for subsequent image stitching and difference analysis. An image stitching algorithm seamlessly stitches images captured at different positions into a complete image, enabling inspectors to comprehensively visualize the condition of the roadbed cover. Furthermore, by analyzing the differences between images at different positions within the stitched image, the cause of anomalies can be more accurately determined, improving both detection accuracy and efficiency. Three cameras are positioned at the same location, capturing images from different angles or using different parameters (such as exposure time, focal length, and aperture), forming a multi-camera array strategy. This strategy provides more comprehensive and diverse information about the roadbed cover. This multi-angle, multi-parameter approach helps capture subtle changes in the roadbed cover, providing more clues for anomaly detection. Image registration and fusion techniques are used to precisely align and fuse images captured by different cameras. The fused image not only retains useful information from each camera but also incorporates features from sub-bands of different scales through a weighted average method, further enriching image features and improving the accuracy of anomaly detection. A model is established to correlate image differences with anomaly factors. The cause of anomalies is determined based on the differences in the fused image. This model can comprehensively consider multiple factors, such as lighting changes and differences in shooting angles, to more accurately determine whether the anomaly stems from the roadbed cover itself or other external factors.

[0017] The present invention is further configured as follows: in step S1, the depth value of the roadbed cover is calculated based on the parallax of the same target track area in different camera imaging, and the three-dimensional model of the roadbed cover is constructed, including: dividing the image obtained by one camera into multiple small image blocks, searching for the most similar area in the image of another camera, obtaining the pixel point parallax by calculating the image block displacement, calculating the depth value according to the triangulation principle, and constructing the three-dimensional model of the roadbed cover.

[0018] In this technical solution, the image captured by one camera is divided into multiple small image blocks, and the area most similar to it is searched in the image of the other camera. By calculating the displacement of the image blocks, the parallax of the pixel points is obtained, and then the depth value of the roadbed cover is accurately calculated based on the principle of triangulation. This method based on the stereo matching algorithm can fully utilize the principle of binocular vision to improve the accuracy and reliability of depth value calculation. Using the calculated depth value, a three-dimensional model of the roadbed cover can be constructed, which can intuitively display the shape, position and spatial relationship of the roadbed cover, providing rich three-dimensional spatial data for subsequent anomaly detection. Through the three-dimensional model, inspectors can have a more comprehensive understanding of the status of the roadbed cover, which helps to detect potential anomalies or defects. Based on the accurately calculated depth value and the constructed three-dimensional model, detailed inspection of the roadbed cover can be achieved. By comparing the three-dimensional model with the model in its normal state, anomalies or deformations of the roadbed cover can be more easily detected, which not only improves the accuracy of anomaly detection but also helps to detect potential safety hazards in advance. The stereo matching algorithm simulates the binocular vision principle and has strong robustness. It can work stably under different lighting conditions, shooting angles and distances. This robustness enables the method to adapt to various complex railway track environments and improves the stability and reliability of the system.

[0019] The present invention is further configured as follows: in step S2, performing real-time compensation processing on the image during high-speed travel of the train includes:

[0020] An algorithm based on a motion blur model is used to calculate the degree of image motion blur according to the train speed and camera shooting parameters, and the blurred image is restored through an inverse filtering method.

[0021] Feature points are extracted from continuous frame images through the feature point detection algorithm. The motion vectors between adjacent frames of the feature points are calculated according to the optical flow method to obtain the overall motion information of the image. The blurred image is reversely compensated based on the overall motion information of the image. The bilinear interpolation algorithm is used for resampling. The image is processed in different regions according to the adaptive histogram equalization algorithm.

[0022] In this technical solution, an algorithm based on a motion blur model, combined with train speed and camera parameters, accurately calculates the degree of motion blur in an image. Using an inverse filtering method, the blurred image is restored, effectively eliminating the blur caused by the high-speed train movement, resulting in sharper edges and richer details. A feature point detection algorithm is used to extract feature points from consecutive frames, and optical flow is used to calculate the motion vectors between adjacent frames of the feature points, thereby obtaining overall motion information. Based on this information, the blurred image is inversely compensated to correct image distortion caused by motion blur. Subsequently, a bilinear interpolation algorithm is used to resample the image, further smoothing it and eliminating aliasing or distortion caused by the compensation process, significantly improving image quality. An adaptive histogram equalization algorithm is used to process the image in different regions. This algorithm automatically adjusts contrast based on the brightness distribution of different image regions, making dark details clearer and bright details richer. Furthermore, this region-by-region processing approach avoids the noise amplification and detail loss associated with global histogram equalization, resulting in a more natural and balanced improvement in image contrast. By enhancing the contrast of the image, the texture and edges of the roadbed cover are more prominent, helping the subsequent anomaly detection algorithm to more accurately identify anomalies such as missing or warped roadbed covers. At the same time, the increased contrast also allows inspectors to observe the image more clearly, improving detection efficiency and accuracy.

[0023] The present invention is further configured as follows: in step S3, the feature points of the image under each action are extracted, and feature matching and depth value fusion are performed, including: using an image feature extraction and matching algorithm to process images taken under different actions, extracting feature points and calculating descriptors between feature points, matching feature points by comparing descriptor similarities; and analyzing the differences in feature points between images of different actions in combination with depth values.

[0024] This technical solution utilizes an image feature extraction and matching algorithm to accurately extract significant feature points in images captured under different motions and calculate descriptors between these feature points. By comparing the similarity of the descriptors, feature points in different images can be accurately matched, providing a reliable foundation for subsequent depth value fusion and anomaly detection. Combined with depth values, the differences in feature points between images captured under different motions are analyzed. Depth values provide the three-dimensional coordinates of feature points, helping to understand their spatial position and relationship. By analyzing feature point differences, it is possible to determine whether these differences are due to changes in the track cover itself, such as missing or warped anomalies, or to factors such as camera capture parameters or changes in train speed. This depth value fusion analysis method improves the accuracy and comprehensiveness of anomaly detection. Based on the results of feature matching and depth value fusion, the presence of anomalies in the track cover can be accurately determined and their location can be located. This not only improves the accuracy of anomaly detection but also facilitates timely repair measures, thereby ensuring safe operation of the railway track. The feature extraction and matching algorithm boasts high computational efficiency and can rapidly process large amounts of image data. At the same time, combined with the depth value analysis, the detection method has stronger robustness to factors such as different lighting conditions, shooting angles and distances, and can work stably in various complex railway track environments.

[0025] The present invention is further configured such that: in step S4, the determining that the abnormality is caused by a change in the roadbed cover body, camera shooting parameters, or train speed includes:

[0026] By comparing the changes in the matching values and depth values of feature points under different action sequences, if the feature point matching fails or the depth value has a local depth mutation or the surface is uneven, it is determined that the roadbed cover body is abnormal;

[0027] Analyze the impact of different exposure times, focal lengths, and apertures on image features. If feature point matching is successful but overall depth offset or image blur occurs after depth fusion, it is determined that the camera shooting parameters are abnormal.

[0028] The duration of the action sequence is set according to the train speed and camera shooting frequency. If motion blur or image misalignment appears in the feature point matching and depth value fusion results, it is determined to be an abnormality caused by the change in train speed.

[0029] In this technical solution, when feature point matching fails or the depth values show a sudden local depth change or surface unevenness, it is determined to be an anomaly in the roadbed cover. This allows direct detection of physical changes in the roadbed cover, such as missing or warped parts, providing accurate information for timely repair measures. When feature point matching succeeds but depth fusion results in overall depth offset or image blur, it is determined to be an anomaly in the camera capture parameters. This allows identification of image quality issues caused by improper camera parameter settings (such as exposure time, focal length, and aperture), facilitating adjustment of camera parameters for higher-quality images. The duration of the action sequence is set based on the train's speed and camera capture frequency. When motion blur or image misalignment is observed in the feature point matching and depth value fusion results, it is determined to be an anomaly caused by a change in train speed. This allows detection of the impact of train speed changes on the image acquisition and detection process, helping to optimize the detection algorithm to accommodate varying train speeds. By comprehensively considering multiple factors to determine the source of an anomaly, it avoids misjudgments and missed detections caused by a single factor, improving the accuracy of anomaly determination and making the detection method more comprehensive and reliable.

[0030] The present invention is further configured as follows: in step S5, constructing the afterimage regularity model based on the abnormal result of the judgment includes:

[0031] The BakeMesh method is used to create the afterimage effect by duplicating the geometric model of the roadbed cover and adjusting its transparency and position at different time points;

[0032] Using screen post-processing technology, the collected images of the roadbed cover are post-processed to generate an afterimage effect. By comparing the afterimage images at different time points, the movement trajectory and state changes of the roadbed cover are analyzed.

[0033] The vertex position of the roadbed cover geometric model is adjusted through the vertex offset method to simulate its state changes at different time points and generate a continuous afterimage effect.

[0034] This technical solution utilizes the BakeMesh method to replicate the geometric model of the roadbed cover and adjust its transparency and position at different time points to create an afterimage effect. This method visually demonstrates the state changes of the roadbed cover at different time points, providing analysts with a clearer visual experience and allowing them to more clearly observe the movement trajectory and state change trends of the roadbed cover. Using screen post-processing technology, the captured roadbed cover images are post-processed to generate the afterimage effect, further enhancing the fidelity of the afterimage effect, making the afterimage image closer to reality and providing analysts with more accurate visual information. Using the vertex offset method, the vertex positions of the roadbed cover geometric model are adjusted to simulate its state changes at different time points, generating a continuous afterimage effect. This allows for more precise control over the afterimage generation process, resulting in a more continuous and natural afterimage effect, providing analysts with a smoother observation experience. By comparing afterimage images at different time points, the movement trajectory and state changes of the roadbed cover can be analyzed, helping to understand the movement patterns and state change trends of the roadbed cover at different time points, providing important data support for subsequent anomaly detection and state assessment. At the same time, by analyzing the afterimage patterns, the future status changes of the roadbed cover can also be predicted, providing a basis for taking repair measures in advance.

[0035] The present invention is further configured such that: in step S5, the real-time correction of the subsequent image includes:

[0036] The collected roadbed cover images are preprocessed by denoising and enhancing contrast based on the convolutional neural network;

[0037] Integrate a multi-head self-attention module into a deep learning network to capture complex features in roadbed cover images;

[0038] A cross-layer weighted cascade structure is designed to integrate deep network insights with shallow layer information and optimize the regression convergence of defect boundaries.

[0039] In this technical solution, a convolutional neural network is used to preprocess the collected roadbed cover images by denoising and enhancing contrast. This effectively removes noise from the images, improving image clarity and contrast. This makes subsequent feature extraction and defect detection more accurate, providing higher-quality image data for the entire inspection process. Integrating a multi-head self-attention module into the deep learning network enables the network to capture complex features in roadbed cover images. The self-attention mechanism focuses on key information in the image, while the multi-head design captures diverse features, improving the diversity and robustness of feature extraction. This facilitates a more comprehensive understanding of the roadbed cover's condition and improves the accuracy of anomaly detection. A cross-layer weighted cascade structure is designed to integrate the insights of the deep network with shallow-layer information, optimizing the regression convergence of defect boundaries. Leveraging the strengths of both deep and shallow networks, this approach improves the detection accuracy and convergence speed of defect boundaries, enabling the detection model to more accurately locate roadbed cover defects and provide more precise information for subsequent repair work. The detection model can dynamically adjust to image changes at different time points and in different environments, improving the real-time and accuracy of detection. This makes the entire detection process more efficient and reliable, and provides strong support for abnormal detection of roadbed covers on railway tracks.

[0040] The present invention is further configured such that: in step S5, generating abnormal candidate regions by using a region proposal network includes:

[0041] Extract implicit semantic features from the original input image based on convolutional neural networks;

[0042] The region proposal network is learned through supervised learning methods to generate a set of target proposals at each position of the feature map, which are used as candidate regions for abnormalities.

[0043] The candidate regions generated by the region proposal network are classified and regressed, and the characteristics of railway images are learned by training a large amount of data.

[0044] In this technical solution, a region proposal network (RPN) uses supervised learning to generate a set of object proposals at each location in the feature map, which serve as candidate anomaly regions. This efficiently generates candidate regions, avoiding the computational complexity of exhaustively enumerating all candidate anomaly regions in traditional methods, significantly improving detection efficiency. The object proposals generated by the RPN are based on implicit semantic features extracted from the original input image by a convolutional neural network. They capture key information within the image, allowing the generated object proposals to more accurately locate candidate anomaly regions. This helps the subsequent classification and position regression steps more accurately identify anomalies. Classification and position regression are performed on the candidate regions generated by the RPN, learning the characteristics of railway images through extensive training data. This makes classification and position regression more accurate, thereby improving detection accuracy. Furthermore, since the candidate regions have already been preliminarily screened, subsequent processing only focuses on these regions, further improving detection precision. The RPN's supervised learning approach allows the method to adapt to railway track cover anomaly detection in diverse scenarios. Regardless of lighting conditions, shooting angles, or track cover types, this method generates accurate candidate regions, providing a reliable foundation for subsequent detection.

[0045] The present invention is further configured such that the optical axis of the camera forms a certain angle with the roadbed plane to ensure that the field of view of the two cameras completely covers the roadbed cover area. The angle between the optical axis of the camera and the roadbed plane is calculated as follows:

[0046] ;

[0047] Where θ is the angle between the camera optical axis and the track bed plane, W is the width of the track bed, and H is the height of the camera relative to the track bed plane when the train is running.

[0048] The beneficial effects of the present invention are as follows: (1) the depth value of the roadbed cover is obtained, the abnormal lifting of the cover is identified, and the image is compensated in real time to improve the accuracy and efficiency of detection; (2) at least two cameras are used to synchronously collect images of the track area from different angles, simulating the binocular vision principle, and accurately calculating the depth value of the roadbed cover, constructing a fine three-dimensional model, providing rich three-dimensional spatial data for subsequent abnormality detection, and helping to more comprehensively understand the status of the roadbed cover. During the high-speed running of the train, the image is compensated in real time, effectively eliminating motion blur and enhancing contrast, ensuring that clear and high-quality images can be obtained even in a high-speed motion environment. A periodic action sequence including different light intensities, focus adjustment and shooting frame rate changes is set, and the image is adjusted according to the train movement. The duration of each action is set according to the driving speed and camera shooting frequency, so that the detection method can adapt to different environmental conditions, improve the robustness and applicability of the detection, extract the feature points of the image under each action, perform feature matching and depth value fusion, and accurately judge whether the anomaly is caused by the roadbed cover itself, the camera shooting parameters or the change of train driving speed. Taking into account multiple factors, the accuracy and comprehensiveness of anomaly judgment are improved. Based on the judgment of anomaly results, a residual image law model is constructed, and the subsequent images are corrected and analyzed in real time using the deep learning network. The candidate areas of anomalies are generated by the region proposal network, and classification and position regression are performed. It can achieve accurate positioning and classification of anomalies such as missing and lifted roadbed covers, which not only improves the accuracy of detection, but also significantly improves the detection efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 Schematic diagram of the flow of the method for detecting abnormality of the roadbed cover of the railway track according to the present invention;

[0050] Figure 2 The structure diagram of the roadbed cover in the present invention is as follows Figure 1 ;

[0051] Figure 3 The structure diagram of the roadbed cover in the present invention is as follows Figure 2 ;

[0052] Figure 4 The structure diagram of the roadbed cover in the present invention is as follows Figure 3 . DETAILED DESCRIPTION

[0053] In order to make the objectives, technical solutions and advantages of the present invention more clear, the present invention is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific implementation method described herein is only an optimal embodiment of the present invention, which is only used to explain the present invention and does not limit the scope of protection of the present invention. All other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0054] like Figures 1 to 4 As shown in the first embodiment of the present invention, a method for detecting abnormality of a roadbed cover of a railway track is characterized by comprising the following steps:

[0055] S1, using at least two cameras to synchronously capture images of the track area from different angles, calculating the depth value of the track bed cover based on the parallax of the same target track area in the images of different cameras, and constructing a three-dimensional model of the track bed cover;

[0056] S2, performs real-time compensation processing on the image while the train is running at high speed;

[0057] S3, setting a periodic action sequence including different lighting intensities, focus adjustments, and shooting frame rate changes. The duration of each action is set according to the train speed and camera shooting frequency, and multiple sets of track images are collected during each action.

[0058] S4 extracts the feature points of the image under each action, performs feature matching and depth value fusion, and determines whether the abnormality is caused by the roadbed cover itself, camera shooting parameters, or changes in train speed;

[0059] S5 builds a residual image regularity model based on the abnormal judgment results, performs real-time correction on subsequent images, generates abnormal candidate regions through the region proposal network, classifies and regresses the candidate regions, and determines whether the roadbed cover is missing or lifted, and locates it.

[0060] In this embodiment, at least two cameras simultaneously capture images of the track area from different angles. Using a stereo matching algorithm to simulate binocular vision, the depth of the track cover can be accurately calculated, creating a detailed three-dimensional model. This provides rich spatial data for subsequent anomaly detection and facilitates a more comprehensive understanding of the track cover's condition. During high-speed train travel, real-time image compensation is performed on the images, effectively eliminating motion blur and enhancing contrast, ensuring clear, high-quality images even under high-speed conditions. A periodic action sequence is established, including varying light intensity, focus adjustment, and frame rate. The duration of each action is set based on the train's speed and camera capture frequency. This allows the detection method to adapt to diverse environmental conditions, improving its robustness and applicability. Feature points are extracted from the image for each action, and feature matching and depth value fusion are performed. This allows for accurate determination of whether anomalies originate from the track cover itself, camera capture parameters, or changes in train speed. This comprehensive consideration of multiple factors improves the accuracy and comprehensiveness of anomaly detection. Based on the abnormal results, a residual image regularity model is constructed, and the deep learning network is used to perform real-time correction and analysis on subsequent images. The abnormal candidate areas are generated through the region proposal network, and classification and position regression are performed. It can achieve accurate positioning and classification of abnormalities such as missing and lifted roadbed covers, which not only improves the detection accuracy, but also significantly improves the detection efficiency.

[0061] In one embodiment of the present invention, the method further includes camera setting optimization, wherein the camera setting optimization is:

[0062] A camera is set up to shoot at three different positions. The positions are selected according to the layout of the roadbed cover and the candidate areas of abnormalities. The camera is controlled to switch between different positions by a mechanical device. After each switch, the calibration device is used to calibrate the camera's internal and external parameters by shooting images of a calibration plate of known size. The differences between images at different positions are compared, and the images are stitched together using an image stitching algorithm. The differences between images at different positions in the stitched images are analyzed to assist in determining the cause of the abnormality.

[0063] Alternatively, three cameras are set up at the same location to capture images from different angles or using different parameters. When shooting at different angles, the angle selection ensures that information on different sides of the track area cover is obtained. When shooting with different parameters, the parameters include exposure time, focal length, and aperture. Automatic control and synchronous shooting technology are used to ensure that the image time base is the same. When comparing images, image registration and fusion technology are used to decompose the image into sub-bands of different scales. According to the characteristics of each sub-band of each scale of images from different cameras, the weighted average method is used for fusion. A corresponding relationship model between image differences and abnormal factors is established, and the cause of the abnormality is determined based on the difference points.

[0064] In this technical solution, a single camera is controlled by a mechanical device to switch between three different positions, forming a multi-position camera switching strategy to ensure that the camera can fully cover all areas of the roadbed cover, especially those areas prone to abnormalities. After each switch, a calibration device is used to capture an image of a calibration plate of known size to accurately calibrate the camera's internal and external parameters, thereby ensuring that the images captured at different positions have consistent geometric relationships, laying a solid foundation for subsequent image stitching and difference analysis. Using an image stitching algorithm, images captured at different positions are seamlessly stitched together into a complete image, making it easier for inspectors to fully observe the status of the roadbed cover. At the same time, by analyzing the differences between images at different positions in the stitched image, it is possible to more accurately assist in determining the cause of the abnormality, thereby improving the accuracy and efficiency of detection.

[0065] Three cameras are set up at the same location to capture images from different angles or using different parameters (such as exposure time, focal length, aperture, etc.), forming a multi-camera array strategy. This allows for more comprehensive and diverse information about the roadbed cover. This multi-angle, multi-parameter shooting method helps capture subtle changes in the roadbed cover, providing more clues for anomaly detection. Using image registration and fusion technology, images captured by different cameras are precisely aligned and fused. The fused image not only retains the useful information captured by each camera, but also incorporates the features of sub-bands of different scales through a weighted averaging method, further enriching the image features and improving the accuracy of anomaly detection. A model is established to establish the correspondence between image differences and abnormal factors. The cause of the anomaly is determined based on the differences in the fused image. This model can comprehensively consider multiple factors, such as changes in lighting and differences in shooting angles, to more accurately determine whether the anomaly originates from the roadbed cover itself or other external factors.

[0066] As can be understood, image registration is the process of matching and overlaying two or more images acquired at different times, using different sensors (imaging devices), or under different conditions (such as weather conditions, illumination, camera position and angle). The specific steps include feature detection, matching, transformation model calculation, and image resampling. Feature detection can be manual or automatic detection of significant and unique objects (such as closed boundary regions, edges, contours, intersections, corner points, etc.). Feature matching establishes the correlation between the features of the scene image and the reference image. Transformation model calculation is used to calculate the type and parameters of the mapping function that aligns the sensed image with the reference image. Image resampling transforms the sensed image using the mapping function and uses appropriate interpolation techniques to calculate image values at non-integer coordinates.

[0067] As you can understand, image fusion involves combining image data collected from multiple channels, focusing on the same target, with image processing and computer technology to maximize the extraction of useful information from each channel, ultimately synthesizing a high-quality image. Image fusion aims to improve image quality and interpretation accuracy, enhance the reliability of computer interpretation, and improve the spatial and spectral resolution of the original image. Fusion methods include multi-sensor image fusion (such as visible light and infrared image fusion) and single-sensor multi-focus image fusion.

[0068] In one embodiment of the present invention, in step S1, the calculation of the depth value of the roadbed cover based on the parallax of the same target track area in different camera imaging includes: dividing the image acquired by one camera into multiple small image blocks, searching for the most similar area in the image of another camera, obtaining the pixel parallax by calculating the displacement of the image blocks, calculating the depth value according to the triangulation principle, and constructing a three-dimensional model of the roadbed cover.

[0069] In this technical solution, the image captured by one camera is divided into multiple small image blocks, and the area most similar to it is searched in the image of the other camera. By calculating the displacement of the image blocks, the disparity of the pixel points is obtained, and then the depth value of the roadbed cover is accurately calculated based on the principle of triangulation. This method based on the stereo matching algorithm can fully utilize the principle of binocular vision to improve the accuracy and reliability of depth value calculation. Using the calculated depth value, a three-dimensional model of the roadbed cover can be constructed, which can intuitively display the shape, position, and spatial relationship of the roadbed cover, providing rich three-dimensional spatial data for subsequent anomaly detection. Through the 3D model, inspectors can gain a more comprehensive understanding of the status of the roadbed cover, helping to detect potential anomalies or defects. Based on the accurately calculated depth value and the constructed 3D model, detailed inspection of the roadbed cover can be achieved. By comparing the 3D model with the model in its normal state, anomalies or deformations of the roadbed cover can be more easily detected, which not only improves the accuracy of anomaly detection but also helps to identify potential safety hazards in advance. The stereo matching algorithm simulates the binocular vision principle and has strong robustness. It can work stably under different lighting conditions, shooting angles and distances. This robustness enables the method to adapt to various complex railway track environments and improves the stability and reliability of the system.

[0070] As you can understand, in stereo matching, the most similar region refers to the point where most of the points visible in one view are also visible in the other view, and the matched image regions are similar in appearance. To find the most similar region, a dense correlation method is usually used, that is, for each pixel, the most similar matching point is found in the other image. This can be achieved by calculating the similarity of a small window (patch) around the pixel, for example using the sum of squared differences or cross-correlation methods.

[0071] As you can understand, triangulation is a key technology for acquiring three-dimensional information in computer vision. It utilizes epipolar geometry and homography matrices to account for camera motion and feature point depth. Specifically, for matching feature points in two images, their coordinates in three-dimensional space can be calculated using the principle of triangulation. This principle involves epipolar geometry constraints and homography matrices, and typically requires least squares methods to approximate the depth of feature points. In practice, due to noise, two lines of sight (the lines connecting the optical centers of the two cameras to the feature points) often fail to precisely intersect.

[0072] As you can understand, the method for creating a 3D model of a trackbed cover is similar to that of a rail, and can be constructed using 2D contour reconstruction technology. The key is to determine the position of the trackbed cover, which can be determined based on the position of the track centerline and the track gauge. By obtaining the 2D contour of the trackbed cover and utilizing 3D reconstruction technology from computer graphics, a 3D model of the trackbed cover can be constructed, allowing for a more intuitive display and analysis of its structure and status.

[0073] In one embodiment of the present invention, in step S2, performing real-time compensation processing on the image during high-speed travel of the train includes:

[0074] An algorithm based on a motion blur model is used to calculate the degree of image motion blur according to the train speed and camera shooting parameters, and the blurred image is restored through an inverse filtering method.

[0075] Feature points are extracted from continuous frame images through the feature point detection algorithm. The motion vectors between adjacent frames of the feature points are calculated according to the optical flow method to obtain the overall motion information of the image. The blurred image is reversely compensated based on the overall motion information of the image. The bilinear interpolation algorithm is used for resampling. The image is processed in different regions according to the adaptive histogram equalization algorithm.

[0076] In this technical solution, an algorithm based on a motion blur model, combined with train speed and camera parameters, accurately calculates the degree of motion blur in an image. Using an inverse filtering method, the blurred image is restored, effectively eliminating the blur caused by the high-speed train movement, resulting in sharper edges and richer details. A feature point detection algorithm is used to extract feature points from consecutive frames, and optical flow is used to calculate the motion vectors between adjacent frames of the feature points, thereby obtaining overall motion information. Based on this information, the blurred image is inversely compensated to correct image distortion caused by motion blur. Subsequently, a bilinear interpolation algorithm is used to resample the image, further smoothing it and eliminating any aliasing or distortion caused by the compensation process, significantly improving image quality. An adaptive histogram equalization algorithm is used to process the image in different regions. This algorithm automatically adjusts contrast based on the brightness distribution of different image regions, enhancing detail in dark areas and detail in bright areas. Furthermore, this region-by-region processing approach avoids the noise amplification and detail loss associated with global histogram equalization, resulting in a more natural and balanced improvement in image contrast. By enhancing the contrast of the image, the texture and edges of the roadbed cover are more prominent, helping the subsequent anomaly detection algorithm to more accurately identify anomalies such as missing or warped roadbed covers. At the same time, the increased contrast also allows inspectors to observe the image more clearly, improving detection efficiency and accuracy.

[0077] As you can understand, algorithms based on motion blur models are used to simulate and address image blur caused by rapid object movement. Motion blur algorithms include linear motion blur, rotational motion blur, and scaling motion blur. These algorithms typically leverage the property of visual inertia: when light acting on the human eye suddenly disappears, the perceived brightness does not disappear immediately, but rather gradually decreases exponentially. By simulating this motion blur effect, images can be deblurred or generated with motion blur.

[0078] As you can understand, there are multiple methods for calculating the degree of image motion blur, including Fourier transform, wavelet transform, Laplace operator, and gradient method. The Fourier transform method implements high-pass filtering by masking the central region of the spectrum, retaining high-frequency information such as image edges, and then calculating the mean of the spectrum to assess the degree of blur. The wavelet transform method uses wavelet transform to decompose the image at multiple scales, extracting high-frequency detail information, and then determines the degree of blur by analyzing the energy distribution of this detail information. The Laplace operator method and gradient method respectively reflect edge information by calculating the second-order guide map and gradient of the image to assess the degree of blur.

[0079] As you can understand, feature point detection algorithms are used to detect points with significant features in an image, such as corners and edges. Common feature point detection algorithms include FAST, SURF, and SIFT. The FAST algorithm detects corners by comparing pixel brightness, making it fast. The SURF and SIFT algorithms extract feature points by constructing a scale pyramid and calculating the main orientation of key points. They are scale-invariant and rotation-invariant, making them suitable for feature point detection in images of varying scales and rotation angles.

[0080] As you can understand, optical flow is a technique in computer vision that calculates the motion of each pixel in an image between consecutive frames. The Lucas-Kanade optical flow algorithm is a commonly used optical flow algorithm. It calculates the motion vector of each pixel by matching images within a local window and assuming that the pixels within the window have the same motion. The algorithm calculates the motion vector of each pixel within the window by minimizing an error function, which typically involves solving a system of linear equations. Optical flow has a wide range of applications in motion calculation, object tracking, and video compression.

[0081] As you can understand, bilinear interpolation is an image interpolation algorithm used to calculate the pixel values of a new image during operations such as image scaling and rotation. The algorithm uses the pixel values of the four nearest neighbors in the original image and calculates the pixel values of the new image using a bilinear interpolation formula. Bilinear interpolation is widely used in image processing and computer vision, and is effective in maintaining image smoothness and continuity.

[0082] As can be understood, the adaptive histogram equalization algorithm is a technology used for image enhancement. It changes the image contrast by calculating the local histogram of the image and redistributing the brightness. Unlike the traditional histogram equalization method, the adaptive histogram equalization performs equalization based on the local area of the image, thus avoiding the problems of over-enhancement or distortion caused by global equalization. The algorithm can significantly enhance the contrast of local areas and retain image details, and is suitable for images with uneven contrast. In the railway track roadbed cover anomaly detection method, the adaptive histogram equalization algorithm can be used to enhance the collected roadbed cover images, improve the contrast and clarity of the image, and thus more accurately detect roadbed cover anomalies.

[0083] It can be understood that restoring a blurred image by an inverse filtering method includes the following steps:

[0084] Perform Fourier transform on the blurred image to convert it from the spatial domain to the frequency domain to obtain the blur function;

[0085] Calculate the Fourier transform of the blur function;

[0086] In the frequency domain, an estimate of the Fourier transform of the original image is obtained by dividing the blur function by the Fourier transform of the blur function;

[0087] An inverse Fourier transform is performed on the estimate of the Fourier transform of the original image, which is converted back to the spatial domain to obtain the restored image.

[0088] In step S3, the feature points of the image under each action are extracted and feature matching and depth value fusion are performed, including: using image feature extraction and matching algorithms to process images taken under different actions, extracting feature points and calculating descriptors between feature points, matching feature points by comparing descriptor similarities; and analyzing the differences in feature points between images of different actions in combination with depth values.

[0089] This technical solution utilizes an image feature extraction and matching algorithm to accurately extract significant feature points in images captured under different motions and calculate descriptors between these feature points. By comparing the similarity of the descriptors, feature points in different images can be accurately matched, providing a reliable foundation for subsequent depth value fusion and anomaly detection. Combined with depth values, the differences in feature points between images captured under different motions are analyzed. Depth values provide the three-dimensional coordinates of feature points, helping to understand their spatial position and relationship. By analyzing feature point differences, it is possible to determine whether these differences are due to changes in the track cover itself, such as missing or warped anomalies, or to factors such as camera capture parameters or changes in train speed. This depth value fusion analysis method improves the accuracy and comprehensiveness of anomaly detection. Based on the results of feature matching and depth value fusion, the presence of anomalies in the track cover can be accurately determined and their location can be located. This not only improves the accuracy of anomaly detection but also facilitates timely repair measures, thereby ensuring safe operation of the railway track. The feature extraction and matching algorithm boasts high computational efficiency and can rapidly process large amounts of image data. At the same time, combined with the depth value analysis, the detection method has stronger robustness to factors such as different lighting conditions, shooting angles and distances, and can work stably in various complex railway track environments.

[0090] Preferably, in step S4, determining whether the abnormality is caused by a change in the roadbed cover body, camera shooting parameters, or train speed includes:

[0091] By comparing the changes in the matching values and depth values of feature points under different action sequences, if the feature point matching fails or the depth value has a local depth mutation or the surface is uneven, it is determined that the roadbed cover body is abnormal;

[0092] Analyze the impact of different exposure times, focal lengths, and apertures on image features. If feature point matching is successful but overall depth offset or image blur occurs after depth fusion, it is determined that the camera shooting parameters are abnormal.

[0093] The duration of the action sequence is set according to the train speed and camera shooting frequency. If motion blur or image misalignment appears in the feature point matching and depth value fusion results, it is determined to be an abnormality caused by the change in train speed.

[0094] In this technical solution, when feature point matching fails or the depth values show a sudden local depth change or surface unevenness, it is determined to be an anomaly in the roadbed cover. This allows direct detection of physical changes in the roadbed cover, such as missing or warped parts, providing accurate information for timely repair measures. When feature point matching succeeds but depth fusion results in overall depth offset or image blur, it is determined to be an anomaly in the camera capture parameters. This allows identification of image quality issues caused by improper camera parameter settings (such as exposure time, focal length, and aperture), facilitating adjustment of camera parameters for higher-quality images. The duration of the action sequence is set based on the train's speed and camera capture frequency. When motion blur or image misalignment is observed in the feature point matching and depth value fusion results, it is determined to be an anomaly caused by a change in train speed. This allows detection of the impact of train speed changes on the image acquisition and detection process, helping to optimize the detection algorithm to accommodate varying train speeds. By comprehensively considering multiple factors to determine the source of an anomaly, it avoids misjudgments and missed detections caused by a single factor, improving the accuracy of anomaly determination and making the detection method more comprehensive and reliable.

[0095] In step S5, the construction of the afterimage regularity model based on the abnormality determination result includes:

[0096] The BakeMesh method is used to create the afterimage effect by duplicating the geometric model of the roadbed cover and adjusting its transparency and position at different time points;

[0097] Using screen post-processing technology, the collected images of the roadbed cover are post-processed to generate an afterimage effect. By comparing the afterimage images at different time points, the movement trajectory and state changes of the roadbed cover are analyzed.

[0098] The vertex position of the roadbed cover geometric model is adjusted through the vertex offset method to simulate its state changes at different time points and generate a continuous afterimage effect.

[0099] This technical solution utilizes the BakeMesh method to replicate the geometric model of the roadbed cover and adjust its transparency and position at different time points to create an afterimage effect. This method visually demonstrates the state changes of the roadbed cover at different time points, providing analysts with a clearer visual experience and allowing them to more clearly observe the movement trajectory and state change trends of the roadbed cover. Using screen post-processing technology, the captured roadbed cover images are post-processed to generate the afterimage effect, further enhancing the fidelity of the afterimage effect, making the afterimage image closer to reality and providing analysts with more accurate visual information. Using the vertex offset method, the vertex positions of the roadbed cover geometric model are adjusted to simulate its state changes at different time points, generating a continuous afterimage effect. This allows for more precise control over the afterimage generation process, resulting in a more continuous and natural afterimage effect, providing analysts with a smoother observation experience. By comparing afterimage images at different time points, the movement trajectory and state changes of the roadbed cover can be analyzed, helping to understand the movement patterns and state change trends of the roadbed cover at different time points, providing important data support for subsequent anomaly detection and state assessment. At the same time, by analyzing the afterimage patterns, it is also possible to predict future changes in the status of the roadbed cover, providing a basis for taking repair measures in advance.

[0100] Preferably, in step S5, the real-time correction of the subsequent image includes:

[0101] The collected roadbed cover images are preprocessed by denoising and enhancing contrast based on the convolutional neural network;

[0102] Integrate a multi-head self-attention module into a deep learning network to capture complex features in roadbed cover images;

[0103] A cross-layer weighted cascade structure is designed to integrate deep network insights with shallow layer information and optimize the regression convergence of defect boundaries.

[0104] The collected roadbed cover images are pre-processed using a convolutional neural network to denoise and enhance contrast. This effectively removes noise from the images, improving image clarity and contrast. This makes subsequent feature extraction and defect detection more accurate, providing higher-quality image data for the entire inspection process. A multi-head self-attention module is integrated into the deep learning network, enabling the network to capture complex features in the roadbed cover images. The self-attention mechanism focuses on key information within the image, while the multi-head design captures diverse features, improving the diversity and robustness of feature extraction. This facilitates a more comprehensive understanding of the roadbed cover's condition and enhances the accuracy of anomaly detection. A cross-layer weighted cascade architecture is designed to integrate insights from the deep network with shallow-layer information, optimizing the regression convergence of defect boundaries. Leveraging the strengths of both deep and shallow networks, this approach improves the detection accuracy and convergence speed of defect boundaries, enabling the detection model to more accurately locate roadbed cover defects and provide more precise information for subsequent repair work. The detection model can dynamically adjust to image changes at different time points and in different environments, enhancing the real-time and accuracy of detection. This makes the entire detection process more efficient and reliable, and provides strong support for abnormal detection of roadbed covers on railway tracks.

[0105] The camera optical axis forms a certain angle with the roadbed plane to ensure that the field of view of the two cameras completely covers the roadbed cover area. The angle between the camera optical axis and the roadbed plane is calculated as follows:

[0106] ;

[0107] Where θ is the angle between the camera optical axis and the track bed plane, W is the width of the track bed, and H is the height of the camera relative to the track bed plane when the train is running.

[0108] In step S5, generating abnormal candidate regions through the region proposal network includes:

[0109] Extract implicit semantic features from the original input image based on convolutional neural networks;

[0110] The region proposal network is learned through supervised learning methods to generate a set of target proposals at each position of the feature map, which are used as candidate regions for abnormalities.

[0111] The candidate regions generated by the region proposal network are classified and regressed, and the characteristics of railway images are learned by training a large amount of data.

[0112] The region proposal network (RPN) uses supervised learning to generate a set of object proposals at each location in the feature map, which serve as candidate anomaly regions. This efficient generation of candidate regions avoids the computational overhead of exhaustively enumerating all candidate anomaly regions in traditional methods, significantly improving detection efficiency. The object proposals generated by the RPN are based on implicit semantic features extracted from the original input image by a convolutional neural network. They capture key information within the image, enabling the generated object proposals to more accurately locate candidate anomaly regions. This helps the subsequent classification and position regression steps more accurately identify anomalies. Classification and position regression are performed on the candidate regions generated by the RPN. By training on a large amount of data, the RPN learns the characteristics of railway images. This results in more accurate classification and position regression, thereby improving detection accuracy. Furthermore, since the candidate regions have already been preliminarily screened, subsequent processing only focuses on these regions, further enhancing detection precision. The RPN's supervised learning approach allows the method to adapt to railway track cover anomaly detection in diverse scenarios. Regardless of lighting conditions, shooting angles, or track cover types, the method generates accurate candidate regions, providing a reliable foundation for subsequent detection.

[0113] The specific description of the present invention in the above embodiments is only used to further illustrate the present invention and cannot be understood as limiting the scope of protection of the present invention. Technical engineers in this field may make some non-essential improvements and adjustments to the present invention based on the contents of the above invention, which fall within the scope of protection of the present invention.

Claims

1. A method for detecting abnormalities in a railway track bed cover, characterized in that: The following steps are involved: S1, using at least two cameras to synchronously capture images of the track area from different angles, calculating the depth value of the track bed cover based on the parallax of the same target track area in the images of different cameras, and constructing a three-dimensional model of the track bed cover; S2, performs real-time compensation processing on the image while the train is running at high speed; S3, setting a periodic action sequence including different lighting intensities, focus adjustments, and shooting frame rate changes. The duration of each action is set according to the train speed and camera shooting frequency, and multiple sets of track images are collected during each action. S4 extracts the feature points of the image under each action, performs feature matching and depth value fusion, and determines whether the abnormality is caused by the roadbed cover itself, camera shooting parameters, or changes in train speed; By comparing the changes in the matching values and depth values of feature points under different action sequences, if the feature point matching fails or the depth value has a local depth mutation or the surface is uneven, it is determined that the roadbed cover body is abnormal; Analyze the impact of different exposure times, focal lengths, and apertures on image features. If feature point matching is successful but overall depth offset or image blur occurs after depth fusion, it is determined that the camera shooting parameters are abnormal. The duration of the action sequence is set according to the train speed and camera shooting frequency. If motion blur or image misalignment appears in the feature point matching and depth value fusion results, it is determined to be an abnormality caused by the change in train speed.

2. The method for detecting abnormality of a railway track track cover according to claim 1, characterized in that: The method further includes camera setting optimization, wherein the camera setting optimization is: A camera is set up to shoot at three different positions. The positions are selected according to the layout of the roadbed cover and the candidate areas of abnormalities. The camera is controlled to switch between different positions by a mechanical device. After each switch, the calibration device is used to calibrate the camera's internal and external parameters by shooting images of a calibration plate of known size. The differences between images at different positions are compared, and the images are stitched together using an image stitching algorithm. The differences between images at different positions in the stitched images are analyzed to assist in determining the cause of the abnormality. Alternatively, three cameras are set up at the same location to capture images from different angles or using different parameters. When shooting at different angles, the angle selection ensures that information on different sides of the track area cover is obtained. When shooting with different parameters, the parameters include exposure time, focal length, and aperture. Automatic control and synchronous shooting technology are used to ensure that the image time base is the same. When comparing images, image registration and fusion technology are used to decompose the image into sub-bands of different scales. According to the characteristics of each sub-band of each scale of images from different cameras, the weighted average method is used for fusion. A corresponding relationship model between image differences and abnormal factors is established, and the cause of the abnormality is determined based on the difference points.

3. The method for detecting abnormality of a railway track track cover according to claim 1, characterized in that: In step S1, the depth value of the roadbed cover is calculated based on the parallax of the same target track area in different camera imaging, and the three-dimensional model of the roadbed cover is constructed, which includes: dividing the image obtained by one camera into multiple small image blocks, searching for the most similar area in the image of another camera, obtaining the pixel point parallax by calculating the displacement of the image blocks, calculating the depth value according to the triangulation principle, and constructing the three-dimensional model of the roadbed cover.

4. The method for detecting abnormality of a railway track track cover according to claim 1, wherein: In step S2, performing real-time compensation processing on the image during high-speed train travel includes: An algorithm based on a motion blur model is used to calculate the degree of image motion blur according to the train speed and camera shooting parameters, and the blurred image is restored through an inverse filtering method. Feature points are extracted from continuous frame images through the feature point detection algorithm. The motion vectors between adjacent frames of the feature points are calculated according to the optical flow method to obtain the overall motion information of the image. The blurred image is reversely compensated based on the overall motion information of the image. The bilinear interpolation algorithm is used for resampling. The image is processed in different regions according to the adaptive histogram equalization algorithm.

5. The method for detecting abnormality of a railway track track cover according to claim 1, characterized in that: In step S3, the feature points of the image under each action are extracted and feature matching and depth value fusion are performed, including: using image feature extraction and matching algorithms to process images taken under different actions, extracting feature points and calculating descriptors between feature points, matching feature points by comparing descriptor similarities; and analyzing the differences in feature points between images of different actions in combination with depth values.

6. A method for detecting abnormalities in a railway track track cover according to any one of claims 1 to 5, characterized in that: The camera optical axis forms a certain angle with the roadbed plane to ensure that the field of view of the two cameras completely covers the roadbed cover area. The angle between the camera optical axis and the roadbed plane is calculated as follows: ； Where θ is the angle between the camera optical axis and the track bed plane, W is the width of the track bed, and H is the height of the camera relative to the track bed plane when the train is running.

Images