Steel rail comprehensive detection method and device

Abstract

The invention discloses a comprehensive detection method and device for a steel rail. The method comprises the steps that multi-modal original data collected by a multi-modal sensor are obtained; performing weighted fusion processing on the multi-modal original data to generate fusion feature data; on the basis of the fused feature data, identifying a potential defect area of the steel rail, extracting multi-modal feature response data in the potential defect area, and calculating the correlation degree between the feature response data of different modals; and comparing the correlation degree with a preset correlation degree threshold value, determining that the potential defect area is a real defect area according to a comparison result, and generating a steel rail defect detection report. According to the invention, the false alarm rate under complex working conditions is reduced, so that the accuracy of steel rail defect detection is improved.

Description

Technical Field

[0001] This invention relates to the field of rail inspection technology, and in particular to a comprehensive rail inspection method and apparatus. Background Technology

[0002] As a critical load-bearing structure of railway tracks, the health of rails directly affects train operation safety. Traditional rail inspection technologies mainly rely on single inspection modes, such as optical visual inspection, electromagnetic eddy current inspection, or ultrasonic inspection. However, single-mode inspection methods have inherent limitations when facing complex and ever-changing railway field conditions: optical inspection is easily affected by rail surface contamination, water stains, oil stains, and lighting conditions, resulting in false defects or missed detections; electromagnetic inspection is affected by uneven rail material, surface oxide layers, and residual stress, resulting in poor signal stability; and ultrasonic inspection has limited detection capabilities for certain internal defects due to factors such as coupling state fluctuations, surface blind zones, and internal structural noise.

[0003] With the increasing demands for detection accuracy and reliability, rail inspection technology is evolving from single-modal to multi-modal information fusion. Some existing multi-modal inspection systems attempt to integrate various sensors, including optical, electromagnetic, and ultrasonic sensors. However, these systems are essentially just a simple superposition of multiple individual detection methods, failing to achieve true synergy and integration. This results in an inability to adapt to complex working conditions such as humidity, corrosion, oil contamination, high speeds, and curved sections, leading to inaccurate rail inspection results and a high false alarm rate.

[0004] Therefore, a comprehensive rail inspection method is urgently needed to solve the above problems. Summary of the Invention

[0005] This invention provides a comprehensive rail inspection method to reduce the false alarm rate under complex working conditions and improve the accuracy of rail defect detection. The method includes:

[0006] Acquire multimodal raw data collected by multimodal sensors; the multimodal raw data includes optical image data, 3D point cloud data, electromagnetic response data and ultrasonic echo data under the same cross section of the rail;

[0007] The original multimodal data is weighted and fused to generate fused feature data;

[0008] Based on fused feature data, potential defect areas of rails are identified, multimodal feature response data are extracted from potential defect areas, and the correlation between feature response data of different modes is calculated.

[0009] The correlation degree is compared with the preset correlation degree threshold. Based on the comparison result, the potential defect area is confirmed as the real defect area, and a rail defect detection report is generated.

[0010] This invention also provides a comprehensive rail inspection device to reduce the false alarm rate under complex working conditions and improve the accuracy of rail defect detection. The device includes:

[0011] The multimodal data acquisition module is used to acquire multimodal raw data collected by multimodal sensors; the multimodal raw data includes optical image data, three-dimensional point cloud data, electromagnetic response data and ultrasonic echo data under the same cross section of the rail;

[0012] The feature fusion module is used to perform weighted fusion processing on the original multimodal data to generate fused feature data;

[0013] The correlation calculation module is used to identify potential defect areas of rails based on fused feature data, extract multimodal feature response data from potential defect areas, and calculate the correlation between feature response data of different modes.

[0014] The rail defect detection report generation module is used to compare the correlation degree with a preset correlation degree threshold, confirm the potential defect area as the real defect area based on the comparison result, and generate a rail defect detection report.

[0015] This invention also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the above-described comprehensive rail inspection method.

[0016] This invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described comprehensive rail inspection method.

[0017] This invention also provides a computer program product, which includes a computer program that, when executed by a processor, implements the above-described comprehensive rail inspection method.

[0018] In this embodiment of the invention, multimodal raw data collected by a multimodal sensor is acquired. This multimodal raw data includes optical image data, 3D point cloud data, electromagnetic response data, and ultrasonic echo data from the same cross-section of the rail. The multimodal raw data is weighted and fused to generate fused feature data. Based on the fused feature data, potential defect areas of the rail are identified, and multimodal feature response data from these potential defect areas is extracted. The correlation between the feature response data of different modes is calculated. The correlation is compared with a preset correlation threshold, and based on the comparison result, the potential defect area is confirmed as a real defect area, generating a rail defect detection report. In this process, this embodiment of the invention acquires multimodal raw data from the same cross-section of the rail and performs weighted fusion processing to generate fused feature data. After identifying potential defect areas based on the fused feature data, the feature response data of these areas under multimodal conditions is further extracted. By calculating the correlation between the feature response data of different modes and comparing it with a preset correlation threshold, the real defect area is effectively identified, and a detection report is generated. This reduces the false alarm rate under complex working conditions, thereby improving the overall accuracy of rail defect detection. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. In the drawings:

[0020] Figure 1 This is a flowchart of the comprehensive rail inspection method in an embodiment of the present invention;

[0021] Figure 2 This is a flowchart illustrating the generation of fused feature data in an embodiment of the present invention;

[0022] Figure 3 This is a flowchart of obtaining feature response data in an embodiment of the present invention;

[0023] Figure 4 This is a schematic diagram of the rail comprehensive inspection device in an embodiment of the present invention;

[0024] Figure 5 This is a schematic diagram of a computer device in an embodiment of the present invention. Detailed Implementation

[0025] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the embodiments of the present invention will be further described in detail below with reference to the accompanying drawings. Here, the illustrative embodiments of the present invention and their descriptions are used to explain the present invention, but are not intended to limit the present invention.

[0026] Figure 1 This is a flowchart of a comprehensive rail inspection method according to an embodiment of the present invention. The method includes:

[0027] Step 101: Acquire the multimodal raw data collected by the multimodal sensor; the multimodal raw data includes optical image data, three-dimensional point cloud data, electromagnetic response data and ultrasonic echo data under the same cross section of the rail;

[0028] Step 102: Perform weighted fusion processing on the original multimodal data to generate fused feature data;

[0029] Step 103: Based on the fused feature data, identify potential defect areas of the rail, extract multimodal feature response data from the potential defect areas, and calculate the correlation between feature response data of different modes.

[0030] Step 104: Compare the correlation degree with the preset correlation degree threshold, confirm the potential defect area as the real defect area based on the comparison result, and generate a rail defect detection report.

[0031] Each step is explained in detail below.

[0032] In step 101, multimodal raw data collected by the multimodal sensor is acquired; the multimodal raw data includes optical image data, three-dimensional point cloud data, electromagnetic response data and ultrasonic echo data under the same cross section of the rail.

[0033] In a specific embodiment, the rail inspection system is mounted on a rail inspection vehicle and includes the following core sensor units. All units are rigidly fixed to the same inspection beam, and their installation positions are precisely calibrated:

[0034] 1. Two-dimensional optical unit: It adopts a high-resolution area array camera and is equipped with a coaxial light source to acquire two-dimensional texture images of the rail surface, i.e., optical image data;

[0035] 2. Three-dimensional optical unit: A three-dimensional scanning camera based on structured light projection is used to acquire the three-dimensional topography point cloud of the rail surface, i.e., three-dimensional point cloud data.

[0036] 3. Magnetic eddy current unit: integrates a magnetoacoustic surface wave probe and an array of eddy current probes to excite and receive electromagnetic response data characterizing the shallow surface state of the rail.

[0037] 4. Ultrasonic Unit: Employs a multi-channel ultrasonic probe wheel with water as a coupling agent to collect ultrasonic echo data reflecting the internal condition of the rail (such as the rail head and rail web).

[0038] Through the above synchronous acquisition process, the rail inspection system finally outputs a set of multimodal raw data.

[0039] In a specific embodiment, the spatial collaborative layout of hardware and the spatiotemporal alignment algorithm of software fundamentally solve the problem of spatiotemporal correlation discontinuity of multi-source data, providing a precise and homogeneous data foundation for subsequent fusion and verification.

[0040] First, there is the spatial collaborative layout of multimodal sensors:

[0041] 1) Coordinate system establishment: A two-dimensional coordinate system is defined based on the cross-section of the measured rail. The origin is set at the theoretical center of the rail head, the X-axis is along the width of the rail, and the Y-axis is along the height of the rail. This coordinate system serves as a unified reference for spatial alignment of all sensor data.

[0042] 2) Optical Unit Layout: The 2D area array camera and the 3D structured light camera are rigidly mounted side-by-side on the same detection beam along the X-axis. Through precise optical design, the overlap area of ​​their optical fields of view on the target rail cross-section is ensured to be ≥95%. This design guarantees that the texture information and 3D topography information of the same surface area can be accurately mapped one-to-one at the pixel level / point cloud level.

[0043] 3) Electromagnetic and Ultrasonic Unit Layout: The magnetoacoustic surface wave and arrayed eddy current composite probe are arranged along the Y-axis, with a 15-25mm overlap between their effective detection area and the coverage of the optical unit in the Y-axis direction. The multi-channel ultrasonic probe wheel is also arranged along the Y-axis closer to the rail head. This layered focusing layout aims to achieve continuous coverage and data correlation of the same physical cross-section from the surface (optical), shallow layer (electromagnetic), to the interior (ultrasonic).

[0044] 4) Synchronous triggering mechanism: The acquisition actions of all sensor units are triggered by a unified central controller. The trigger signal is phase-locked with the pulse signal of the high-precision odometer encoder and the satellite timing (GNSS) signal to ensure the global synchronization of the acquisition events.

[0045] Secondly, there is the processing of spatiotemporal synchronization and coordinate unification:

[0046] 1) Time Synchronization: Utilizing high-precision time signals (such as PPS pulses) provided by the GNSS module, each frame of acquired data is timestamped with microsecond-level accuracy. Combined with the output of a high-precision odometer encoder, the timestamps are converted into precise mileage location information, achieving "time-mileage" synchronization. The mileage synchronization error is controlled within ≤1 meter.

[0047] 2) Spatial Coordinate Unification: During the data preprocessing stage, all 2D image pixels, 3D point clouds, electromagnetic signal scan lines, and ultrasonic A-scan time series are mapped to the aforementioned "rail cross-section coordinate system" through a rigid body transformation model. System calibration is performed on-site using a calibration plate and standard test blocks to ensure that the spatial alignment accuracy of each modal data within this coordinate system is ≤0.1mm. This eliminates spatial misalignment caused by differences in the physical positions of the sensors.

[0048] Finally, a set of original multimodal datasets of rail cross-sections with unified spatiotemporal references and consistent spatial coordinates is output.

[0049] In one embodiment, acquiring the raw multimodal data collected by the multimodal sensor further includes:

[0050] The multimodal raw data is aligned to obtain aligned multimodal raw data; the alignment process includes spatiotemporal synchronization and spatial coordinate unification.

[0051] Among them, spatiotemporal synchronization uses a unified synchronization clock signal and mileage encoding signal to assign a unified timestamp and mileage location label to the multimodal raw data;

[0052] Among them, the spatial coordinate unification means mapping the multimodal raw data to the same coordinate system based on the rail cross section.

[0053] By achieving spatiotemporal synchronization and unified spatial coordinate processing, the spatiotemporal misalignment and coordinate inconsistency issues caused by independent acquisition of multi-source sensor data were eliminated. Optical, 3D, electromagnetic, and ultrasonic data were aligned to a unified spatiotemporal and geometric benchmark of the same rail cross-section. This laid a data foundation for the accurate calculation of subsequent multimodal feature correlations, ensuring the effectiveness of the overall detection process and the accuracy of the final results.

[0054] In step 102, the multimodal raw data is weighted and fused to generate fused feature data.

[0055] In a specific embodiment, intelligent fusion of multimodal data is achieved through a dynamic weight fusion algorithm adapted to operating conditions. This algorithm, based on the logic of analyzing the physical effects of operating conditions, quantifying reliability, and dynamically adapting weights, dynamically adjusts the contribution weights of each modality of data in the fusion process. The specific implementation process is as follows:

[0056] 1. Confidence Quantification: A confidence score is calculated for the raw data of each modality to quantify the reliability of the current detection results. The confidence score aims to internally assess the intrinsic quality of the output signal of each modality at this moment, such as the texture sharpness of optical images, the signal-to-noise ratio of electromagnetic signals, and the stability of ultrasonic echoes. These assessments are based on digital signal processing and image processing algorithms (such as improved LBP, FFT, wavelet transform, DTW, etc.) to quantify the impact of the sensor's own state and environmental random noise.

[0057] 1) Calculation of the reliability (C1) of optical image data: For optical image data, an improved LBP algorithm is used to extract the surface texture features of the rail, and the texture entropy value (H) is calculated. According to the formula:

[0058]

[0059] Texture sharpness scores are calculated, where Hmax and Hmin are preset standard values. Simultaneously, the Canny operator is used for edge detection, and the average gradient magnitude (G) of defect edges is calculated according to the formula:

[0060]

[0061] Calculate the edge sharpness score. Finally, determine the optical image confidence level:

[0062]

[0063] Where 'a' is a preset weighting coefficient.

[0064] 2) Calculation of 3D point cloud data reliability (C2): Statistically count the number of point clouds per unit area (D) within the region of interest of the rail cross-section, and calculate: This serves as a point cloud density score. Simultaneously, the point cloud data is compared with a standard rail CAD model to calculate the mean absolute error (E) of depth measurement, according to the formula:

[0065]

[0066] Calculate the depth error score. Finally, determine the credibility of the 3D point cloud. , where b is a preset weighting coefficient.

[0067] 3) Calculation of electromagnetic response data credibility (C3): Perform Fast Fourier Transform (FFT) on the magnetoacoustic surface wave signal, calculate its signal-to-noise ratio (SNR), and obtain the SNR score. For arrayed eddy current signals, the similarity (M) between the signal and a preset defect signal template is calculated using the Dynamic Time Warping (DTW) algorithm, and this similarity is used as the feature matching score. Finally, the reliability of the electromagnetic response data is determined. , where c is the preset weighting coefficient.

[0068] 4) Calculation of Ultrasonic Echo Data Reliability (C4): The ultrasonic echo signal is denoised using wavelet transform, and the ratio (R) of the defect echo peak value to the background noise is calculated to obtain the echo sharpness score. Simultaneously, the peak values ​​of multiple consecutive echo signals are analyzed, and their coefficient of variation (CV) is calculated to obtain a coupling stability score. Ultimately, the reliability of the ultrasound echo data... , where d is the preset weighting coefficient.

[0069] 2. Determination of Operating Condition Correction Coefficients: Operating condition parameters, including humidity (H), corrosion level (R), oil coverage (O), detection speed (V), and curve curvature (C), are acquired in real time by a dedicated sensor module. Based on a pre-defined physical influence model, correction coefficients (Ki) for each modal data under the current operating condition are calculated. The aim is to establish a modal attenuation model for each typical operating condition based on its physical mechanism and quantify it as a correction coefficient. For example:

[0070] 1) Damp working conditions

[0071] Technical basis: A water film forms on the wet rail surface. The specular reflection effect of this film obscures surface texture details (consistent with the laws of optical reflection; the refractive index of the water film, n≈1.33, differs significantly from that of air, n≈1), resulting in blurred edges of defects in the two-dimensional optical modes. Water is a weakly conductive medium, having little impact on the damping of magnetoacoustic surface wave propagation (magnetoacoustic surface waves rely on the magnetoelastic coupling effect of ferromagnetic materials, which the water film lacks), only slightly absorbing signal energy. The two-dimensional optical modes are affected by reflection, with its correction coefficient K_1 decreasing as the humidity increases. The magnetic eddy current composite signal is slightly affected, with its correction coefficient K_3 decreasing slightly with increasing humidity.

[0072] Correction calculations: Humidity parameter H is acquired by a humidity sensor. The two-dimensional optical mode correction coefficient K_1 = 1 - 0.01 × min(H - 60, 40) (K_1 = 1 when H ≤ 60; K_1 = 0.6 when H ≥ 100); the magnetic eddy current composite signal correction coefficient K_3 = 1 - 0.005 × min(H - 60, 40) (K_3 = 1 when H ≤ 60; K_3 = 0.8 when H ≥ 100); the three-dimensional optical mode K_2 = 1, and the ultrasonic mode K_4 = 1 (unaffected by humid conditions).

[0073] 2) Severe corrosion conditions (corrosion level ≥ 3)

[0074] Technical basis: The rust layer has a loose and porous structure (mainly Fe2O3, a non-conductive and non-ferromagnetic material), and its color differs greatly from the rail body, generating false textures that interfere with two-dimensional optical recognition. It also increases the propagation loss of magnetoacoustic surface waves and reduces the penetration depth of array eddy currents (consistent with the principle of electromagnetic induction). Two-dimensional texture recognition is affected by rust, and its correction coefficient K_1 decreases with increasing rust level; the magnetic eddy current composite signal is affected by the oxide layer, and its correction coefficient K_3 decreases with increasing rust level.

[0075] Correction calculations: The corrosion level R is identified and output using a deep learning model through visual image recognition (levels 1-5). The two-dimensional texture correction coefficient K_1 = 1 - 0.2 × (R-2) (K_1 = 1 when R ≤ 2; K_1 = 0.4 when R ≥ 5); the magnetic eddy current composite signal correction coefficient K_3 = 1 - 0.15 × (R-2) (K_3 = 1 when R ≤ 2; K_3 = 0.55 when R ≥ 5); the three-dimensional optical mode K_2 = 1 - 0.05 × (R-2) (K_2 = 1 when R ≤ 2; K_2 = 0.85 when R ≥ 5); and the ultrasonic mode K_4 = 1 (basically unaffected by corrosion).

[0076] 3) High-speed operating conditions

[0077] Technical basis: Ultrasonic testing relies on a sufficient number of sampling points. As the testing speed increases, the sampling time per unit length decreases, and the pulse emission frequency of the ultrasonic probe cannot be increased synchronously (limited by the hardware sampling rate), resulting in insufficient sampling of defect echoes (consistent with the sampling theorem). Since the ultrasonic sampling rate is limited, its correction coefficient K_4 decreases appropriately as the testing speed increases.

[0078] Correction calculation: The detection speed V (unit: km / h) is acquired by a speed sensor. The ultrasonic modal correction coefficient K_4 = 1 - 0.005 × min(V - 80, 40) (K_4 = 1 when V ≤ 80; K_4 = 0.8 when V ≥ 120); other modes K_1 = K_2 = K_3 = 1 (not significantly affected by high-speed conditions).

[0079] 4) Oily working conditions

[0080] Technical Basis: Oil contamination is a transparent or semi-transparent insulating medium that fills the surface's minute textures and reduces contrast (optical scattering theory), hinders the energy transfer of magnetoacoustic surface waves, and increases the impedance of eddy current interfaces (impedance matching principle). Three-dimensional optics only focuses on the physical contours and is largely unaffected by the track surface. When oil contamination such as engine oil or grease is present, the two-dimensional texture is obscured by the oil contamination, and its correction coefficient K_1 decreases as the oil contamination coverage increases. The magnetic eddy current composite signal is affected by the insulation of the oil contamination, and its correction coefficient K_3 decreases as the oil contamination thickness increases. Because three-dimensional optics only focuses on the physical contours, its correction coefficient K_2 remains basically stable.

[0081] Correction calculations: Oil stain coverage O (unit: %) is identified and output using a deep learning model through visual image recognition. The two-dimensional texture correction coefficient K_1=1-0.01×min(O,100) (K_1=1 when O=0; K_1=0 when O=100); the magnetic eddy current composite signal correction coefficient K_3=1-0.008×min(O,100) (K_3=1 when O=0; K_3=0.2 when O=100); the three-dimensional optical mode K_2=1, and the ultrasonic mode K_4=1-0.003×min(O,100) (K_4=1 when O=0; K_4=0.7 when O=100).

[0082] 5) Curved Section Working Conditions

[0083] Technical basis: Rail side wear in curved sections causes the rail head reference plane to shift, resulting in distortion of structured light fringes (triangulation geometry principle) and deviation in 3D profile detection; irregular rail head shape leads to unstable ultrasonic coupling, causing echo refraction and scattering (ultrasonic propagation geometry principle). When the rail is located on a curved section, the correction coefficient K_2 for 3D profile detection decreases with increasing side wear due to the reference shift caused by rail head side wear; the correction coefficient K_4 for ultrasonic detection decreases with increasing curve curvature due to echo interference caused by irregular rail head shape; the combined signal of 2D optical and magnetic eddy current is less affected by shape, and its correction coefficient remains basically stable.

[0084] Correction calculations: Curve curvature C is acquired by an inertial sensor; 3D contour correction coefficient K_2=1-0.1×min(C,5) (K_2=1 when C≤0; K_2=0.5 when C≥5); Ultrasonic detection correction coefficient K_4=1-0.08×min(C,5) (K_4=1 when C≤0; K_4=0.6 when C≥5); 2D optical mode K_1=1; Magnetic eddy current composite signal K_3=1 (less affected by shape).

[0085] 3. Dynamic Weight Calculation and Feature Fusion: Based on the current detection task (e.g., focusing on surface or internal defects), the basic weights (αi) for each modality are set. The dynamic fusion weights (Wi) are calculated according to the formula... The calculation proceeds further. Weights can be enhanced based on the identified defect type (e.g., for surface defects, increase the weights for optical and 3D modalities). Finally, the defect feature vectors extracted from each modality are weighted and fused using dynamic weights (Wi) to generate comprehensive fused feature data.

[0086] Figure 2 This is a flowchart illustrating the generation of fused feature data in an embodiment of the present invention. In one embodiment, weighted fusion processing is performed on the original multimodal data to generate fused feature data, including:

[0087] Step 201: Obtain working condition parameters reflecting the current rail condition; based on the working condition parameters and the quality assessment results of the original data of each mode, determine the fusion weight corresponding to each mode data; wherein the quality assessment results are determined based on at least one of the following indicators: signal clarity, feature stability, or noise level of the corresponding mode data.

[0088] Step 202: Use fusion weights to perform weighted fusion of the defect-related feature data extracted from each modality data to generate fused feature data for characterizing defect information.

[0089] In a specific embodiment, the quality assessment results of the raw data for each modality are calculated, and the specific process is as follows:

[0090] For quality assessment of optical image data, the corresponding texture entropy value is calculated based on the optical image data to evaluate noise level and texture sharpness. In potential defect regions in the image, the Canny operator is used to calculate the average gradient magnitude of the edges to evaluate the sharpness of the defect contours.

[0091] For quality assessment of 3D point cloud data, the number of points per unit area in the rail region is calculated to evaluate the density stability of the data acquisition. Simultaneously, the point cloud is compared with a standard model to calculate the contour error and assess the noise level of the measurement.

[0092] For quality assessment of electromagnetic response data, spectral analysis is performed on the magnetoacoustic surface wave signal to calculate its signal-to-noise ratio, which directly reflects the clarity and noise level of the signal.

[0093] For quality assessment of ultrasound data, the ratio of defect echo peak value to background noise is calculated to measure the signal clarity of the echo. Simultaneously, the coefficient of variation of echo amplitudes across multiple consecutive frames is analyzed to evaluate the characteristic stability of the signal caused by changes in coupling state.

[0094] In a specific embodiment, feature data related to the current potential defect is extracted from the modal alignment data. For example, the grayscale distribution vector of the potential defect region is extracted from the optical image; the depth distribution vector of the region is extracted from the 3D point cloud; the amplitude and phase feature vectors of the abnormal signal are extracted from the magnetic eddy current signal; and the arrival time and amplitude features of the defect reflection wave are extracted from the ultrasonic echo.

[0095] The system multiplies the values ​​of each dimension of each feature data by its corresponding modal weight, and then concatenates or adds all the weighted feature vectors element-wise to form a fused feature vector. By quantifying the quality assessment results of each modal data in real time and considering the physical effects of specific operating conditions, the fusion weights are dynamically adjusted. This solves the problem of fixed-weight fusion failing as a whole due to the failure of a single sensor under complex operating conditions. For example, in humid and oily conditions, the system automatically reduces the weights of susceptible optical modes.

[0096] In step 103, based on the fused feature data, potential defect areas of the rail are identified, multimodal feature response data in the potential defect areas are extracted, and the correlation between feature response data of different modes is calculated.

[0097] In a specific embodiment, after identifying potential defect areas, this invention uses a multi-dimensional cross-validation mechanism to quantitatively assess the authenticity of defects in those areas. This mechanism includes two dimensions: feature correlation verification and parameter deviation verification. The specific implementation process is as follows:

[0098] 1. Feature Correlation (S1) Calculation: This assesses the spatial or morphological consistency of the features exhibited by the same defect across different modal data. True defects (such as cracks and scratches) should show spatial or geometric correlation under sensors based on different physical principles. False defects (such as oil stains or rust spots) typically only appear in a single or a few modalities.

[0099] For surface defects (such as scratches): The 2D contour region (A) of the defect is extracted from optical image feature data using the Otsu algorithm and morphological processing. From 3D point cloud feature data, the depth difference with the standard model is calculated to extract the depth anomaly region (B). A and B are then pixel-level registered to the same coordinate system, and the percentage of their overlapping area is calculated. It can effectively distinguish between oil stains (which have optical changes but no or very shallow three-dimensional deformation) and real scratches (which are highly overlapping).

[0100] For shallow cracks: The geometric center of the electromagnetic signal anomaly region is determined from the electromagnetic response feature data using envelope detection and centroid algorithms. The centerline of the crack texture is obtained from the optical image feature data using the Zhang-Suen skeleton extraction algorithm, and its center point is determined. The ratio of the Euclidean distance (d) between the two center points to the crack feature length (L) is calculated to obtain... The focal point of the electromagnetic response of a real crack should correspond to the location of the main line of the visual crack.

[0101] For defects in curved sections: ICP registration is performed between the 3D point cloud feature data and the standard railhead CAD model to calculate the offset (Δ) of the railhead center. Where W is the standard railhead width. This is used to adapt to the baseline changes caused by curve wear, and to avoid misjudging normal wear as a defect.

[0102] 2. Parameter Deviation (S2) Calculation: Utilizing the measurement capabilities of different modes for the same defect's physical parameters (such as depth), cross-validation is performed based on physical laws. The measured parameter values ​​of real defects should exhibit reasonable consistency.

[0103] For crack depth: The depth (h_em) is calculated from the electromagnetic response characteristic data based on the magnetoacoustic surface wave propagation time. The depth (h_ultra) is calculated from the ultrasonic echo characteristic data using the pulse reflection method. The deviation between the two methods is calculated. Based on the principle that the propagation speeds of elastic waves (magnetoacoustic surface waves) and acoustic waves (ultrasound) in steel are not significantly different, the measurement results for the same crack depth should be similar. Significant discrepancies in crack depth may indicate a false defect or misjudgment.

[0104] For scratch depth: The maximum depth (h_3d) is directly extracted from the 3D point cloud feature data. Based on the gray-level gradient of the optical image feature data, the depth (h_2d) is derived using a pre-defined gray-level-depth mapping model. Calculation Using 3D measurements as the true value, the reliability of depth derivation based on optical image shadow / grayscale models is verified.

[0105] For defects under conditions of heavy oil pollution: Depth (h_3d) is extracted from 3D point cloud feature data. The shallow surface thickness (t) is inverted from the electromagnetic response feature data based on the array eddy current impedance variation. Calculation... For oil-contaminated conditions, correlation verification was conducted by leveraging the fact that oil contamination does not affect three-dimensional geometric measurements and that eddy currents are sensitive to near-surface anomalies.

[0106] 3. Consistency Comprehensive Scoring and Judgment: The feature correlation degree (S1) and parameter deviation degree (S2) are fused according to a preset weight (e) to obtain a comprehensive consistency verification score. This score is compared to a preset judgment threshold (e.g., 0.7) as the core basis for determining the authenticity of defects in subsequent steps. A relatively high judgment threshold (e.g., 0.7) is set. Only when feature correlation and parameter verification show a high degree of consistency can the V value exceed the threshold, thus being judged as a high-confidence genuine rail defect. Otherwise, it will be marked as a suspected defect, prompting manual review, thereby significantly reducing false alarms caused by rail-specific pseudo-defects such as rust, oil stains, and oxide layers.

[0107] Figure 3 This is a flowchart illustrating the acquisition of feature response data in an embodiment of the present invention. In one embodiment, based on fused feature data, potential defect areas of the rail are identified, and multimodal feature response data in the potential defect areas are extracted, including:

[0108] Step 301: Based on the defect information represented by the fused feature data, locate the potential defect area of ​​the rail;

[0109] Step 302: Extract optical image feature data, three-dimensional point cloud feature data, electromagnetic response feature data and ultrasonic echo feature data corresponding to potential defect areas from the multimodal raw data, as multimodal feature response data.

[0110] In a specific embodiment, a preset judgment threshold is set for the fused feature data. When the value of the fused feature vector exceeds the judgment threshold, a potential defect area is determined to exist within the cross-section. Furthermore, by analyzing the spatial distribution information in the fused feature data, the approximate location and range of the potential defect area in the rail cross-section coordinate system can be preliminarily determined.

[0111] Based on the spatial range of the potential defect region in a unified coordinate system, the two-dimensional contour of the defect is extracted from optical image data using an adaptive threshold segmentation method to generate optical image feature data; from three-dimensional point cloud data, the difference between the measured depth and the standard model depth is calculated to extract the three-dimensional region representing the morphological anomaly, generating three-dimensional point cloud feature data; from electromagnetic response data, envelope detection of the magnetoacoustic surface wave signal is performed to locate the anomaly region, and the array eddy current signal features at the corresponding location are extracted to generate electromagnetic response feature data; from ultrasonic echo data, wavelet transform denoising and echo peak detection are used to extract the defect reflection waveform within the corresponding time window to generate ultrasonic echo feature data; thus, multimodal feature response data for the same potential defect is obtained.

[0112] In one embodiment, calculating the correlation between characteristic response data of different modalities includes:

[0113] Based on optical image feature data, two-dimensional image features representing the visual morphology of defects are extracted; based on three-dimensional point cloud feature data, three-dimensional geometric features representing the spatial contour of defects are extracted.

[0114] Two-dimensional image features and three-dimensional geometric features are mapped to the same spatial coordinate system. The percentage of overlapping area between the two-dimensional image features and the three-dimensional geometric features is calculated, and the percentage of overlapping area is used as the correlation between optical image feature data and three-dimensional point cloud feature data.

[0115] In a specific embodiment, the closer the calculated overlap area ratio is to 1, the more closely the surface anomalies observed by the optical method match the deformation measured in three dimensions in terms of spatial location.

[0116] In one embodiment, calculating the correlation between characteristic response data of different modalities includes:

[0117] Based on electromagnetic response feature data, the geometric region of electromagnetic signal anomaly is determined; based on optical image feature data, the texture skeleton features of two-dimensional cracks are extracted.

[0118] Calculate the deviation distance between the center position of the geometric region of electromagnetic signal anomaly and the center position of the texture skeleton feature of the two-dimensional crack;

[0119] Based on the deviation distance, the correlation between electromagnetic response feature data and optical image feature data is determined.

[0120] In a specific embodiment, envelope detection and analysis are performed on the array eddy current signal in the electromagnetic response feature data to identify abnormal segments where the signal amplitude exceeds the normal fluctuation range, and the centroid algorithm is used to calculate the geometric center coordinates O1 of the abnormal region. For the optical image feature data, a skeleton extraction algorithm is used to identify the texture center line of the crack, and the midpoint coordinates O2 of the center line are calculated.

[0121] Under a unified cross-sectional coordinate system, the Euclidean distance between the center O1 of the electromagnetic anomaly region and the center O2 of the optical crack texture is calculated to obtain the center position deviation distance D.

[0122] The correlation degree is calculated based on the center position deviation distance D. The correlation degree can be defined as: 1-(D / L), where L is the length (feature size) of the optical crack texture. The smaller the deviation distance D, the closer the correlation degree is to 1.

[0123] In one embodiment, calculating the correlation between characteristic response data of different modalities includes:

[0124] Based on electromagnetic response characteristic data, the defect depth is calculated according to the propagation time of magnetoacoustic surface waves and used as the first defect depth.

[0125] Based on the ultrasonic echo characteristic data, the defect depth is calculated using the pulse reflection method and used as the second defect depth.

[0126] Calculate the depth difference between the first defect depth and the second defect depth;

[0127] Based on the depth difference, the correlation between electromagnetic response characteristic data and ultrasonic echo characteristic data is determined.

[0128] In a specific embodiment, the correlation degree is calculated based on the depth difference ΔH. The correlation degree can be defined as: 1 - (ΔH / Hmax); where Hmax is a preset depth reference value. The smaller the depth difference ΔH, the higher the correlation degree.

[0129] In one embodiment, calculating the correlation between characteristic response data of different modalities includes:

[0130] Based on 3D point cloud feature data, the maximum depth value of the defect region is extracted as the 3D defect depth; based on electromagnetic response feature data, the shallow layer thickness of the defect is calculated according to the impedance change of the array eddy current.

[0131] Calculate the thickness difference between the three-dimensional defect depth and the shallow surface layer thickness;

[0132] Based on the thickness difference, the correlation between 3D point cloud feature data and electromagnetic response feature data is determined.

[0133] In a specific embodiment, the minimum depth value (or maximum depression value) of the point cloud in the defect region is extracted from the 3D point cloud feature data and taken as the 3D defect depth H3. Based on the array eddy current signal in the electromagnetic response feature data, and according to its impedance change, the shallow surface thickness T of the defect is calculated by inversion using a pre-calibrated impedance-thickness relationship curve. The absolute value of the difference between the 3D defect depth H3 and the shallow surface thickness T is calculated to obtain the thickness difference ΔT. The correlation degree is calculated based on the thickness difference ΔT, which can be defined as: 1-(ΔT / H3). The smaller the thickness difference ΔT, the higher the correlation degree.

[0134] In a specific embodiment, multiple modal parameters reflecting the physical characteristics of the same defect can be extracted from the feature response data of different modalities. Based on the physical relationship between the multiple modal parameters, the deviation between the multiple modal parameters is calculated to determine the parameter deviation degree used to evaluate the authenticity of the defect. Based on a preset weight, the correlation between the calculated feature response data of different modalities and the deviation between the multiple modal parameters are weighted and fused to calculate a comprehensive verification result.

[0135] By extracting and comparing the physical parameters of the same defect from different modal data, parameter consistency verification was achieved. This verification result was then fused with feature correlation, thus completing cross-validation of feature correlation and parameter verification dimensions. This cross-validation mechanism upgrades the judgment of potential defects from empirical judgment relying on a single-dimensional threshold to multi-dimensional quantitative decision-making, improving the credibility and accuracy of the final rail inspection results.

[0136] In step 104, the correlation degree is compared with the preset correlation degree threshold. Based on the comparison result, the potential defect area is confirmed as the real defect area, and a rail defect detection report is generated.

[0137] In a specific embodiment, the comprehensive verification result is compared with a preset consistency judgment threshold;

[0138] If the comprehensive verification results are not lower than the consistency judgment threshold, the potential defect area is confirmed as a real defect area; otherwise, it is marked as a suspected defect area to be verified; a rail defect detection report containing defect location, type and size information is generated.

[0139] In step 104, the correlation degree, i.e., the calculated comprehensive consistency verification score (V), is compared with a preset threshold: if V ≥ the threshold, the potential defect area is confirmed as a real defect; otherwise, it is marked as a suspected defect requiring further verification. Finally, the system generates a rail defect detection report containing information such as defect location, type, size, and confidence level (based on the V value).

[0140] This invention also provides a comprehensive rail inspection device, as described in the following embodiments. Since the principle by which this device solves the problem is similar to that of the comprehensive rail inspection method, the implementation of this device can refer to the implementation of the comprehensive rail inspection method; repeated details will not be elaborated further.

[0141] Figure 4 This is a schematic diagram of a comprehensive rail inspection device according to an embodiment of the present invention. The device includes:

[0142] The multimodal data acquisition module 401 is used to acquire multimodal raw data collected by the multimodal sensor; the multimodal raw data includes optical image data, three-dimensional point cloud data, electromagnetic response data and ultrasonic echo data under the same cross section of the rail;

[0143] The feature fusion module 402 is used to perform weighted fusion processing on the multimodal raw data to generate fused feature data;

[0144] The correlation calculation module 403 is used to identify potential defect areas of rails based on fused feature data, extract multimodal feature response data in potential defect areas, and calculate the correlation between feature response data of different modes.

[0145] The rail defect detection report generation module 404 is used to compare the correlation degree with the preset correlation degree threshold, confirm the potential defect area as the real defect area based on the comparison result, and generate a rail defect detection report.

[0146] In one embodiment, the multimodal data acquisition module 401 is further configured to:

[0147] The multimodal raw data is aligned to obtain aligned multimodal raw data; the alignment process includes spatiotemporal synchronization and spatial coordinate unification.

[0148] Among them, spatiotemporal synchronization uses a unified synchronization clock signal and mileage encoding signal to assign a unified timestamp and mileage location label to the multimodal raw data;

[0149] Among them, the spatial coordinate unification means mapping the multimodal raw data to the same coordinate system based on the rail cross section.

[0150] In one embodiment, the feature fusion module 402 is specifically used for:

[0151] Obtain working condition parameters reflecting the current rail condition, and determine the fusion weight corresponding to each modal data based on the working condition parameters and the quality assessment results of the original data of each modality; wherein the quality assessment results are determined based on at least one of the following indicators: signal clarity, feature stability, or noise level of the corresponding modal data.

[0152] By using fusion weights, the defect-related feature data extracted from each modality are weighted and fused to generate fused feature data for characterizing defect information.

[0153] In one embodiment, the correlation calculation module 403 is specifically used for:

[0154] Based on the defect information represented by the fused feature data, the potential defect area of ​​the rail is located.

[0155] From the multimodal raw data, optical image feature data, three-dimensional point cloud feature data, electromagnetic response feature data, and ultrasonic echo feature data corresponding to potential defect areas are extracted as multimodal feature response data.

[0156] In one embodiment, the correlation calculation module 403 is specifically used for:

[0157] Based on optical image feature data, two-dimensional image features representing the visual morphology of defects are extracted; based on three-dimensional point cloud feature data, three-dimensional geometric features representing the spatial contour of defects are extracted.

[0158] Two-dimensional image features and three-dimensional geometric features are mapped to the same spatial coordinate system. The percentage of overlapping area between the two-dimensional image features and the three-dimensional geometric features is calculated, and the percentage of overlapping area is used as the correlation between optical image feature data and three-dimensional point cloud feature data.

[0159] In one embodiment, the correlation calculation module 403 is specifically used for:

[0160] Based on electromagnetic response feature data, the geometric region of electromagnetic signal anomaly is determined; based on optical image feature data, the texture skeleton features of two-dimensional cracks are extracted.

[0161] Calculate the deviation distance between the center position of the geometric region of electromagnetic signal anomaly and the center position of the texture skeleton feature of the two-dimensional crack;

[0162] Based on the deviation distance, the correlation between electromagnetic response feature data and optical image feature data is determined.

[0163] In one embodiment, the correlation calculation module 403 is specifically used for:

[0164] Based on electromagnetic response characteristic data, the defect depth is calculated according to the propagation time of magnetoacoustic surface waves and used as the first defect depth.

[0165] Based on the ultrasonic echo characteristic data, the defect depth is calculated using the pulse reflection method and used as the second defect depth.

[0166] Calculate the depth difference between the first defect depth and the second defect depth;

[0167] Based on the depth difference, the correlation between electromagnetic response characteristic data and ultrasonic echo characteristic data is determined.

[0168] In one embodiment, the correlation calculation module 403 is specifically used for:

[0169] Based on 3D point cloud feature data, the maximum depth value of the defect region is extracted as the 3D defect depth; based on electromagnetic response feature data, the shallow layer thickness of the defect is calculated according to the impedance change of the array eddy current.

[0170] Calculate the thickness difference between the three-dimensional defect depth and the shallow surface layer thickness;

[0171] Based on the thickness difference, the correlation between 3D point cloud feature data and electromagnetic response feature data is determined.

[0172] This invention also provides a computer device. Figure 5 This is a schematic diagram of a computer device in an embodiment of the present invention. The computer device 500 includes a memory 510, a processor 520, and a computer program 530 stored in the memory 510 and executable on the processor 520. When the processor 520 executes the computer program 530, it implements the above-mentioned comprehensive rail inspection method.

[0173] This invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described comprehensive rail inspection method.

[0174] This invention also provides a computer program product, which includes a computer program that, when executed by a processor, implements the above-described comprehensive rail inspection method.

[0175] In this embodiment of the invention, multimodal raw data collected by a multimodal sensor is acquired. This multimodal raw data includes optical image data, 3D point cloud data, electromagnetic response data, and ultrasonic echo data from the same cross-section of the rail. The multimodal raw data is weighted and fused to generate fused feature data. Based on the fused feature data, potential defect areas of the rail are identified, and multimodal feature response data from these potential defect areas is extracted. The correlation between the feature response data of different modes is calculated. The correlation is compared with a preset correlation threshold, and based on the comparison result, the potential defect area is confirmed as a real defect area, generating a rail defect detection report. In this process, this embodiment of the invention acquires multimodal raw data from the same cross-section of the rail and performs weighted fusion processing to generate fused feature data. After identifying potential defect areas based on the fused feature data, the feature response data of these areas under multimodal conditions is further extracted. By calculating the correlation between the feature response data of different modes and comparing it with a preset correlation threshold, the real defect area is effectively identified, and a detection report is generated. This reduces the false alarm rate under complex working conditions, thereby improving the overall accuracy of rail defect detection.

[0176] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0177] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0178] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0179] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0180] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above descriptions are merely specific embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A comprehensive inspection method for rails, characterized in that, include: Acquire multimodal raw data collected by multimodal sensors; the multimodal raw data includes optical image data, 3D point cloud data, electromagnetic response data and ultrasonic echo data under the same cross section of the rail; The original multimodal data is weighted and fused to generate fused feature data; Based on fused feature data, potential defect areas of rails are identified, multimodal feature response data are extracted from potential defect areas, and the correlation between feature response data of different modes is calculated. The correlation degree is compared with the preset correlation degree threshold. Based on the comparison result, the potential defect area is confirmed as the real defect area, and a rail defect detection report is generated.

2. The method as described in claim 1, characterized in that, Acquiring raw multimodal data from multimodal sensors also includes: The multimodal raw data is aligned to obtain aligned multimodal raw data; the alignment process includes spatiotemporal synchronization and spatial coordinate unification. Among them, spatiotemporal synchronization uses a unified synchronization clock signal and mileage encoding signal to assign a unified timestamp and mileage location label to the multimodal raw data; Among them, the spatial coordinate unification means mapping the multimodal raw data to the same coordinate system based on the rail cross section.

3. The method as described in claim 1, characterized in that, The multimodal raw data is weighted and fused to generate fused feature data, including: Obtain working condition parameters reflecting the current rail condition, and determine the fusion weight corresponding to each modal data based on the working condition parameters and the quality assessment results of the original data of each modality; wherein the quality assessment results are determined based on at least one of the following indicators: signal clarity, feature stability, or noise level of the corresponding modal data. By using fusion weights, the defect-related feature data extracted from each modality are weighted and fused to generate fused feature data for characterizing defect information.

4. The method as described in claim 3, characterized in that, Based on fused feature data, potential defect areas of the rail are identified, and multimodal feature response data of the potential defect areas are extracted, including: Based on the defect information represented by the fused feature data, the potential defect area of ​​the rail is located. From the multimodal raw data, optical image feature data, three-dimensional point cloud feature data, electromagnetic response feature data, and ultrasonic echo feature data corresponding to potential defect areas are extracted as multimodal feature response data.

5. The method as described in claim 4, characterized in that, Calculate the correlation between characteristic response data of different modalities, including: Based on optical image feature data, two-dimensional image features representing the visual morphology of defects are extracted; based on three-dimensional point cloud feature data, three-dimensional geometric features representing the spatial contour of defects are extracted. Two-dimensional image features and three-dimensional geometric features are mapped to the same spatial coordinate system. The percentage of overlapping area between the two-dimensional image features and the three-dimensional geometric features is calculated, and the percentage of overlapping area is used as the correlation between optical image feature data and three-dimensional point cloud feature data.

6. The method as described in claim 4, characterized in that, Calculate the correlation between characteristic response data of different modalities, including: Based on electromagnetic response feature data, the geometric region of electromagnetic signal anomaly is determined; based on optical image feature data, the texture skeleton features of two-dimensional cracks are extracted. Calculate the deviation distance between the center position of the geometric region of electromagnetic signal anomaly and the center position of the texture skeleton feature of the two-dimensional crack; Based on the deviation distance, the correlation between electromagnetic response feature data and optical image feature data is determined.

7. The method as described in claim 4, characterized in that, Calculate the correlation between characteristic response data of different modalities, including: Based on electromagnetic response characteristic data, the defect depth is calculated according to the propagation time of magnetoacoustic surface waves and used as the first defect depth. Based on the ultrasonic echo characteristic data, the defect depth is calculated using the pulse reflection method and used as the second defect depth. Calculate the depth difference between the first defect depth and the second defect depth; Based on the depth difference, the correlation between electromagnetic response characteristic data and ultrasonic echo characteristic data is determined.

8. The method as described in claim 4, characterized in that, Calculate the correlation between characteristic response data of different modalities, including: Based on 3D point cloud feature data, the maximum depth value of the defect region is extracted as the 3D defect depth; based on electromagnetic response feature data, the shallow layer thickness of the defect is calculated according to the impedance change of the array eddy current. Calculate the thickness difference between the three-dimensional defect depth and the shallow surface layer thickness; Based on the thickness difference, the correlation between 3D point cloud feature data and electromagnetic response feature data is determined.

9. A comprehensive rail inspection device, characterized in that, include: The multimodal data acquisition module is used to acquire multimodal raw data collected by multimodal sensors; the multimodal raw data includes optical image data, three-dimensional point cloud data, electromagnetic response data and ultrasonic echo data under the same cross section of the rail; The feature fusion module is used to perform weighted fusion processing on the original multimodal data to generate fused feature data; The correlation calculation module is used to identify potential defect areas of rails based on fused feature data, extract multimodal feature response data from potential defect areas, and calculate the correlation between feature response data of different modes. The rail defect detection report generation module is used to compare the correlation degree with a preset correlation degree threshold, confirm the potential defect area as the real defect area based on the comparison result, and generate a rail defect detection report.

10. The apparatus as claimed in claim 9, characterized in that, The multimodal data acquisition module is also used for: The multimodal raw data is aligned to obtain aligned multimodal raw data; the alignment process includes spatiotemporal synchronization and spatial coordinate unification. Among them, spatiotemporal synchronization uses a unified synchronization clock signal and mileage encoding signal to assign a unified timestamp and mileage location label to the multimodal raw data; Among them, the spatial coordinate unification means mapping the multimodal raw data to the same coordinate system based on the rail cross section.

11. The apparatus as claimed in claim 9, characterized in that, The feature fusion module is specifically used for: Obtain working condition parameters that reflect the current rail condition, and determine the fusion weights corresponding to each modal data based on the working condition parameters and the quality assessment results of the original data of each modality. The quality assessment results are determined based on at least one of the following indicators: signal clarity, feature stability, or noise level of the corresponding modal data. By using fusion weights, the defect-related feature data extracted from each modality are weighted and fused to generate fused feature data for characterizing defect information.

12. The apparatus as claimed in claim 11, characterized in that, The correlation calculation module is specifically used for: Based on the defect information represented by the fused feature data, the potential defect area of ​​the rail is located. From the multimodal raw data, optical image feature data, three-dimensional point cloud feature data, electromagnetic response feature data, and ultrasonic echo feature data corresponding to potential defect areas are extracted as multimodal feature response data.

13. The apparatus as claimed in claim 12, characterized in that, The correlation calculation module is specifically used for: Based on optical image feature data, two-dimensional image features representing the visual morphology of defects are extracted; based on three-dimensional point cloud feature data, three-dimensional geometric features representing the spatial contour of defects are extracted. Two-dimensional image features and three-dimensional geometric features are mapped to the same spatial coordinate system. The percentage of overlapping area between the two-dimensional image features and the three-dimensional geometric features is calculated, and the percentage of overlapping area is used as the correlation between optical image feature data and three-dimensional point cloud feature data.

14. The apparatus as claimed in claim 12, characterized in that, The correlation calculation module is specifically used for: Based on electromagnetic response feature data, the geometric region of electromagnetic signal anomaly is determined; based on optical image feature data, the texture skeleton features of two-dimensional cracks are extracted. Calculate the deviation distance between the center position of the geometric region of electromagnetic signal anomaly and the center position of the texture skeleton feature of the two-dimensional crack; Based on the deviation distance, the correlation between electromagnetic response feature data and optical image feature data is determined.

15. The apparatus as claimed in claim 12, characterized in that, The correlation calculation module is specifically used for: Based on electromagnetic response characteristic data, the defect depth is calculated according to the propagation time of magnetoacoustic surface waves and used as the first defect depth. Based on the ultrasonic echo characteristic data, the defect depth is calculated using the pulse reflection method and used as the second defect depth. Calculate the depth difference between the first defect depth and the second defect depth; Based on the depth difference, the correlation between electromagnetic response characteristic data and ultrasonic echo characteristic data is determined.

16. The apparatus as claimed in claim 12, characterized in that, The correlation calculation module is specifically used for: Based on 3D point cloud feature data, the maximum depth value of the defect region is extracted as the 3D defect depth; based on electromagnetic response feature data, the shallow layer thickness of the defect is calculated according to the impedance change of the array eddy current. Calculate the thickness difference between the three-dimensional defect depth and the shallow surface layer thickness; Based on the thickness difference, the correlation between 3D point cloud feature data and electromagnetic response feature data is determined.

17. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method of any one of claims 1 to 8.

18. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the method of any one of claims 1 to 8.

19. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the method of any one of claims 1 to 8.

Images