Multi-source data fusion railway line early warning method

Abstract

The invention provides a railway line early warning method based on multi-source data fusion, and the method comprises the following steps: A, inputting a to-be-inspected railway line and a ramp map into an unmanned plane management platform, and making an inspection route; setting a flight task including time and an inspection route sequence; b, the unmanned aerial vehicle management platform controls the unmanned aerial vehicle to fly and inspect, and image acquisition is carried out through a high-definition camera on the unmanned aerial vehicle during inspection; c, the unmanned aerial vehicle management platform performs image algorithm analysis on the acquired image, determines the real-time position of inspection, marks a target when finding the early warning target, and automatically generates early warning information; d, polling the unloaded train at the stop platform, and checking whether remnants which are missed to be loaded and unloaded exist or not; and E, the control terminal pops up an early warning prompt based on each piece of early warning information. According to the invention, whether obstacles, personnel and animal invasion exist on railway lines, whether foreign matters are left on empty vehicles in station tracks, and whether landslides exist on side slopes can be inspected in a timed manner by the unmanned aerial vehicle.

Description

Technical Field

[0001] This invention relates to the field of intelligent detection technology for unmanned aerial vehicles (UAVs), specifically to a method for early warning of railway lines using multi-source data fusion. Background Technology

[0002] Currently, railway line inspections primarily rely on manual labor. These manual inspections are typically carried out by experienced workers who visually inspect the railway lines and their ancillary facilities for potential hazards and disasters. This method is relatively inefficient and cannot meet the growing demand for railway safety inspections. The difficulty and effectiveness of inspections are significantly impacted, especially at night or in inclement weather. Furthermore, their automation level is low, requiring a substantial investment of human resources for these operations.

[0003] Given the above issues and the current shortage of human resources in various railway positions, the need for using drones for railway line inspection has become increasingly urgent. Drone inspection, with its advantages of efficiency, flexibility, and comprehensiveness, represents a major innovation in the field of railway inspection, and is expected to safeguard railway safety and promote the development of railway inspection work towards intelligence and automation. Therefore, it is necessary to conduct further research on drone inspection technology. Summary of the Invention

[0004] This invention aims to provide a railway line early warning method based on multi-source data fusion. This method integrates multiple technologies such as UAV technology, wireless communication technology, image intelligent analysis technology, and database technology to enable UAV-based timed inspections of railway lines for obstacles, personnel, or animal encroachment; whether empty cars in station tracks have foreign objects left behind; and whether slopes are prone to landslides.

[0005] The technical solution of the present invention is as follows:

[0006] The aforementioned multi-source data fusion-based railway line early warning method includes the following steps:

[0007] A. Input the railway line and slope map to be inspected into the drone management platform, formulate the inspection route; set the flight mission, including time and inspection route sequence;

[0008] B. The UAV management platform controls the UAV to conduct flight inspections according to the flight mission. During the inspection, images are collected by the high-definition camera on the UAV and transmitted back to the UAV management platform.

[0009] C. The drone management platform will perform image algorithm analysis on the acquired images to determine the real-time location of the inspection. When a warning target is detected, the target will be marked and a warning message will be automatically generated and transmitted to the control terminal. The warning targets are people, animals, and obstacles appearing in the railway line control area; whether the railway slope is damaged, whether there is a landslide, and whether there is vegetation growth that is about to invade the railway line control area.

[0010] D. Inspect the unloading train cars parked at the station platform, identify the car number of each car, check each car to see if there are any leftover items that have not been loaded or unloaded. If any leftover items are found, identify their type, and then transmit the car number and the type of leftover items to the control terminal.

[0011] E. The control terminal displays warning alerts based on various warning information, allowing relevant personnel to view and process the warning information in a timely manner.

[0012] In step C, the method for determining the real-time location of the inspection includes the following steps:

[0013] a. Extraction of orbital reference information through real-time edge processing, which includes the following steps:

[0014] Illumination-adaptive preprocessing: First, the video image is enhanced by the Retinex image enhancement algorithm, then the CLAHE contrast-limited histogram equalization is performed, and finally, guided filtering is used to keep the edges smooth and handle track reflections and shadows to obtain the illumination-adaptive preprocessing result.

[0015] Then, the image is segmented through an image semantic segmentation network to obtain a segmentation mask binary image. After the segmentation mask binary image is fitted by multiple constraints, a fitted image is obtained. The orbital reference information is calculated based on the fitted image.

[0016] b. Mileage statistics based on orbital reference information:

[0017] The real-time preliminary track mileage is obtained by acquiring GNSS signals, then the images acquired by the UAV are decoded by the OCR network to identify the on-site mileage signs. At the same time, based on SLAM fusion processing using standard sleeper spacing, the loss of GNSS signals is compensated by visual inertial odometry (VIO) to obtain the corrected real-time track mileage.

[0018] For tracks with large curvature, the UAV uses segmented shooting, and the segmented track images are stitched together. The stitching algorithm is as follows: phase correlation matching based on sleeper features, then three-dimensional consistency constraints, and finally Gaussian process regression smoothing to obtain track images with large curvature.

[0019] The specific algorithm for stitching together segmented track images is as follows:

[0020] Frames are extracted from the segmented video, distortion removal is performed, and the distortion-removed frame images are output. Then, sleepers are located by using edge detection and morphological operations. The displacement of sleepers between adjacent frames is calculated, and then Gaussian regression is used for smoothing to obtain a smoothed image.

[0021] Then, the Frenet frame is calculated on the 3D point set of the smoothed image to obtain the curvature; then, points with κ>κthreshold are selected, which are the curve area points. The curve area points and the original video texture are projected onto the 3D model, and an orthogonal view is generated along the track normal plane to obtain the stitched track image.

[0022] The formula for calculating the displacement of the sleeper between adjacent frames is as follows: Let the two image frames be I1(x,y) and I2(x,y), and their Fourier transforms be F1 and F2:

[0023] Calculate the cross power spectrum:

[0024]

[0025] The impulse response is obtained by inverse Fourier transform:

[0026]

[0027] Displacement peak detection:

[0028]

[0029] The Gaussian process regression smoothing process is as follows:

[0030] Define a Gaussian process y ~ GP(m(x), k(x, x′)), where the mean function m(x) = 0;

[0031] The covariance function uses the Matern 5 / 2 kernel formula as follows:

[0032]

[0033] Where d = ∥xi - xj∥, l is the length scale, and σf is the signal variance.

[0034] Then predict the value of the new position x*, and the predicted distribution is a Gaussian distribution as follows:

[0035] p(y*|X,y,x*)=N(μ*,σ*2) (5)

[0036] in:

[0037]

[0038] Where K is the covariance matrix of the training points (Kij=k(xi,xj)), k * Let be the covariance vector between the test point and the training point.

[0039] Orbit reference information parameters include:

[0040]

[0041] In step D, the method for identifying the carriage number of each carriage is as follows:

[0042] The U-Net network is used to segment the carriage outline, and then the flight direction of the UAV is determined by combining the inspection route. Based on the baseline carriage number, the detected features at the carriage connection are counted. Combining the flight direction, the carriage connection feature count, and the current vehicle system grouping information, the carriage number is calculated.

[0043] The specific process for counting the detected features at the connection points between the carriages is as follows:

[0044] Using a topology sensing algorithm, the train intervals are identified through the coupler and buffer device at the train connection, and the train formation structure is updated in real time.

[0045] The topology sensing algorithm process is as follows:

[0046] a. The spatial position of the positioning hook and buffer device is calculated based on the YOLOv7 network, and the loss function is as follows:

[0047] Ldet=λ box L CIoU +λ cls L cls +λ obj L obj

[0048] Where L CIoU To improve the crossover ratio loss, L cls For classification loss, L obj Target confidence loss, λ box , λ cls , λ obj These are the loss weighting coefficients;

[0049] b. Calculate the carriage spacing d i The calculation formula is as follows:

[0050]

[0051] Among them, (x i ,y i ),(x i+1 ,y i+1) represents the coordinates of the center point of the adjacent hook and buffer device; η is the calibration parameter (pixels / meter), which is calibrated in real time using reference objects such as sleeper spacing;

[0052] c. Construct a train carriage grouping topology diagram. The topology diagram construction includes the following:

[0053] Node information: v per carriage i The attributes include carriage type and length L. i Edge information: Connection relationship e(v) i vi +1 The weight is the distance between carriages d. i ;

[0054] Among them, the real-time determination of the distance d between carriages i Connections are defined as those less than the threshold, and the distance between carriages is d. i The abnormal expansion triggered a splitting alarm.

[0055] d. Smooth the topology graph based on time-series filtering to suppress transient noise;

[0056] The process is as follows:

[0057] Define a state vector st = [d1(t), d2(t), ..., dn(t)], and apply the following formula for exponentially weighted moving average to suppress instantaneous noise:

[0058] s^t=αst+(1-α)s^t-1,α∈[0.2,0.5] (8) e. After smoothing, the topology graph is output, along with the grouped topology information, which is as follows:

[0059] Topological descriptor: adjacency matrix

[0060] Grouping sequence:

[0061] In step D, the process of detecting and determining the type of items left in the carriage is as follows:

[0062] a. Image preprocessing: Extract a standardized top view of the carriage through perspective correction and illumination equalization;

[0063] b. Abnormal target detection: The image is identified using the YOLOv7 network, and the bounding boxes of the remaining objects are output.

[0064] c. Position mapping: By converting image coordinates to the car coordinate system, the relative position of the abandoned object inside the car is output.

[0065] d. Based on the carriage segmentation results, output: Group X - Section N - Remains Type, and store it in the database.

[0066] The process of converting image coordinates to carriage coordinates is as follows:

[0067] Constructing the camera calibration matrix

[0068]

[0069] Define the car coordinate system:

[0070] Origin point O: Center of the carriage floor

[0071] X: Length direction of the carriage, from front to rear;

[0072] Y: In the direction of carriage width, from left to right;

[0073] Z: Vertical direction, from floor to roof;

[0074] Inverse transformation of perspective projection

[0075] Input: Image point Pimg = (u, v); corresponding to the floor height zfloor = 0, assuming the object is on the ground;

[0076] Then, distortion is removed, the ray direction vector is calculated, the floor intersection point is determined, and the coordinates are transformed to the car coordinate system. The car coordinate formula is as follows:

[0077] P car =R -1 (λv-t) (10).

[0078] The derivation of formula (10) is as follows:

[0079] Let the equation of the carriage floor be:

[0080] n T P+d=0 (n=[0,0,1]) T (d=0) (11)

[0082] Point P in the camera coordinate system c satisfy:

[0083]

[0084] Substitute into the plane equation:

[0085]

[0086] The formula for the car body coordinates is obtained, namely formula (10).

[0087] The control terminal is a PC; the drone management platform is the DJI Sikong 2 drone management platform.

[0088] The present invention has the following beneficial effects:

[0089] The method of this invention transmits the collected video image information to an intelligent analysis server through network communication technology, enabling intelligent detection and real-time early warning of track and vehicle status. Simultaneously, it accurately locates problematic vehicles through multi-source data fusion, providing valuable data support for troubleshooting and offering railway-related departments an efficient inspection technique with promising application prospects. Detailed Implementation

[0090] The present invention will now be described in detail with reference to specific embodiments.

[0091] Example 1

[0092] The railway line early warning method based on multi-source data fusion includes the following steps:

[0093] A. Input the railway line and slope map to be inspected into the drone management platform, formulate the inspection route; set the flight mission, including time and inspection route sequence;

[0094] B. The UAV management platform controls the UAV to conduct flight inspections according to the flight mission. During the inspection, images are collected by the high-definition camera on the UAV and transmitted back to the UAV management platform.

[0095] C. The drone management platform will perform image algorithm analysis on the acquired images to determine the real-time location of the inspection. When a warning target is detected, the target will be marked and a warning message will be automatically generated and transmitted to the control terminal. The warning targets are people, animals, and obstacles appearing in the railway line control area; whether the railway slope is damaged, whether there is a landslide, and whether there is vegetation growth that is about to invade the railway line control area.

[0096] The method for determining the real-time location of the inspection includes the following steps:

[0097] a. Extraction of orbital reference information through real-time edge processing, which includes the following steps:

[0098] Illumination-adaptive preprocessing: First, the video image is enhanced by the Retinex image enhancement algorithm, then the CLAHE contrast-limited histogram equalization is performed, and finally, guided filtering is used to keep the edges smooth and handle track reflections and shadows to obtain the illumination-adaptive preprocessing result.

[0099] Then, the image is segmented through an image semantic segmentation network to obtain a segmentation mask binary image. After the segmentation mask binary image is fitted by multiple constraints, a fitted image is obtained. The orbital reference information is calculated based on the fitted image.

[0100] b. Mileage statistics based on orbital reference information:

[0101] The real-time preliminary track mileage is obtained by acquiring GNSS signals, then the images acquired by the UAV are decoded by the OCR network to identify the on-site mileage signs. At the same time, based on SLAM fusion processing using standard sleeper spacing, the loss of GNSS signals is compensated by visual inertial odometry (VIO) to obtain the corrected real-time track mileage.

[0102] For tracks with large curvature, the UAV uses segmented shooting, and the segmented track images are stitched together. The stitching algorithm is as follows: phase correlation matching based on sleeper features, then three-dimensional consistency constraints, and finally Gaussian process regression smoothing to obtain track images with large curvature.

[0103] The specific algorithm for stitching together segmented track images is as follows:

[0104] Frames are extracted from the segmented video, distortion removal is performed, and the distortion-removed frame images are output. Then, sleepers are located by using edge detection and morphological operations. The displacement of sleepers between adjacent frames is calculated, and then Gaussian regression is used for smoothing to obtain a smoothed image.

[0105] Then, the Frenet frame is calculated on the 3D point set of the smoothed image to obtain the curvature; then, points with κ>κthreshold are selected, which are the curve area points. The curve area points and the original video texture are projected onto the 3D model, and an orthogonal view is generated along the track normal plane to obtain the stitched track image.

[0106] The formula for calculating the displacement of the sleeper between adjacent frames is as follows: Let the two image frames be I1(x,y) and I2(x,y), and their Fourier transforms be F1 and F2:

[0107] Calculate the cross power spectrum:

[0108]

[0109] The impulse response is obtained by inverse Fourier transform:

[0110]

[0111] Displacement peak detection:

[0112]

[0113] The Gaussian process regression smoothing process is as follows:

[0114] Define a Gaussian process y ~ GP(m(x), k(x, x′)), where the mean function m(x) = 0;

[0115] The covariance function uses the Matern 5 / 2 kernel formula as follows:

[0116]

[0117] Where d = ∥xi - xj∥, l is the length scale, and σf is the signal variance.

[0118] Then predict the value of the new position x*, and the predicted distribution is a Gaussian distribution as follows:

[0119] p(y*|X,y,x*)=N(μ*,σ*2) (5)

[0120] in:

[0121]

[0122] Where K is the covariance matrix of the training points (Kij=k(xi,xj)), k * This is the covariance vector between the test points and the training points. The orbital baseline information parameters include:

[0123]

[0124] D. Inspect the unloading train cars parked at the station platform, identify the car number of each car, check each car to see if there are any leftover items that have not been loaded or unloaded. If any leftover items are found, identify their type, and then transmit the car number and the type of leftover items to the control terminal.

[0125] The method for identifying the carriage numbers of each train car is as follows:

[0126] The U-Net network is used to segment the carriage outline, and then the flight direction of the UAV is determined by combining the inspection route. Based on the baseline carriage number, the detected features at the carriage connection are counted. Combining the flight direction, the carriage connection feature count, and the current vehicle system grouping information, the carriage number is calculated.

[0127] The specific process for counting the detected features at the connection points between the carriages is as follows:

[0128] Using a topology sensing algorithm, the train intervals are identified through the coupler and buffer device at the train connection, and the train formation structure is updated in real time.

[0129] The topology sensing algorithm process is as follows:

[0130] a. The spatial position of the positioning hook and buffer device is calculated based on the YOLOv7 network, and the loss function is as follows:

[0131] Ldet=λ box L CIoU +λ cls L cls +λ obj L obj

[0132] Where L CIoU To improve the crossover ratio loss, L cls For classification loss, L obj Target confidence loss, λ box , λ cls , λ obj These are the loss weighting coefficients;

[0133] b. Calculate the carriage spacing d i The calculation formula is as follows:

[0134]

[0135] Among them, (x i ,y i ),(x i+1 ,y i+1 ) represents the coordinates of the center point of the adjacent hook and buffer device; η is the calibration parameter (pixels / meter), which is calibrated in real time using reference objects such as sleeper spacing;

[0136] c. Construct a train carriage grouping topology diagram. The topology diagram construction includes the following:

[0137] Node information: v per carriage i The attributes include carriage type and length L. i Edge information: Connection relationship e(v) i vi +1 The weight is the distance between carriages d. i ;

[0138] Among them, the real-time determination of the distance d between carriages i Connections are defined as those less than the threshold, and the distance between carriages is d. i The abnormal expansion triggered a splitting alarm.

[0139] d. Smooth the topology graph based on time-series filtering to suppress transient noise;

[0140] The process is as follows:

[0141] Define a state vector st = [d1(t), d2(t), ..., dn(t)], and apply the following formula for exponentially weighted moving average to suppress instantaneous noise:

[0142] s^t=αst+(1-α)s^t-1,α∈[0.2,0.5] (8) e. After smoothing, the topology graph is output, along with the grouped topology information, which is as follows:

[0143] Topological descriptor: adjacency matrix

[0144] Grouping sequence:

[0145] The process for detecting and determining the type of items left in the train carriage is as follows:

[0146] a. Image preprocessing: Extract a standardized top view of the carriage through perspective correction and illumination equalization;

[0147] b. Abnormal target detection: The image is identified using the YOLOv7 network, and the bounding boxes of the remaining objects are output.

[0148] c. Position mapping: By converting image coordinates to the car coordinate system, the relative position of the abandoned object inside the car is output.

[0149] d. Based on the carriage segmentation results, output: Group X - Section N - Remains Type, and store it in the database.

[0150] The process of converting image coordinates to carriage coordinates is as follows:

[0151] Constructing the camera calibration matrix

[0152]

[0153] Define the car coordinate system:

[0154] Origin point O: Center of the carriage floor

[0155] X: Length direction of the carriage, from front to rear;

[0156] Y: In the direction of carriage width, from left to right;

[0157] Z: Vertical direction, from floor to roof;

[0158] Inverse transformation of perspective projection

[0159] Input: Image point Pimg = (u, v); corresponding to the floor height zfloor = 0, assuming the object is on the ground;

[0160] Then, distortion is removed, the ray direction vector is calculated, the floor intersection point is determined, and the coordinates are transformed to the car coordinate system. The car coordinate formula is as follows:

[0161] P car =R -1 (λv-t) (10).

[0162] The derivation of formula (10) is as follows:

[0163] Let the equation of the carriage floor be:

[0164] n T P+d=0 (n=[0,0,1]) T (d=0) (11)

[0166] Point P in the camera coordinate system c satisfy:

[0167]

[0168] Substitute into the plane equation:

[0169]

[0170] The formula for the car body coordinates is obtained, namely formula (10).

[0171] E. The control terminal displays warning alerts based on various warning information, allowing relevant personnel to view and process the warning information in a timely manner.

[0172] The control terminal is a PC; the drone management platform is the DJI Sikong 2 drone management platform.

Claims

1. A railway line early warning method based on multi-source data fusion, characterized in that, Includes the following steps: A. Input the map of the railway line and slope to be inspected into the drone management platform and formulate the inspection route; Set the flight mission, including the time and the order of the inspection route; B. The UAV management platform controls the UAV to conduct flight inspections according to the flight mission. During the inspection, images are collected by the high-definition camera on the UAV and transmitted back to the UAV management platform. C. The drone management platform will perform image algorithm analysis on the acquired images to determine the real-time location of the inspection. When a warning target is detected, the target will be marked and a warning message will be automatically generated and transmitted to the control terminal. The warning targets are people, animals, and obstacles appearing in the railway line control area; whether the railway slope is damaged, whether there is a landslide, and whether there is vegetation growth that is about to invade the railway line control area. D. Inspect the unloading train cars parked at the station platform, identify the car number of each car, check each car to see if there are any leftover items that have not been loaded or unloaded. If any leftover items are found, identify their type, and then transmit the car number and the type of leftover items to the control terminal. E. The control terminal displays warning alerts based on various warning information, allowing relevant personnel to view and process the warning information in a timely manner.

2. The railway line early warning method based on multi-source data fusion as described in claim 1, characterized in that: In step C, the method for determining the real-time location of the inspection includes the following steps: a. Extraction of orbital reference information through real-time edge processing, which includes the following steps: Illumination-adaptive preprocessing: First, the video image is enhanced by the Retinex image enhancement algorithm, then the CLAHE contrast-limited histogram equalization is performed, and finally, guided filtering is used to keep the edges smooth and handle track reflections and shadows to obtain the illumination-adaptive preprocessing result. Then, the image is segmented through an image semantic segmentation network to obtain a segmentation mask binary image. After the segmentation mask binary image is fitted by multiple constraints, a fitted image is obtained. The orbital reference information is calculated based on the fitted image. b. Mileage statistics based on orbital reference information: The real-time preliminary track mileage is obtained by acquiring GNSS signals, then the images acquired by the UAV are decoded by the OCR network to identify the on-site mileage signs. At the same time, based on SLAM fusion processing using standard sleeper spacing, the loss of GNSS signals is compensated by visual inertial odometry (VIO) to obtain the corrected real-time track mileage. For tracks with large curvature, the UAV uses segmented shooting, and the segmented track images are stitched together. The stitching algorithm is as follows: phase correlation matching based on sleeper features, then three-dimensional consistency constraints, and finally Gaussian process regression smoothing to obtain track images with large curvature.

3. The railway line early warning method based on multi-source data fusion as described in claim 2, characterized in that: The specific algorithm for stitching together segmented track images is as follows: Frames are extracted from the segmented video, distortion removal is performed, and the distortion-removed frame images are output. Then, sleepers are located by using edge detection and morphological operations. The displacement of sleepers between adjacent frames is calculated, and then Gaussian regression is used for smoothing to obtain a smoothed image. Then, the Frenet frame is calculated on the 3D point set of the smoothed image to obtain the curvature; then, points with κ>κthreshold are selected, which are the curve area points. The curve area points and the original video texture are projected onto the 3D model, and an orthogonal view is generated along the track normal plane to obtain the stitched track image. The formula for calculating the displacement of the sleeper between adjacent frames is as follows: Let the two image frames be I1(x,y) and I2(x,y), and their Fourier transforms be F1 and F2: Calculate the cross power spectrum: （1） The impulse response is obtained by inverse Fourier transform: （2） Displacement peak detection: （3） The Gaussian process regression smoothing process is as follows: Define a Gaussian process y∼GP(m(x),k(x,x′)), where the mean function m(x)=0; The covariance function is calculated using the Matern 5 / 2 kernel formula as follows: （4） Where d = ∥xi−xj∥, ℓ is the length scale, and σf is the signal variance; Then, the value of the new position x* is predicted, and the predicted distribution is a Gaussian distribution as follows: p ( y ∗∣X,y,x∗)=N( μ *, σ *2) (5) in: （6） Where K is the covariance matrix of the training points (Kij=k(xi,xj)), k ∗ Let be the covariance vector between the test points and the training points.

4. The railway line early warning method based on multi-source data fusion as described in claim 2, characterized in that... ： Orbit reference information parameters include: 。 5. The railway line early warning method based on multi-source data fusion as described in claim 1, characterized in that: In step D, the method for identifying the carriage number of each carriage is as follows: The U-Net network is used to segment the carriage outline, and then the flight direction of the UAV is determined by combining the inspection route. Based on the baseline carriage number, the detected features at the carriage connection are counted. Combining the flight direction, the carriage connection feature count, and the current vehicle system grouping information, the carriage number is calculated.

6. The railway line early warning method based on multi-source data fusion as described in claim 5, characterized in that: The specific process for counting the detected features at the connection points between the carriages is as follows: Using a topology sensing algorithm, the train intervals are identified through the coupler and buffer device at the train connection, and the train formation structure is updated in real time. The topology sensing algorithm process is as follows: a. The spatial position of the positioning hook and buffer device is calculated based on the YOLOv7 network, and the loss function is as follows: Ldet=λ box L CIoU +λ cls L cls +λ obj L obj Where L CIoU To improve the crossover ratio loss, L cls For classification loss, L obj Target confidence loss, λ box , λ cls , λ obj These are the loss weighting coefficients; b. Calculate the carriage spacing d i The calculation formula is as follows: （7） Among them, (x i , y i ), (x i+1 , y i+1 () represents the coordinates of the center points of adjacent hook and buffer devices; To calibrate the parameters (pixels / meter), real-time calibration is performed using reference objects such as sleeper spacing. c. Construct a train carriage grouping topology diagram. The topology diagram construction includes the following: Node information: v per carriage i The attributes include carriage type and length L. i Edge information: Connection relationship e(v) i vi +1 The weight is the distance between carriages d. i ; Among them, the real-time determination of the distance d between carriages i Connections are defined as less than a threshold, and the distance between carriages is d. i The abnormal expansion triggered a splitting alarm. d. Smooth the topology graph based on time-series filtering to suppress transient noise; The process is as follows: Define state vector s t =[ d 1( t ), d 2( t ),…, dn ( t The following formula is used to apply an exponentially weighted moving average to suppress instantaneous noise: s^t=as t +(1−α)s^ t −1,α∈[0.2,0.5] (8) e. After smoothing, the topology graph is output, along with the grouped topology information, which is as follows: Topological descriptor: adjacency matrix A = a 11 a 21⋮ an 1 a 12 a 22⋮ an 2⋯⋯⋱⋯ a 1 na 2 n ⋮ ann , aij ∈{0,1}; Grouping sequence: .

7. The railway line early warning method based on multi-source data fusion as described in claim 5, characterized in that: In step D, the process of detecting and determining the type of items left in the carriage is as follows: a. Image preprocessing: Extract a standardized top view of the carriage through perspective correction and illumination equalization; b. Abnormal target detection: The image is identified using the YOLOv7 network, and the bounding boxes of the remaining objects are output. c. Position mapping: By converting image coordinates to the car coordinate system, the relative position of the abandoned object inside the car is output. d. Based on the carriage segmentation results, output: Group X - Section N - Remains Type, and store it in the database.

8. The railway line early warning method based on multi-source data fusion as described in claim 7, characterized in that: The process of converting image coordinates to carriage coordinates is as follows: (1) Constructing the camera calibration matrix （9） (2) Define the car coordinate system: Origin point O: Center of the carriage floor; X: Length direction of the carriage, from front to rear; Y: In the direction of carriage width, from left to right; Z: Vertical direction, from floor to roof; (3) Inverse transformation of perspective projection; Input: Image point Pimg = (u, v); corresponding to the floor height zfloor = 0, assuming the debris is on the ground; then remove distortion, calculate the ray direction vector, solve for the floor intersection point, and transform to the car coordinate system. The car coordinate formula is as follows: （10）。 9. The railway line early warning method based on multi-source data fusion as described in claim 8, characterized in that: The derivation of formula (10) is as follows: Let the equation of the carriage floor be: (11) Point P in the camera coordinate system c satisfy: (12) Substitute into the plane equation: （13） The formula for the car body coordinates is obtained, namely formula (10).

10. The railway line early warning method based on multi-source data fusion as described in claim 1, characterized in that: The control terminal is a PC; the drone management platform is the DJI Sikong 2 drone management platform.

Images