Railway line information acquisition device and acquisition method based on deep learning

Abstract

The invention relates to the field of railway assistance, in particular to a railway line information acquisition device and method based on deep learning, and the device comprises a motion structure which comprises a small rail car which is a core bearing part of the device and is internally integrated with an intelligent acquisition module to realize data acquisition, processing and storage, and an information acquisition component which is used for acquiring information of the small rail car. Data acquired by the information acquisition component are uploaded to the data processing system through an RTSP protocol, real-time video streams of all the cameras are received, timestamp marking is carried out on the acquired data, and time sequence synchronism of data of different visual angles is ensured. And performing preprocessing steps of distortion removal, geometric correction and the like, and performing feature matching and alignment by using an image registration algorithm. A complete track section image is generated based on an image splicing technology, and a trackside three-dimensional reconstruction model is generated in combination with 3D point cloud data.

Description

Technical Field

[0001] The present invention relates to railway intelligent technology and its application field, and in particular to a railway line information collection device and collection method based on deep learning. Background Art

[0002] With the rapid development of the railway transportation industry, the requirements for the safety, reliability and operational efficiency of railway facilities are increasing. As a key link to ensure the safety of railway operations and improve efficiency, information collection and monitoring along the railway line has received increasing attention. At present, there are many devices for detecting geometric shapes and positions along railway lines. However, due to the vast territory of my country and the complex geographical conditions along the railway lines, it is often difficult for collection devices to accurately locate under the influence of various environments. There is still a lack of collection and correlation of line characteristics and external environmental characteristics. This device integrates multiple positioning methods for positioning and calibration, and is capable of accurate positioning in various complex situations, improving the efficiency and accuracy of collecting information about railways and along the lines, and generating three-dimensional scene reconstructions along the railway lines.

[0003] While using information collection devices along the railway to collect three-dimensional railway scene data, we can also use the collection devices to establish a three-dimensional scene database for the later development of intelligent inspection systems; combine historical data and real-time collection information to analyze the degradation trends of tracks and facilities, predict potential failures, and arrange maintenance work in advance; use the information in the three-dimensional scene database to provide high-precision autonomous positioning services for trains; through the trackside long-range information in the three-dimensional scene database, assist trains in perceiving the surrounding environment, optimize route planning, and improve operational safety and efficiency; use full-section images, trackside facility locations and other information in the three-dimensional scene database to provide three-dimensional scene data support for new line planning or existing line reconstruction, and optimize design solutions; provide a verification platform for new sensors, positioning technologies, communication technologies, etc., and promote technological innovation in the railway field. Summary of the Invention

[0004] The purpose of the present invention is to provide a railway information collection device based on deep learning, which aims to accurately collect data along the railway track and establish a railway three-dimensional scene database.

[0005] The above technical problems are solved by the following technical solutions: The present invention proposes the following technical solutions: a railway line information collection device based on deep learning, which includes a motion structure, the motion structure includes a track trolley; and,

[0006] The information collection component comprises a sleeper counting device arranged at the bottom of the rail trolley, a trackside close-up view collection structure arranged at the top of the rail trolley, and a far- and near-view collection structure arranged at the side of the rail trolley.

[0007] As a preferred embodiment of the railway line information collection device based on deep learning of the present invention: a searchlight is provided on the side of the rail trolley, and a data processing system is provided inside the rail trolley.

[0008] As a preferred embodiment of the railway line information collection device based on deep learning of the present invention: the sleeper counting device is an infrared camera, and the sleeper counting device interacts with the data processing system.

[0009] As a preferred embodiment of the railway line information collection device based on deep learning of the present invention: the trackside close-up collection structure includes a telescopic rod arranged on the top of the rail trolley, and a first camera is arranged on the top of the telescopic rod.

[0010] As a preferred embodiment of the railway line information collection device based on deep learning of the present invention: the long and short view collection structure includes a telescopic mechanical arm arranged on the side of the rail trolley, a rotating arm is arranged on the side of the telescopic mechanical arm, and a second camera is arranged on the top of the rotating arm.

[0011] The present invention also discloses a method for collecting information along a railway line, which includes a railway line information collection device based on deep learning as described in any of the above items, and also includes continuously photographing the scenery features in front of the vehicle body and beside the side tracks. The scenery features are transmitted to a data processing system, and the data processing system uses the collected scenery features to establish a railway line information and scene matching database.

[0012] As a preferred embodiment of the method for collecting information along the railway line of the present invention: the sleeper counting device photographs the sleepers at the bottom, and the infrared camera transmits the sleeper information to the data processing system for counting, thereby calculating the mileage traveled by the rail trolley.

[0013] As a preferred embodiment of the method for collecting information along the railway line of the present invention: a GPS sensor is also provided inside the rail trolley, and the GPS sensor calculates the current position of the trolley by receiving satellite signals. The rail trolley is also equipped with an IMU inertial measurement unit, which collects vibration data of the rail trolley for correcting the positioning error of the rail trolley body and compensating for the positioning error of the rail trolley caused by the lack of GPS signals or visual positioning. The sleeper counting, GPS sensor, IMU inertial measurement unit and image-based visual positioning jointly locate the travel position of the rail trolley, and finally the position of the rail trolley is determined by weighted voting.

[0014] As a preferred embodiment of the method for collecting information along the railway line of the present invention: the data processing system on the rail trolley generates point cloud data from the data, and then generates a sparse map through post-processing, and stores the final map in a three-dimensional scene database to achieve three-dimensional reconstruction of the trackside scene.

[0015] In a preferred embodiment of the present invention's railway information collection method, the camera is calibrated using Zhang's calibration method, CNN is introduced to onlinely correct camera extrinsic parameters, and image frames are precisely aligned with IMU data using timestamps. Depth estimation is performed on images captured by the camera using the Monodepth2 depth model. The camera coordinate system is mapped to the world coordinate system using the SuperPoint network and a pinhole model. This data is input into the VINS-Mono model, and the output 3D point cloud is used to generate a sparse map. The resulting map is stored in a 3D scene database.

[0016] The beneficial effects of the present invention are as follows: when collecting data around the track, the device achieves accurate positioning through multiple positioning methods. At the same time, it also builds a three-dimensional scene database through information collection components. Through the trackside perspective information in the three-dimensional scene database, it assists the train in perceiving the surrounding environment, optimizes path planning, and improves operational safety and efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings of the embodiments of the present invention. Obviously, the drawings described below only relate to some embodiments of the present invention and are not intended to limit the present invention. Among them:

[0018] Figure 1 It is a schematic diagram of the overall structure of the present invention.

[0019] Figure 2 A schematic diagram of the database establishment process in the present invention.

[0020] Figure 3 The figure is a schematic diagram of the data collection process of the camera in the present invention.

[0021] Figure 4 It is a schematic diagram of the positioning flow of various methods in the present invention.

[0022] Figure 5 A data flow diagram is established for deep learning participation in the present invention.

[0023] Figure 6 This is a mapping diagram from the camera coordinate system to the world coordinate system in the present invention. DETAILED DESCRIPTION

[0024] In order to enable those skilled in the art to better understand the present invention, the present invention is further described in detail below with reference to specific embodiments and the accompanying drawings.

[0025] The terms used in the present invention are those commonly used in the art in view of the functions of the present invention, but these terms may vary according to the intentions of those skilled in the art, precedents, or new technologies in the art. In addition, specific terms may be selected by the applicant, and in such cases, their detailed meanings will be described in the detailed description of the present invention. Therefore, the terms used in the specification should not be understood as simple names, but rather as the meanings of the terms and the overall description of the present invention.

[0026] Reference Figures 1 to 6 This embodiment provides a railway information collection device based on deep learning, including a motion structure 1, which includes a track trolley 11. The track trolley 11 is the core bearing part of the device, and has an internal integrated intelligent collection module to realize data collection, processing and storage. It is externally connected to functional modules such as cameras, searchlights 12, and telescopic brackets to provide comprehensive support for detection tasks. The car body has a dustproof and waterproof design to ensure the long-term stable operation of the trolley in a complex track environment. Preferably, the track trolley can be equipped with a solar power generation system to power it, which can provide power for the trolley to move, the camera to shoot, and the data processing system to process data; and

[0027] The information collection component 2 includes a sleeper counting device 21 disposed at the bottom of the track trolley 11, a trackside close-up acquisition structure 22 disposed at the top of the track trolley 11, and a long-range and short-range acquisition structure 23 disposed on the side of the track trolley 11. The data acquired by the information collection component 2 is uploaded to the data processing system via the RTSP protocol, which receives real-time video streams from each camera and timestamps the acquired data to ensure the temporal synchronization of data from different perspectives. Preprocessing steps such as de-distortion and geometric correction are performed, and feature matching and alignment are performed using image registration algorithms. A complete track cross-sectional image is generated based on image stitching technology, and a three-dimensional reconstruction model of the trackside is generated by combining 3D point cloud data.

[0028] Example 2, reference Figures 1 to 6 , which is the second embodiment of the present invention. Different from the previous embodiment, a searchlight 12 is provided on the side of the rail trolley 11. The searchlight 12 illuminates the rail trolley 11 when it travels at night or in poor lighting conditions. A data processing system is provided inside the rail trolley 11 to process the collected data.

[0029] Furthermore, the sleeper counting device is an infrared camera, which exchanges data with the data processing system, and the infrared camera photographs the sleeper rails at the bottom of the rail trolley 11 .

[0030] Furthermore, the trackside close-up acquisition structure 22 includes a telescopic rod 221 arranged on the top of the rail trolley 11, and a first camera 222 is arranged on the top of the telescopic rod 221. The telescopic rod 221 can simulate the viewing angle of the train front by adjusting the height, adapt to the operating height of different types of trains, and obtain the longitudinal front track and the environment along the line to obtain the status information of the track in front.

[0031] Furthermore, the long and short view acquisition structure 23 includes a telescopic mechanical arm 231 arranged on the side of the rail trolley 11, and a rotating arm 232 is arranged on the side of the telescopic mechanical arm 231. A second camera 233 is arranged on the top of the rotating arm 232. The second camera 233 continuously captures images on both sides of the vehicle body. When collecting mountainous areas and vegetation scenery features outside the track protection net, the telescopic mechanical arm 231 will drive the second camera 233 to rise and shoot distant long-range features. When it is necessary to shoot signs and scenery near the close-up track, such as ground signs, transformers, electrical boxes, etc., the second camera 233 will be lowered and adjusted to a suitable angle.

[0032] Usage process: When the rail trolley 11 is moving, the first camera 222 takes a longitudinal image of the track in front and the environment along the track to obtain the status information in front of the track, and the telescopic mechanical arm 231 will extend to drive the second camera 233 to extend to shoot the scenes on both sides of the track. The rotating arm 232 will drive the second camera 233 to rotate to adjust the shooting angle. When a train comes from the side of the rail trolley 11, the telescopic mechanical arm 231 will retract and drive the second camera 233 to retract to prevent collision with the oncoming vehicle.

[0033] Example 3, reference Figures 1 to 6 , which is the third embodiment of the present invention. Different from the previous embodiment, it also includes a method for collecting information along the railway line: the first camera 222 continuously captures the scenery features in front of the vehicle body and transmits them to the data processing system to establish a three-dimensional scene database along the line; the second camera 233 continuously captures the scenery features beside the track on the side of the vehicle body, including long and short views along the track, ground signs, trackside facilities, etc., and transmits them to the data processing system to establish a database for matching information along the line with scenes.

[0034] Furthermore, the infrared camera in the sleeper counting device 21 captures the sleepers at the bottom, and the infrared camera transmits the sleeper information to the data processing system for counting, so as to calculate the mileage traveled by the rail trolley 11. The standard distance between two adjacent sleepers is 0.6m. By recording the number of sleepers passed by the rail trolley, the mileage is calculated using the following formula: Mileage = Number of sleepers × Sleeper spacing.

[0035] Furthermore, a GPS sensor is also provided inside the rail trolley 11. The GPS sensor calculates the current position of the trolley by receiving satellite signals. The GPS sensor calculates the current position of the trolley (longitude, latitude, altitude) by receiving satellite signals. At the same time, the existing railway line map can be used for path tracking. By aligning the GPS positioning with the railway map, the specific position of the trolley on the railway line can be identified. The rail trolley 11 is also equipped with an IMU inertial measurement unit. The IMU inertial measurement unit collects vibration data of the rail trolley 11 and is used for coordinate correction of the rail trolley 11 body to compensate for the positioning error of the rail trolley 11 caused by the lack of GPS signals or visual positioning. The IMU inertial measurement unit is composed of an accelerometer and a gyroscope. The accelerometer measures acceleration data and obtains velocity (velocity = acceleration * time) through a single integration. Then, displacement (displacement = velocity * time) is obtained through a secondary integration. The gyroscope measures angular velocity and obtains angle or attitude (angle = angular velocity * time) through integration.

[0036] Furthermore, the sleeper count, GPS sensor, and IMU inertial measurement unit each give their own predictions for the target position, and ultimately determine the position of the rail trolley 11 through a weighted voting method. When positioning the rail trolley 11, the rail trolley 11 provides four positioning methods: GPS, IMU, vision, and sleeper count:

[0037] 1. The GPS sensor calculates the vehicle's current location (longitude, latitude, and altitude) by receiving satellite signals and can also use existing railway map data for path tracking. By aligning the GPS positioning with the railway map, the vehicle's specific location on the railway line can be identified.

[0038] However, due to the complex environment in which trains pass, satellite signals weaken when encountering deep mountains or tunnels, and positioning may be inaccurate. Therefore, the positioning needs to be calibrated. To this end, we introduce the following standards to evaluate GPS positioning:

[0039] Signal-to-noise ratio (C / N0):

[0040] If C / N0>40dB-Hz, the signal is considered very strong and the corresponding confidence level can be set to 1.0.

[0041] When C / N0 is between 30–40 dB-Hz, it can be proportionally mapped to 0.5–1.0; for example, if C / N0 = 35 dB-Hz, the confidence level is approximately 0.75.

[0042] If C / N0 < 30dB-Hz, the signal is weak and the confidence level may be lower than 0.5.

[0043]

[0044] Horizontal Dilution of Precision (HDOP):

[0045] When HDOP < 1.5, the positioning accuracy is high and a higher confidence level (e.g., 1.0) is assigned;

[0046] When HDOP is between 1.5 and 3.0, the accuracy is moderate and the confidence decreases linearly. For example:

[0047]

[0048] When HDOP>3.0, the positioning accuracy is poor, and the confidence is set to a lower value (such as 0.0 or 0.2,

[0049] Adjust according to actual needs).

[0050] RTK fixed / floating status:

[0051] If the status is FIXED, then C RTK =1.0

[0052] If the status is FLOAT, then C RTK =0.5

[0053] If the state is SINGLE, or other states, then C RTK =0.2

[0054] C GPS =α·C SNR +β·C HDOP +γ·C RTK (α+β+γ=1)

[0055] 2. Visual comparison: After running the car on the route multiple times, collecting data along the route and dynamically building and adjusting the 3D scene database, we can compare the images taken by the car with the 3D data in the 3D scene database while the car is moving to determine the specific position of the car. However, when using the camera for positioning, we also need to calibrate it according to the ambient light intensity. Ambient light (Lux value):

[0056] During the day (>1000 Lux), the visual data quality is high and the confidence level can be set to 1.0;

[0057] At dusk or low light (100-500 Lux), the confidence level drops to 0.5-0.8;

[0058] At night or in tunnels (<100 Lux), the visual confidence level may be only 0.2 to 0.3.

[0059] Track identification or target detection confidence:

[0060] Using the YOLOv5s deep learning model to detect railroad sleepers, and optical character recognition technology to detect features such as the numbers and symbols in the kilometer markers, if the detection confidence level is greater than 0.8, it is considered clear.

[0061] If the value is less than 0.5, the detection is considered unstable.

[0062]

[0063] Among them, α and β can be adjusted according to the experiment (for example, α = 0.5, β = 0.5), C img Indicates vision, C track Indicates detection.

[0064] For example:

[0065] Daytime (Lux = 1200, C img =0.9)→C img =1.0 (visually credible).

[0066] Night (Lux = 50, C img =0.4)→C img =0.3 (visual weight reduced).

[0067] 3. The IMU inertial measurement unit consists of an accelerometer and a gyroscope. The accelerometer measures acceleration data, integrates it to get velocity (velocity = acceleration * time), and then integrates it twice to get displacement (displacement = velocity * time). The gyroscope measures angular velocity, and integrates it to get angle or attitude (angle = angular velocity * time).

[0068] Angular velocity drift (θ draft ):

[0069] When θ draft When <0.01° / s, the confidence level is high (e.g., 0.95 to 1.0);

[0070] When θ draft When the value is >0.1° / s, the confidence level is reduced to 0.2-0.3.

[0071] Acceleration integral error:

[0072] Cumulative displacement error δ IMU When <5m, the data is considered stable; otherwise, the weight is reduced.

[0073]

[0074] For example:

[0075] Low drift (θ draft =0.005° / s)→C IMU =0.95 (IMU credible).

[0076] High drift (θ draft =0.08° / s)→C IMU =0.2 (IMU weight is reduced).

[0077] 4. Sleeper counting (or kilometer mark detection)

[0078] Standard sleeper spacing: usually about 0.6m.

[0079] Detection accuracy evaluation:

[0080] If the standard deviation of the detected sleeper spacing σ sleeper <0.002m, the confidence level is close to 1.0;

[0081] If σ sleeper If the value is larger (e.g. 0.1m), the confidence level is reduced to 0.8 or lower.

[0082]

[0083] For example:

[0084] where δ sleeper Indicates the average sleeper spacing deviation

[0085] Small standard deviation (σ sleeper =0.02m,δ sleeper =0.6m)→C sleeper =0.97 (credible).

[0086] Large standard deviation (σ sleeper =0.1m,δ sleeper =0.1m)→C sleeper =0.83 (the sleeper counting accuracy decreases).

[0087] When finally performing position positioning, we need to use the above four positioning methods to calibrate each other to obtain the final accurate positioning.

[0088] Example (combined with voting mechanism)

[0089] Voting Mechanism: The voting mechanism is applied to the decision-making multi-sensor fusion stage. In this mechanism, each sensor or positioning algorithm can be regarded as a "voter," each of which makes its own prediction of the target location based on its current confidence or weight. The final positioning result can be determined by majority voting or weighted voting.

[0090] Example 1: Night

[0091] Scenario description:

[0092] At night, due to insufficient light, the trackside scenes and track markings captured by the camera are unclear; at the same time, the GPS may be slightly weakened by the multipath effect; while the IMU and sleeper counts are less affected by light and perform more stably.

[0093] Assume confidence:

[0094] C GPS =0.3, C img =0.4, C IMU =0.8, C sleeper =0.9

[0095] Normalized calculation weights:

[0096] Total S = 0.3 + 0.4 + 0.8 + 0.9 = 2.4

[0097] ω GPS =0.3 / 2.4≈0.125,ω img =0.4 / 2.4≈0.167,ω IMU =0.8 / 2.4≈0.333,

[0098] ω sleeper =0.9 / 2.4≈0.375

[0099] Positioning results output by each sensor (voting):

[0100] GPS Positioning GPS =100.0m, visual positioning P img =102.0 meters, IMU positioning P IMU =101.0m, sleeper counting and positioning P sleeper =101.5 meters

[0101] Weighted voting fusion calculation:

[0102] P final =0.125×100.0+0.167×102.0+0.333×101.0+0.375×101.5≈101.23 meters

[0103] Example 2: Mountainous area

[0104] Scenario description:

[0105] In mountainous areas, the GPS signal is weak due to terrain obstruction and signal attenuation. However, in daytime with sufficient sunlight, the camera captures the environment and track markings more clearly. The IMU data stability is average, and the sleeper count detection is normal.

[0106] Assume confidence:

[0107] C GPS =0.3, Cimg =0.9, C IMU =0.7, C sleeper =0.8

[0108] Normalized calculation weights:

[0109] Total S = 0.3 + 0.9 + 0.7 + 0.8 = 2.7

[0110] ω GPS =0.3 / 2.7≈0.111,ω img =0.9 / 2.7≈0.333,ω IMU =0.7 / 2.7≈0.259,

[0111] ω sleeper =0.8 / 2.7≈0.296

[0112] Positioning results output by each sensor (voting):

[0113] GPS Positioning GPS =150.0m, visual positioning P img =148.0 meters, IMU positioning P IMU =149.0m, sleeper counting and positioning P sleeper =149.5 meters

[0114] Weighted voting fusion calculation:

[0115] P final =0.111×150.0+0.333×148.0+0.259×149.0+0.296×149.5≈148.71 meters

[0116] Example 3: Tunnel

[0117] Scenario description:

[0118] Due to insufficient light and signal obstruction in tunnels, GPS signals are extremely weak and vision is limited. Although the IMU may drift, it remains relatively stable in the short term. Sleeper counting is more reliable because it relies on a fixed structure.

[0119] Assume confidence:

[0120] C GPS =0.1, C img =0.2, C IMU =0.8, C sleeper =0.7

[0121] Normalized calculation weights:

[0122] Total S = 0.1 + 0.2 + 0.8 + 0.7 = 1.8

[0123] ω GPS =0.1 / 1.8≈0.056,ω img =0.2 / 1.8≈0.111,ω IMU =0.8 / 1.8≈0.444,

[0124] ω sleeper =0.7 / 1.8≈0.389

[0125] Positioning results output by each sensor (voting):

[0126] GNSS positioning GPS =90.0 meters, visual positioning P img =92.0 meters, IMU positioning P IMU =91.0m, sleeper counting and positioning P sleeper =91.5 meters

[0127] Weighted voting fusion calculation:

[0128] P final =0.056×90.0+0.111×92.0+0.444×91.0+0.389×91.5≈91.23 meters

[0129] Furthermore, the data processing system on the track car generates point cloud data from the trackside positioning data, and then generates a sparse map through post-processing. The final map is stored in the database to achieve three-dimensional reconstruction of the trackside scene. When using the track coordinate system to establish a mapping to the world coordinate system:

[0130] The camera is calibrated using Zhang's calibration method. The camera's intrinsic and extrinsic parameters are solved using OpenCV's calibrateCamera function. The collected image is dedistorted using the calibration results, and the image frame is accurately aligned with the IMU data through the timestamp. The depth d(u,v) of the calibrated image is estimated using the Monodepth2 depth model. The corresponding camera coordinates are obtained using the formula, and the SuperPoint network is used to replace the traditional ORB feature. Next, a mapping from the camera coordinate system to the rail coordinate system (world coordinate system) is established based on the pinhole model. Figure 6. Construct a preliminary three-dimensional point cloud. Since the extrinsic parameters may drift slightly due to vibration, temperature change or installation deviation during the movement of the trolley, an online correction module based on a lightweight CNN is designed. Its input is the current frame image and the initial extrinsic parameter [R|t], and the output is the correction amount ΔR and Δt (R refers to rotation, t refers to translation). The reprojection error is used for self-supervised training. After updating the extrinsic parameters, the new reprojection error is calculated, and the loss is back-propagated to update the correction network parameters online. The corrected image, IMU data and the depth map obtained by Monodepth2 are input into the VINS-Mono model, and sliding window BA optimization is performed to restore the true camera trajectory and three-dimensional structure. The output three-dimensional point cloud is used to generate a sparse map, and the final map is stored in the three-dimensional scene database to achieve three-dimensional reconstruction of the trackside scene.

[0131] Finally, it should be pointed out that the methods and devices described in detail above are merely embodiments, and those skilled in the art can modify these embodiments in different ways without departing from the scope of the present invention.

Claims

1. A railway information collection device based on deep learning, characterized by: include, A motion structure (1), comprising a track trolley (11); and The information collection component (2) comprises a sleeper counting device (21) arranged at the bottom of the rail trolley (11), a trackside near-view collection structure (22) arranged at the top of the rail trolley (11), and a far- and near-view collection structure (23) arranged at the side of the rail trolley (11).

2. The railway line information collection device based on deep learning according to claim 1, characterized in that: A searchlight (12) is provided on the side of the track trolley (11), and a data processing system is provided inside the track trolley (11).

3. The railway line information collection device based on deep learning according to claim 2, characterized in that: The sleeper counting device (21) is an infrared camera, and the sleeper counting device (21) performs data exchange with a data processing system.

4. The railway line information collection device based on deep learning according to claim 3, characterized in that: The trackside close-up scene acquisition structure (22) comprises a telescopic rod (221) arranged on the top of the track trolley (11), and a first camera (222) is arranged on the top of the telescopic rod (221).

5. The railway line information collection device based on deep learning according to claim 4, characterized in that: The near and far view acquisition structure (23) comprises a telescopic mechanical arm (231) arranged on the side of the rail trolley (11), a rotating arm (232) is arranged on the side of the telescopic mechanical arm (231), and a second camera (233) is arranged on the top of the rotating arm (232).

6. A method for collecting information along a railway line, comprising a railway line information collection device based on deep learning according to any one of claims 1 to 5, characterized in that: The scenery features in front of the vehicle and beside the tracks are continuously photographed and transmitted to the data processing system. The data processing system uses the collected scenery features to establish a database of information along the route and scene matching.

7. The method for collecting information along a railway line according to claim 6, wherein: The sleeper counting device (21) photographs the sleepers at the bottom of the rail trolley (11), and the sleeper counting device (21) transmits the sleeper information to the data processing system for counting, thereby calculating the mileage traveled by the rail trolley (11).

8. The method for collecting information along a railway line according to claim 7, wherein: The rail trolley (11) is also provided with a GPS sensor. The GPS sensor calculates the current position of the trolley by receiving satellite signals and processing them through a data processing system. The rail trolley (11) is also equipped with an IMU inertial measurement unit. The IMU inertial measurement unit collects vibration data of the rail trolley (11) for correcting the positioning error of the rail trolley (11) body and compensating for the positioning error of the rail trolley (11) caused by the lack of GPS signals or visual positioning. The sleeper count, the GPS sensor, the IMU inertial measurement unit and the image-based visual positioning system jointly locate the travel position of the rail trolley (11), and finally determine the position of the rail trolley (11) by weighted voting.

9. The method for collecting information along a railway line according to claim 8, wherein: The data processing system on the track trolley (11) generates point cloud data from the trackside positioning data, generates a sparse map after post-processing, and stores the final map in a three-dimensional scene database to achieve three-dimensional reconstruction of the trackside scene.

10. The method for collecting information along a railway line according to claim 9, wherein: The camera is calibrated using Zhang's calibration method, and CNN is introduced to online correct the camera extrinsic parameters. At the same time, the image frames and IMU data are precisely aligned through timestamps. The Monodepth2 depth model is used to estimate the depth of the images acquired by the camera. The SuperPoint network is used and the pinhole model is used to map the camera coordinate system to the world coordinate system. The above data is input into the VINS-Mono model, and the output 3D point cloud is used to generate a sparse map. The final map is stored in the 3D scene database.

Images