Method for train control and automatic danger avoiding through rail line vision and radar imaging monitoring

Abstract

According to the method for train control and automatic danger avoiding through track line vision and radar imaging monitoring, the sensing unit is arranged along the track, the whole track is covered with the sensing unit, the track state, the state of power supply equipment along the track and the surrounding environment are collected, and an image of a certain distance in front of running of a train can be constructed; images around and behind a train can be constructed, and the system is used for providing visual images of the train for remote control personnel, monitoring personnel or drivers on the ground, and can be used for perceiving a longer distance and perceiving disasters and dangers in advance; the distance and the size of an object can be identified through comparison with a reference image, parallax and stereo matching calculation so as to identify foreign matter invasion, the states of a track and power supply equipment along the track and the surrounding environment, and the system is used for automatically identifying track danger and prompting and alarming or controlling a train to enter a safe state.

Description

Technical Field

[0001] This invention relates to the field of remote control of unmanned and automatic driving of rail transit trains. By using image recognition and processing technology, radar recognition and processing technology, etc., images along the line or dot matrix cloud maps are fused and stitched together in real time to form a line image covering the entire line. The line image can be composed of track position kilometer markers. The control equipment can call up and display the line image through the track position kilometer markers, realizing unmanned remote control, assisted manual driving and automatic driving of rail transit train transportation process. Background Technology

[0002] Currently, real-time track status acquisition mostly adopts the method of installing sensing equipment on the train. The train uses sensing units such as cameras and radar to obtain the track and environmental status in front of and around the train for train operation control. However, the detection range of the on-board sensing units is insufficient to cover the safe distance in front of the train due to constraints such as track environment such as curves. Furthermore, the image clarity and recognition of on-board sensing units such as cameras are affected by vibrations during train operation. At the same time, the installation space of on-board displays is limited, making it impossible to provide a large screen to clearly display images of the track along a certain distance in front.

[0003] Some proposals suggest using ground-based cameras along the track to transmit data to onboard equipment via vehicle-to-ground wireless communication. The onboard display then shows an image of the track a certain distance ahead of the train. However, this method requires very high real-time communication volumes of large amounts of data via vehicle-to-ground wireless communication, and the onboard display screen is not large enough to clearly show the image of the track a certain distance ahead.

[0004] The current method of providing images of a certain distance ahead of the train has the aforementioned problems, and therefore these problems need to be avoided.

[0005] According to patent searches, the following patents are mainly related to this invention: 1. Chinese invention patent application number "201110275024.4", application date "2011-09-16", publication number "CN102307299A", publication date "2012-01-04", titled "An Image-Based Remote Observation System for Rail Transit", applicant "Beijing Jiaotong University", describes an image-based remote observation system for rail transit, comprising an image acquisition module, an image processing module, and an on-board image receiving and display terminal. The line image acquisition module consists of cameras installed along the line, responsible for acquiring line images within a certain distance along the line; the on-board image receiving and display terminal is responsible for receiving and displaying the original images from cameras within a certain distance in front of the train or the images of the train ahead processed by the image processing center; the image processing module is mainly responsible for processing the images from the cameras installed along the line. The beneficial effects of this invention are: as a supplement to the existing onboard ATP / ATO system that mainly relies on facilities such as track circuits, ground transponders, and trackside cables, it also provides beyond-line-of-sight image information for the driver and automatic driving system, and improves the redundancy and reliability of the train operation control system by relying on non-track circuit methods with complementary performance.

[0006] 2. A German invention patent with application number "201110463071," application date "2011-12-09," publication number "CN102665084A," publication date "2012-09-12," titled "Ambient Environment Display Device and Method for Displaying Panoramic Images of a Vehicle Equipped with Such an Ambient Environment Display Device," and applicant "Ides GmbH & Co. KG," describes an ambient environment display device comprising at least one image source, an image signal processing device, and at least one image display device. An image input signal provided by the image source is transmitted to the image signal processing device, which displays the transmitted image... An image output signal, which is processed from an input signal into a panoramic image, is further transmitted by an image signal processing device to an image display device for displaying the panoramic image. At least one first region of the panoramic image is compressed without distortion or with a first degree of compression, and at least one second region of the panoramic image is compressed with a second degree of compression. The feature is that the image signal processing device is designed such that portions of the regions expressed without distortion or with compression can be variably selected across the entire panoramic image, such that the respective degree of compression of the compressed regions can be arbitrarily selected, and such changes to the compressed regions can be made during operation of the ambient display device.

[0007] The aforementioned patent only provides a method for remote observation of trains and panoramic imaging of the surrounding environment of vehicles, but it does not solve the aforementioned problems. Summary of the Invention

[0008] The technical problem this invention aims to solve is to address the deficiencies in existing technologies. By deploying sensing devices such as cameras and / or radars along the rail transit line, ground equipment acquires images and / or point cloud images at various locations via wired communication. Through image fusion and image stitching fitting, images required for train operation control are formed. These images can be combined with track positions or track kilometer markers to create real-time updated images and / or point cloud images. Ground equipment can acquire train positions or kilometer marker data in real time. The ground remote control console retrieves and displays images at a certain distance ahead of the train based on its position, and can also retrieve and display images of the train's surroundings and rear. This allows for remote control of train operation and monitoring of track conditions and the surrounding environment. Simultaneously, image processing algorithms can automatically identify track conditions, the status of power supply facilities along the line, and foreign object intrusion, thereby controlling train actions and ensuring train operation safety.

[0009] This invention utilizes sensing units deployed along the track to collect data on track status, power supply equipment status, and the surrounding environment. These sensing units can employ visible light, infrared, or other visual imaging technologies, and can achieve stereo imaging using binocular or multi-view cameras. By comparing images (with a reference image), parallax, and stereo matching, the invention calculates and identifies object distances and sizes, thereby detecting foreign object intrusions, track and power supply equipment status, and the surrounding environment. Alternatively, the sensing units can employ microwave radar, such as millimeter-wave radar (4D or 5D millimeter-wave radar), to perform real-time point cloud imaging of the track, power supply equipment, and surrounding environment. This allows for environmental perception, identification of object distances, sizes, and motion (including slow motion), and comparison with a reference point cloud image to identify foreign object intrusions and monitor track and power supply equipment status. Furthermore, the invention employs lidar imaging to perceive the environment, identify object distances, sizes, and motion (including slow motion), and compare with a reference point cloud image to identify foreign object intrusions and monitor track and power supply equipment status. Image enhancement and multi-sensor fusion technology can be used to further improve environmental perception and display clarity, ensuring train operation safety.

[0010] For double-track lines, when the distance between adjacent tracks can be covered by a single-point sensing unit, the images can be combined by the ground control center or ground remote control console into up and down image data, which are used for up and down train operation control respectively; when the distance between adjacent tracks is far, sensing units can be configured independently; multiple tracks within a station can be configured with multiple sensing units according to the on-site application to achieve full track coverage.

[0011] The sensing units installed along the track can be connected to the backbone network via wireless and wired communication to form a network. Each sensing unit along the track can be coded, and ground control equipment and ground remote control console can acquire the corresponding sensing image data in real time according to the coding of the sensing unit.

[0012] The backbone network can use Ethernet communication, and the transmission medium can be optical fiber. The backbone network can be configured in a redundant manner, and a ring network can be used for specific applications such as subways. The sensing units can be redundantly and compositely configured to improve reliability and availability.

[0013] Images can be compressed before transmission to reduce data transmission volume. Ground control centers and ground remote control consoles can decompress the images to obtain undistorted or minimally distorted images.

[0014] Train-to-ground communication can employ wireless communication methods such as GSM-R, LTE-R, WLAN, and leaky cables to ensure real-time data exchange between the train and the ground, supporting remote control of train operation from the ground. When high-bandwidth continuous wireless train-to-ground communication is used, it can be used to transmit real-time image data of a certain distance ahead of the train, enabling manual driving to see further distances (such as curves) and to detect disasters in advance (such as mudslides, flooding, and tunnel collapses). Train-to-ground communication can be constructed using a new transponder system to meet the data exchange needs between the train and the ground and the train operation control needs of sections and stations under fixed block signaling.

[0015] Point-to-ground communication and wireless continuous train-to-ground communication can be redundant. Point-to-ground communication can serve as a backup for wireless continuous train-to-ground communication. Point-to-ground communication cannot transmit continuous data such as remote control commands from the ground and train status interaction. Point-to-ground communication and wireless continuous train-to-ground communication can work together. Point-to-ground communication can be used for data interaction between the train and the ground in conventional train operation control, such as line data, route information, temporary speed limits, and movement authorization information. Continuous wireless communication between the train and the ground can be used for remote operation commands, image information at a certain distance in front of the train and / or around and behind the train, and real-time position, speed, and status of the train.

[0016] The new transponder system (see patents CN2022112162764 and CN2022109841666) can construct fixed block sections, providing onboard equipment with train positioning and track identification, special section prompts such as horn prompts and phase transition prompts, ground equipment with train occupancy and location tracking (section location and track identification), train integrity checks, and can be used for the joint control of platform screen doors and platform screen doors, and can support automatic train operation.

[0017] Ground equipment can include train control center, interlocking, integrated train-air interlocking equipment, signal-locking system, and ground remote control console, etc. The ground control center can be a general term for train control center, interlocking, integrated train-air interlocking equipment, and signal-locking system.

[0018] Onboard equipment can store track data or acquire it through vehicle-to-ground communication. Track data includes basic track data, route and movement authorization information, and temporary speed limit information. The basic track data includes basic track parameters such as track gradient and length, curve curvature and length, track sections, and transponder link information; track speed information such as maximum permissible speed and maximum permissible speed of switches; and special information such as switch locations and parameters, pantograph locations, unloading stations, loading stations, tunnel entry and exit points, and horn warnings. Route and movement authorization information includes station arrival and departure routes and movement authorization target points within sections. Temporary speed limit information refers to information on speed restrictions imposed on trains due to construction or other reasons.

[0019] Ground equipment (ground control center and ground remote control console) can acquire real-time information such as train position, direction of travel, train speed, and train status. The ground control center, such as the train control center or interlocking, generates train movement authorization information based on dispatching and transportation information. The onboard train control equipment can receive the corresponding movement authorization information to automatically control train operation or supervise the safe operation of the train under manual or remote control. The ground remote control console can acquire information such as the corresponding train position, speed, direction of travel, and status, as well as movement authorization information, to generate a continuous speed control curve for the target distance. At the same time, it can retrieve images of a certain distance in front of the train, as well as images of the train's surroundings and rear. Operators can control train operation based on the speed control curve and by observing the displayed images. The train control commands are sent to the onboard train control equipment via continuous wireless communication between the train and the ground. When foreign objects are observed intruding into the track or other abnormal situations, the ground remote control personnel can control the train to stop in time, thus supporting remote control of train operation.

[0020] Train-onboard equipment can acquire line data, including basic line data, route information, and temporary speed limit information, through vehicle-to-ground communication. Train-onboard equipment can acquire train movement authorization information and generate a continuous speed control curve for the target distance. The train can achieve automatic driving, manual driving under speed control curve monitoring, or the train-onboard control equipment can acquire instructions from the ground for remote control of the train in real time and control the train operation according to the instructions under the limitations of the speed control curve.

[0021] When vehicle-to-ground communication includes continuous wireless communication, train operation can adopt virtual block or moving block mode. Ground equipment can obtain information such as train position and direction and generate train movement authorization information based on dispatching and transportation information. When vehicle-to-ground communication adopts a new transponder system, train operation can adopt an equivalent fixed block mode. Ground equipment can generate train movement authorization information based on information such as train occupancy (block section and track occupancy identification) and running direction, and based on dispatching and transportation information.

[0022] In ground remote control mode: The ground remote control console can obtain information from sensing units near the train based on the train's real-time position (track kilometer marker), direction of operation, and track identification. Then, through image processing technologies such as fusion and stitching, it can construct an image of a certain distance in front of the train in real time, and can construct images of the train's surroundings and rear in real time for ground remote control of train operation. It supports constructing real-time images of a certain distance behind the train as the train moves backward, and can construct images of the train's surroundings and front in real time.

[0023] In automatic train operation mode: The ground control center can obtain information from the sensing units near the train based on the train's real-time location (track kilometer marker), direction of travel, and track identification, thereby obtaining images of a certain distance ahead of the train. Through image analysis and comparison with reference images, dangers are identified, and the train is automatically controlled to enter a safe state.

[0024] For short track lines such as urban rail and underground rail, the ground control center can fuse and stitch together images along the track to create a complete real-time image covering the entire line. The images can be stored and updated in real time by combining track positions or track position kilometer markers. The ground remote control console can access images a certain distance ahead of the train by identifying the train position or train position kilometer markers, direction of travel, and track. It can also access images around and behind the train. By displaying images in real time, it supports the train moving backward, thus enabling remote control of the train's operation from the ground. In automatic train operation mode, the ground control center directly identifies hazards through image processing algorithms and automatically controls the train to enter a safe state.

[0025] A certain distance in front of the train should meet the train's braking distance plus a certain margin distance, thereby ensuring the safety of train operation.

[0026] In manual driving mode, remote ground control mode, and manned driving mode, the ground control center can use real-time updated images covering the entire line to manually confirm the track status, the environmental conditions along the track, and foreign object intrusion. Once a danger is detected, an alarm should be triggered and the train should be brought under emergency control to a safe state. Alternatively, the image algorithm can automatically identify abnormalities such as track status, surrounding environment, and foreign object intrusion, automatically trigger an alarm, and prompt manual confirmation and subsequent operations, such as remotely controlling the train to stop in time, or automatically triggering an alarm and prompting the nearest train at the abnormal location to shorten its movement authorization to behind the fault point or to stop the nearest train, thereby ensuring the safe operation of the train.

[0027] The ground control center can periodically establish and maintain the image baseline for the entire line. The ground control center can generate baseline images during track maintenance windows, which can be manually verified (e.g., no foreign object intrusion, normal track condition and surrounding environment). The image baseline is re-verified when construction or track condition changes. Images acquired in real time during operation can be compared with the baseline images to identify abnormal situations and trigger alarms and indicate faults.

[0028] The ground control center can support autonomous driving, remote operation driving, and manual driving, with the priority being: manual driving > remote operation > autonomous driving.

[0029] For remote control of train operation, each ground train control console can be responsible for the remote control of one train.

[0030] Ground-based remote control of train emergency stops can send emergency braking commands or braking commands to the train via vehicle-to-ground communication, allowing the train's onboard control equipment to execute emergency braking or braking operations. Alternatively, it can disconnect the power supply to the train from the ground to allow the train to coast to a stop or automatically trigger braking to a stop when the train loses power, ensuring train operation safety.

[0031] By installing onboard cameras on the train, the status of critical equipment can be monitored. The images can be sent to ground equipment via train-to-ground communication. Ground control equipment can identify the status of the onboard equipment. The ground equipment judges the equipment status based on the onboard image information and the onboard equipment status information, thereby determining whether the equipment can operate normally. It supports power-on startup of the onboard equipment and remote power-off restart after a fault.

[0032] The train can be equipped with onboard sensing units such as visual cameras or stereo imaging that can fuse infrared images, and point cloud images constructed by lasers, millimeter waves, terahertz radars, etc. The onboard intelligent control equipment can automatically identify the status of the track ahead, power supply equipment, and the surrounding environment, and identify foreign object intrusion. In automatic driving mode, it can automatically control the train to decelerate and stop. In manual driving or remote control mode, it can promptly remind the driver and control center.

[0033] The images retrieved by the ground control center or ground remote control console can be either the driver's indoor view or a wide view outside the vehicle. The wide view outside the vehicle can be used to control the train in key positions, such as underground locomotive ore release, precise train stopping, checking the correct position of switches, confirming the raising and lowering of pantographs in phase separation zones, and accurately positioning the train at loading or unloading stations.

[0034] The images displayed by the ground control center or ground remote control console can be updated according to the position and time of the train's movement. The position range of the displayed images can be formed by combining track kilometer markers or by combining the train's position with sensing units around the train, ensuring that images of a certain distance in front of the train are displayed. Images around and behind the train can be displayed in real time, and the image accuracy of key areas can be improved, such as the turnout area, which can be used to identify the turnout status.

[0035] The beneficial effects of this invention are as follows: This invention belongs to the field of rail transit. By arranging sensing units along the track, the sensing units cover the entire track to collect track status, power supply equipment status and surrounding environment. The ground control center can monitor the track and power supply equipment status and surrounding environment in real time, identify foreign object intrusion, and the ground remote control console can display images of the train ahead and surrounding area for train operation control, thereby ensuring train operation safety. Attached Figure Description

[0036] Figure 1 This is a schematic diagram of a monitoring and operation system constructed through a ground ring network and vehicle-to-ground leaky cable communication. Figure 2 This is a schematic diagram of a monitoring and operation system constructed through a ground-based redundant network and vehicle-to-ground WLAN communication.

[0037] In the diagram: 1-Ground control equipment, 11-Remote control console, 12-Ground monitoring center, 13-Ground control center, 14-Data server, 2-Switch, 3-Sensing unit, 4-Communication gateway, 5-Continuous wireless communication between vehicle and ground, 51-Leaky cable, 52-WLAN communication, 6-On-board display equipment, 7-On-board train control equipment, 8-New transponder system, 81-On-board new transponder system, 82-Ground new transponder system, 83-Ground new transponder control system. Detailed Implementation

[0038] The present invention will be further described below with reference to specific embodiments and accompanying drawings. It should be understood that the specific embodiments described herein are merely illustrative of this application and are not intended to limit this application. All other embodiments obtained by those skilled in the art based on the embodiments provided in this application without creative effort are within the scope of protection of this application.

[0039] This invention proposes to deploy sensing units 3 along the track. These sensing units 3 cover the entire track to collect track conditions, the status of power supply equipment along the track, and the surrounding environment. This allows for the construction of images of a certain distance ahead of the train, as well as images of the train's surroundings and rear. These images can be used to provide visual images of the train to ground-based remote operators, monitoring personnel, or drivers. They can also be used to see further distances (such as curves) and to detect disasters in advance (such as mudslides, flooding, and tunnel collapses). By comparing with a reference image, using parallax and stereo matching to calculate and identify the distance and size of objects, the invention can identify foreign object intrusion, the status of the track and power supply equipment along the track, and the surrounding environment. This allows for the automatic identification of track hazards and the prompting of alarms or the control of the train to a safe state.

[0040] The sensing unit 3 is selected based on the distance between the up and down tracks and the distance between the tracks. If the distance is close, the sensing unit 3 can cover the up and down tracks at the same time. The images can be combined into up and down image data by the ground control center or the ground remote control console, and used for up and down train operation control respectively. When the distance between adjacent tracks is far, the sensing unit 3 can be configured independently. Multiple sensing units 3 can be configured for multiple tracks in the station according to the on-site application to achieve full track coverage.

[0041] The sensing unit 3 can be connected to the backbone network via wireless and wired communication to form a network. The backbone network can use Ethernet communication and optical fiber as the transmission medium. The backbone network can be configured in a redundant manner, and a ring network can be used for specific applications. The sensing unit 3 can be redundantly and compositely configured to improve reliability and availability. Each sensing unit 3 along the track can be encoded. The ground control center 12 and the ground remote control console 11 can obtain the image data of the corresponding sensing unit 3 in real time according to the encoding of the sensing unit 3 (corresponding network address).

[0042] Videos or images can be transmitted after compression, and the receiver can decompress them without loss or with minimal distortion to obtain undistorted or minimally distorted images.

[0043] The sensing unit 3 can perform visual imaging such as visible light and infrared, and can achieve stereo imaging through binocular or multi-view cameras; the sensing unit 3 can use microwave radar such as millimeter-wave radar (4D millimeter-wave radar or 5D millimeter-wave radar) to perform real-time point cloud imaging of the track, power supply equipment along the line and the surrounding environment, thereby sensing the environment, identifying the distance and size of objects and their movement (including slow movement); the sensing unit can use lidar imaging to sense the environment, identify the distance and size of objects and their movement (including slow movement); it can identify the distance and size of objects through multi-sensor fusion technology, image comparison (comparison with reference image), parallax and stereo matching calculation, etc., thereby identifying foreign object intrusion, the status of track and power supply equipment along the line and the surrounding environment for automatic hazard identification.

[0044] Ground control equipment 1 and on-board train control equipment 7 can directly acquire images of a certain distance ahead of the corresponding train in real time via the network using the location number (or network address) of sensing unit 3. They can also acquire images of the area around or behind the train. Through image fusion and stitching fitting, suitable images are formed for display to monitor or assist train operation control, or to automatically identify hazards and respond promptly. For short track lines, ground monitoring center 12 can fuse and stitch images along the track in real time to form a complete image covering the entire line and store it on a data server. The images can be stored and updated in real time by combining track positions or track position kilometer markers. The ground remote control console 11 and the on-board train control equipment 7 can access images a certain distance ahead of the train by identifying the train's position or train position kilometer markers, direction of travel, and track. They can also access images around the train and behind it, which are used to monitor or guide train operation control, or to automatically identify hazards and respond promptly. Through image fusion and processing, images of the train moving backward a certain distance can be displayed, as well as images of the train's surroundings and ahead, thus supporting the train's backward movement. In automatic train operation mode, the ground monitoring center 12 directly identifies hazards and responds through image processing algorithms.

[0045] Image fusion processing can be achieved through image enhancement display and multi-sensor fusion technology, further improving environmental perception capabilities and image display clarity. It can be used to provide remote control console 11, ground monitoring center 12, and vehicle-mounted display equipment 6 with visualized images of the track, power supply equipment and environment at a certain distance ahead of the train. It can also be used to provide ground personnel and drivers with image data at a certain distance ahead of the train, enabling them to see further distances (such as curves) and perceive disasters in advance (such as mudslides, flooding, tunnel collapses, etc.) during manual driving.

[0046] Train-to-ground communication only transmits conventional train operation control data between train and ground (train operation control data exchange required between train and ground in the block system), such as track data, route information, temporary speed limits, and movement authorization information transmitted between ground and train, and train position information (indirectly providing train occupancy and track occupancy information) between train and ground. The new transponder system 8 provides point-to-ground communication and wireless continuous train-to-ground communication 5, which can be redundant. Point-to-ground communication 8 can serve as a backup for wireless continuous train-to-ground communication 5.

[0047] When supporting remote train control and onboard track image display, point-to-point train-to-ground communication 8 can serve as a backup and redundancy for train-to-ground wireless communication 5 in the conventional train operation control data interaction between the train and the ground. Train-to-ground wireless communication 5 is additionally used by ground equipment 1 to obtain the real-time position (track kilometer marker), speed, and train status of the train. Onboard vehicle control equipment 7 can obtain and display remote train control commands and information from sensing units in front of and near the train. Point-to-point train-to-ground communication 8 and continuous wireless train-to-ground communication 5 can work together. Point-to-point train-to-ground communication 8 can be used for data interaction between the train and the ground in the conventional train operation control. Continuous wireless communication 5 can be used by ground equipment 1 to obtain the real-time position (track kilometer marker), speed, and train status of the train, and by onboard vehicle control equipment 7 to obtain and display remote train control commands and image information in front of and near the train.

[0048] The new transponder system 8 enables train positioning, train integrity checks, flexible automatic block control, track identification, vehicle-to-ground control, and the transponder itself, thus achieving train control functions and fully automated train operation. The continuous vehicle-to-ground wireless communication system 5 enables moving block control, vehicle-to-ground control, etc., thus achieving train control functions and fully automated train operation.

[0049] Ground equipment 1 (ground control center 13 and ground remote control console 11) can acquire information such as train position, direction of travel, train speed, and train status in real time. Ground control center 13, such as train control center or interlocking, generates train movement authorization information based on dispatching and transportation information. Onboard train control equipment 7 can receive the corresponding movement authorization information to automatically control train operation, or supervise the safe operation of the train under manual or remote control under movement authorization. Ground remote control console 11 can acquire information such as corresponding train position, speed, direction of travel, and status, as well as movement authorization information, to generate a target distance continuous speed control curve. At the same time, it can retrieve images of a certain distance in front of the train, as well as images of the train's surroundings and rear. Operators can control train operation based on the speed control curve and by observing the displayed images. The train control command is sent to the onboard train control equipment 7 via continuous wireless communication between the train and the ground. When foreign objects are observed intruding into the track or other abnormal situations, the ground remote control personnel can control the train to stop in time, thus supporting remote control of train operation.

[0050] The train's onboard equipment 7 can acquire line data, including basic line data, route information, and temporary speed limit information, through vehicle-to-ground communication. The train's onboard equipment 7 can acquire train movement authorization information and generate a target distance continuous speed control curve. The train can achieve automatic driving, or manual driving under speed control curve monitoring, or the train's onboard control equipment can acquire instructions from the ground for remote train control in real time and control the train's operation according to the instructions under the constraints of the continuous speed control curve.

[0051] In the ground remote control mode: The ground remote control console 11 can obtain information from the sensing unit 3 near the train based on the train's real-time position (track kilometer marker), running direction and track identification, and then form a suitable image through image processing technologies such as fusion and stitching, or retrieve images from the data server through the train's position kilometer marker, and build an image of a certain distance in front of the train in real time. It can also build images of the train's surroundings and rear in real time, which can be used for ground remote control of train operation and precise train stopping, loading and unloading of cargo observation, etc. It supports building a real-time image of a certain distance behind the train when the train moves backward, and can build images of the train's surroundings and front in real time.

[0052] In manual driving mode: The on-board display unit 6 can obtain information from the sensing unit 3 near the train based on the real-time position (track kilometer marker), direction of operation and track identification of the train through vehicle-to-ground communication. Then, it can form a suitable image through image processing technologies such as fusion and stitching, or retrieve the image from the data server through the train position kilometer marker to build an image of a certain distance in front of the train in real time. It can build images around and behind the train in real time to assist the driver in controlling the train operation and accurately stopping the train, observing the loading and unloading of goods, etc., and support the train to provide image data for backward movement.

[0053] In automatic train operation mode, manual driving mode, and remote ground control mode: Ground monitoring center 12 can obtain image information of sensing unit 3 near the train based on the train's real-time position (track kilometer marker), running direction, and track identification, thereby obtaining an image of a certain distance ahead of the train. Through image analysis and comparison with reference images, it can identify dangers and issue alarms. It can be manually confirmed and subsequent operations can be performed, such as remotely controlling the train to stop in time, or the ground control center 13 can automatically control the train to enter a safe state by exchanging information with the ground control center 13, through methods such as authorized relocation, direct control of the train to brake and stop, or cutting off traction power supply; Ground monitoring center 12 can use real-time updated images covering the entire line to manually confirm track conditions, trackside environmental conditions, foreign object intrusion, etc. Once a danger is detected, it should issue an alarm and urgently control the train to enter a safe state.

[0054] The ground monitoring center 12 can periodically establish and maintain the image baseline of the entire line. The ground control center can generate baseline images during track maintenance windows, which can be manually confirmed (e.g., no foreign object intrusion, normal track status and surrounding environment). The image baseline is reconfirmed when construction or track status changes. Images acquired in real time during operation can be compared with the baseline images to identify abnormal situations and trigger alarms and indicate faults.

[0055] For remote control of train operation, each ground train control console 11 can be responsible for the remote control of one train.

[0056] Ground-based remote control of train emergency stops can send emergency braking commands or braking commands to the train via vehicle-to-ground communication, allowing the onboard control equipment 7 to execute emergency braking or braking operations. Alternatively, it can disconnect the power supply to the train from the ground to cause the train to coast to a stop or automatically trigger braking to stop the train in the event of a power failure, thus ensuring train operation safety.

[0057] By installing onboard cameras on the train, the status of critical equipment can be monitored. The images can be sent to ground equipment via train-to-ground communication. Ground control equipment can identify the status of the onboard equipment. The ground equipment judges the equipment status based on the onboard image information and the onboard equipment status information, thereby determining whether the equipment can operate normally. It supports power-on startup of the onboard equipment and remote power-off restart after a fault.

[0058] The train can be equipped with onboard sensing units such as visual cameras or stereo imaging that can fuse infrared images, and point cloud images constructed by lasers, millimeter waves, terahertz radars, etc. The onboard intelligent control equipment can automatically identify the status of the track ahead, power supply equipment, and the surrounding environment, and identify foreign object intrusion. In automatic driving mode, it can automatically control the train to decelerate and stop. In manual driving or remote control mode, it can promptly remind the driver and control center.

[0059] The images retrieved by the ground monitoring center 12 or the ground remote control console 11 can be either the driver's indoor view or a wide view outside the vehicle. The wide view outside the vehicle can be used to control the train in key positions, such as underground locomotive ore release, precise train stopping, checking the correct position of the turnout, confirming the raising and lowering of the pantograph in the phase separation zone, and accurately positioning the train at the loading or unloading station.

[0060] The images displayed by the ground monitoring center 12 or the ground remote control console 11 can be updated according to the position and time of the train's movement. The position range of the displayed images can be formed by combining track kilometer markers or by combining train position with sensing units around the train, ensuring that images of a certain distance in front of the train are displayed. Images around and behind the train can be displayed in real time, and the image accuracy of key areas can be improved, such as the turnout area, which can be used to identify the turnout status.

[0061] The beneficial effects of this invention are as follows: by arranging sensing units along the track, the sensing units cover the entire track to collect track status, power supply equipment status and surrounding environment. The ground control center can monitor the track and power supply equipment status and surrounding environment in real time, identify foreign object intrusion, and the ground remote control console can display images of the train ahead and surrounding area for train operation control, thereby ensuring train operation safety.

[0062] The above embodiments are for illustrative purposes only and are not intended to limit the invention. Those skilled in the art can make various changes or modifications without departing from the spirit and scope of the invention. Therefore, all equivalent technical solutions should also fall within the protection scope of the invention, which should be defined by the claims.

Claims

1. A method for train control and automatic hazard avoidance using visual and radar imaging monitoring along the track, characterized by: By deploying sensing units 3 along the track to collect track status, power supply equipment status and surrounding environment along the entire track, images of a certain distance in front of the train can be constructed, as well as images of the train's surroundings and rear. These images can be used to provide visual images of the train's surroundings to remote train operators, ground monitoring personnel and drivers, and to perceive further distances and detect disasters and dangers in advance. The images displayed by the ground monitoring center 12, the ground remote control console 11 and the vehicle display device 6 can be updated according to the position and time of the train's movement. The position range of the displayed images can be composed of processed images combined with the train's position or images collected by the train's surrounding sensing unit 3 combined with the train's position. It can display images at a certain distance in front of the train and can display images around and behind the train in real time. At the same time, the image accuracy of key areas can be improved. Sensing unit 3 is set according to the distance between adjacent tracks. If the distance is close, sensing unit 3 can be used to cover adjacent tracks simultaneously. The ground monitoring center, ground remote control console, or vehicle display equipment will automatically combine the up and down image data of each track. If the distance between adjacent tracks is far, sensing unit 3 can be configured independently. Multiple sensing units 3 can be configured for multiple tracks in the station according to the on-site application.

2. The method for train control and automatic hazard avoidance using visual and radar imaging monitoring along the track as described in claim 1, characterized in that: The sensing unit 3 can be a visual imaging or stereo imaging such as visible light or infrared. The sensing unit 3 can use microwave radar to perform real-time point cloud imaging or stereo imaging of the track, power supply equipment along the line, and surrounding environment. The sensing unit can use lidar imaging or stereo imaging. The sensing unit 3 can be redundantly and compositely configured, and can be connected to the backbone network through wireless and wired communication to form a network. For specific applications, a ring network can be used. The ground monitoring center 12, the ground remote control console 11, and the vehicle-mounted display device 6 can directly acquire the image data of the corresponding train in real time through the encoding of the sensing unit 3 or the corresponding network address based on the train's position, direction of operation, and track. Alternatively, the ground monitoring center 12 can fuse and stitch together images along the track in real time to form a complete image covering the entire line, and store it in the data server based on the track position combination. The ground remote control console 11 and the vehicle-mounted display device 6 can retrieve images of a certain distance in front of the train from the data server based on the train's position, direction of operation, and track, and can retrieve images of the area around and behind the train.

3. The method for train control and automatic hazard avoidance using visual and radar imaging monitoring along the track as described in claim 1, characterized in that: Ground control equipment 1 and vehicle-mounted display equipment 6 can use the image data directly acquired by the sensing unit 3 to form a suitable image through image fusion and stitching fitting for displaying an image at a certain distance in front of the train, and can display images around the train and behind it for manual monitoring, hazard confirmation or to assist in train operation control. The ground remote control console 11 and the vehicle-mounted display device 6 can call the image data stored on the data server through the train's location, organize and display images at a certain distance in front of the train, and display images around the train and behind it, for manual monitoring, hazard identification and confirmation, or to assist in train operation control. By using image stitching and fitting processing, images of the train moving backward a certain distance can be displayed, as well as images of the area around and in front of the train, thus supporting the train's backward movement. Image fusion processing can be achieved through image enhancement display and multi-sensor fusion technology, further improving environmental perception capabilities and image display clarity.

4. Track-side visual and radar imaging monitoring for train control and automatic hazard avoidance methods, characterized by: Ground monitoring center 12 can obtain image information of sensing unit 3 near the train based on the train's real-time location, direction of operation and track. It can identify dangerous disasters and issue alarms through multi-sensor fusion technology, image comparison and comparison with reference image, image and video analysis, parallax and stereo matching calculation and other methods. It can be manually confirmed and subsequent operations can be performed. It can remotely control the train to stop in time, or it can automatically control the train to enter a safe state by interacting with ground control center 13 through methods such as authorized retraction, direct control of the train to brake and stop or cut off traction power supply. The ground monitoring center 12 can manually confirm the track status, the environmental conditions along the track, and foreign object intrusion by updating images covering the entire line in real time. Once a danger is detected, an alarm should be triggered and the train should be brought under emergency control to a safe state.

5. The method for train control and automatic hazard avoidance using visual and radar imaging monitoring along the track as described in claims 1 and 4, characterized in that: The vehicle-to-ground communication only transmits conventional train operation control data exchange between the vehicle and the ground. The new transponder system 8 provides point-to-ground communication and wireless continuous vehicle-to-ground communication 5, which can be redundant. The point-to-ground communication 8 can serve as a backup for the wireless continuous vehicle-to-ground communication 5. When supporting remote train control and / or onboard display of images near the train, point-to-ground communication 8 can serve as a backup and redundancy for train-to-ground wireless communication 5 in the conventional train operation control data interaction between the train and the ground. Train-to-ground wireless communication 5 is additionally used by ground equipment 1 to obtain real-time train position, speed, and status, etc., and onboard vehicle control equipment 7 can obtain and display remote train control commands and information from sensing units in front of and near the train in real time. Point-to-ground communication 8 and wireless continuous train-to-ground communication 5 can work together. Point-to-ground communication 8 can be used for data interaction between the train and the ground in the conventional train operation control, and train-to-ground continuous wireless communication 5 can be used by ground equipment 1 to obtain real-time train position, speed, and status, etc., and onboard vehicle control equipment 7 and onboard display equipment 6 to obtain and display remote train control commands and image information in front of and near the train.

6. The method for train control and automatic hazard avoidance using visual and radar imaging monitoring along the track as described in claims 1, 4, and 5, characterized in that: The ground remote control console 11 can acquire the corresponding train position, speed, running direction, status, and movement authorization information to generate a target distance continuous speed control curve. At the same time, it can retrieve images of a certain distance in front of the train and images of the train's surroundings and rear. The operator can control the train's operation based on the speed control curve and by observing the displayed images. The train control command is sent to the train's onboard control equipment 7 through continuous vehicle-to-ground wireless communication. When foreign objects are observed intruding into the track or other abnormal situations, the ground remote operator can control the train to stop in time. The train's onboard equipment 7 can acquire line data, including basic line data, route information, and temporary speed limit information, through vehicle-to-ground communication. The train's onboard equipment 7 can acquire train movement authorization information and generate a target distance continuous speed control curve. The train can achieve automatic driving, or manual driving under speed control curve monitoring, or the train's onboard control equipment can acquire instructions from the ground for remote train control in real time and control the train's operation according to the instructions under the constraints of the continuous speed control curve.

7. The method for train control and automatic hazard avoidance using visual and radar imaging monitoring along the track as described in claim 1, characterized in that: In the ground remote control mode: The ground remote control console 11 can obtain information from the sensing unit 3 near the train based on the real-time position, direction of operation and track of the train, and form a suitable image through fusion, splicing and fitting image processing technology, or retrieve the image from the data server through the train position, and fit the image at a certain distance in front of the train in real time. It can build the images around and behind the train in real time, which can be used for ground remote control of train operation, observation of the surrounding environment, confirmation of the actions and status of related equipment, etc. It supports the construction of real-time images at a certain distance behind the train when the train moves backward, and can build the images around and in front of the train in real time. In manual driving mode: the on-board display unit 6 can obtain information from the sensing unit 3 near the train based on the real-time position, direction of travel and track of the train through vehicle-to-ground communication. It can form a suitable image through image processing technologies such as fusion and stitching fitting, or retrieve the image from the data server based on the train position. It can construct an image of a certain distance in front of the train in real time. It can construct images around and behind the train in real time to assist the driver in controlling the train operation, observing the surrounding environment, confirming the actions and status of related equipment, etc., and provide image data for the train to move backward.

8. The method for train control and automatic hazard avoidance using visual and radar imaging monitoring along the track as described in claims 1 and 4, characterized in that: The ground monitoring center 12 can periodically establish and maintain the image baseline for the entire line. The ground monitoring center can generate baseline images during track maintenance windows, which can be manually confirmed. The image baseline is reconfirmed when construction or track conditions change. Images acquired in real time during operation can be compared with baseline images to identify abnormal and dangerous situations, and alarms and instructions for fault handling are given.

9. The method for train control and automatic hazard avoidance using visual and radar imaging monitoring along the track as described in claims 1 and 4, characterized in that: For remote control of train operation, each ground train control console 11 can be responsible for the remote control of one train. Ground-based remote control of train stopping or ground-based automatic control of train stopping can send emergency braking or braking commands to the train via vehicle-to-ground communication, and the train's on-board control equipment 7 will execute the emergency braking or braking operation. Alternatively, the train can be stopped by disconnecting the power supply to the train from the ground, or the train can be stopped automatically by power failure, thus ensuring the safety of train operation. The images retrieved by the ground monitoring center 12, the ground remote control console 11, or the vehicle-mounted display device 6 can be either the driver's indoor view or a wide view outside the vehicle. The wide view outside the vehicle can be used to control the actions of train-related equipment or surrounding equipment, observe the environment, and confirm the status at key locations.

10. The method for train control and automatic hazard avoidance using visual and radar imaging monitoring along the track as described in claims 1 and 4, characterized in that: By installing onboard sensing units on the train, the status of key equipment can be monitored. Images can be sent to ground equipment via vehicle-to-ground communication. Ground control equipment can identify the status of onboard equipment. Ground equipment can judge the status of onboard equipment based on onboard image information and onboard equipment status information, thereby determining whether the equipment can operate normally. It supports power-on startup of onboard equipment and remote power-off restart after a fault. The train can be equipped with an onboard environmental perception unit. The onboard intelligent control equipment can automatically identify the status of the track ahead, power supply equipment, and the surrounding environment, as well as detect foreign object intrusion. In automatic driving mode, it can automatically control the train to slow down and stop. In manual driving or remote control mode, it can promptly remind the driver and the ground control center.

Images