Drone-based train approach control method, medium, and computer device

Abstract

The present invention relates to the field of railway safety technology and specifically provides a drone-based train approach control method, a computer-readable storage medium, and a computer device. [Solution] The drone-based train approach control method includes the steps of: ascending using a drone equipped with a camera to capture video images of the railway line; acquiring and identifying train images from the video images; determining the speed and position of the train based on the train images; and calculating the remaining time until the train arrives at a designated location based on the speed and position of the train. With such a configuration, it is expected that the approach of a train to any designated location required for construction work can be accurately determined, thereby improving safety.

Description

Technical Field

[0001] The present invention relates to the field of railway safety technologies, and specifically provides a train approach control method based on a drone, a computer-readable storage medium, and a computer device.

Background Art

[0002] In a railway system, with the continuous construction of infrastructure and the rapid expansion of the railway network, the maintenance demand for the railway system is increasing. However, the working environment of railway lines is complex. For example, the train length is long and the approaching time is continuous within 24 hours, so there are often non-negligible risks in maintenance work. To reduce risks, accurately predicting the approaching time of a train and promptly sending warning information are essential for ensuring the safety of track maintenance work.

[0003] Therefore, the prediction of the approaching time of a train usually depends on information such as the actual position where the train is currently located and its running speed. Thus, track speed detection using a camera (the running speed of a train running on a track) is currently widely applied and is an effective method. For example, the principle of track speed detection is mainly to process the image data captured by a camera using computer vision technology and the optical flow method to obtain the running speed of the train. However, a fixed camera has limitations such as a small shooting angle and visual dead angles. Therefore, the usual method is to use a radar sensor to obtain the actual position where the train is currently located. However, since a radar sensor is easily affected by the environment, it has limitations such as low operating reliability, low resolution, and limited information acquisition ability.

[0004] Therefore, the current method of sending warning information typically involves using infrared detection technology, Zigbee IoT communication technology, etc., to confirm when a train is approaching the track maintenance work area, and then broadcasting the information to workers in real time. Existing train monitoring systems mainly rely on fixed monitoring equipment, which requires the installation of appropriate ground equipment at the construction site and the design of appropriate installation strategies for different environments, weather / climate, etc. These monitoring systems are expensive, difficult to fully cover in complex environments, are susceptible to environmental influences, and have limited information acquisition capabilities.

[0005] Although data support is provided by technical means, existing train approach warning mechanisms have shortcomings in terms of information acquisition, resulting in some degree of information delay, and it is particularly difficult to obtain real-time train location information in the background or at construction sites. This can increase safety risks. Against this backdrop, a safety warning mode that "primarily relies on human intervention and supplements technical intervention" is especially important, as it can compensate for the limitations of technical means in a given scenario through human analysis and decision-making. For example, typically, lookouts (station lookouts, site lookouts) are stationed at locations such as platforms and construction areas, and when a train approaches the construction area, the station lookouts notify the site lookouts to evacuate. However, manual operation methods are prone to problems such as false alarms, delayed alarms, and leaks.

[0006] To achieve more accurate train approach warnings, this invention combines technology and computer vision to acquire target location information and integrates this location information into a location service API to obtain the distance of the train to a specified location. Then, based on the train's operating speed, the time it will take for the train to approach the specified location can be calculated. By using a drone equipped with a camera to collect images of the train, images can be taken from different altitudes and angles, achieving greater flexibility and coverage.

[0007] Conventional train approach time predictions only consider the travel time between stations, lacking accurate monitoring of the train's immediate position and arrival time within a specific section. Railway maintenance can occur in any area between stations, and accurately predicting the remaining time until the train approaches the construction area is necessary to provide real-time warnings and notify workers to evacuate. However, because the distance between stations is usually long, directly estimating the remaining time using the train's travel time between stations results in large errors, making accurate prediction difficult. Therefore, a method for predicting the time a train approaches in non-station areas is crucial. In this invention, by monitoring the train's operating speed and current position in real time based on computer vision technology, the time a train approaches any area can be accurately predicted. Furthermore, different alarm sounds / warning lights can be emitted according to the train's approach time, and combined with stepped alarm macracy, real-time warnings for approaching trains can be realized.

[0008] For example, Patent Document 1 discloses a method and system for detecting vehicle speed based on road surveillance camera footage, which includes the steps of: acquiring camera video data of a road surface to be monitored; acquiring multiple adjacent frames of image data selected by a user terminal; acquiring the center pixel point of the same vehicle in each of the two image frames and acquiring the vehicle pixel coordinate data of the said center image point; inputting the vehicle pixel coordinate data into a transformation algorithm model and outputting corresponding vehicle longitude and latitude coordinate data; and acquiring the interval time between the two image frames and calculating data such as the vehicle's speed within the interval time based on the change in the longitude and latitude coordinate data in the two image frames for the same vehicle.

[0009] However, Patent Document 1 has the following problems. (1) Because the position of the monitoring equipment that monitors the road surface is relatively fixed, it is not possible to capture images of vehicles from different altitudes / angles, resulting in a single field of view that lacks flexibility. (2) Converting from pixel coordinates to longitude and latitude coordinates requires complex geometric and perspective transformations, and methods using linear models for coordinate transformation lack accuracy. (3) Because it is often impossible to obtain clear identification marks between two image frames to determine the train's travel distance, calculating speed using only two image frames results in unstable and error-prone results. Therefore, this method is not applicable to long-distance modes of transportation such as trains.

[0010] For example, Patent Document 2 proposes a method for detecting the speed of a rail transit train based on knowledge distillation, which includes the following steps. S1: Train footage is acquired, feature points are obtained from the video frames using the Fast Fourier Transform (FFT), these feature points are fused with the image, and input into a student neural network. The student neural network includes a YOLO student network and a Transformer network. S2: The data obtained by fusing the aforementioned feature points with the image is used for feature point detection via the YOLO student network. S3: Feature points detected by the YOLO student network are input to the Transformer network as sequence data, and sequence matching information is obtained via the Transformer network. S4: The student neural network calculates and outputs the train's speed based on the sequence matching information and the video frame rate.

[0011] The student neural network is obtained by training it based on the teacher neural network, which is a fusion of a YOLO teacher network and a Transformer network, with a self-attention mechanism added to the YOLO teacher network to enhance optical flow information between frames, and the teacher neural network is trained using a multi-scene dataset to improve generalization ability, the Transformer network is frozen based on the teacher neural network, and the YOLO teacher network is simplified into a YOLO student network to obtain the student neural network. The student neural network is trained using a single-scene dataset, and knowledge from the teacher neural network is transferred to the student neural network based on knowledge distillation.

[0012] However, Patent Document 2 has the following problems. (1) The video frame image is processed directly using FFT to extract feature points, and because it does not take into account that the image was captured while the camera was zooming, its range of applications is limited. (2) The accuracy of the calculated speed is low because the calculation is performed solely using the optical flow method, without predicting the speed section of the train by combining it with railway train data and calibrating the calculated speed. (3) When student networks are trained on a specific dataset, overfitting may occur, which could lead to decreased performance in actual applications. (4) FFT is mainly used for frequency domain analysis and may not be able to adequately capture important time domain features in train motion, especially dynamic information such as rapid changes in the train (high-speed operation).

[0013] Thus, existing train approach time predictions only estimate the travel time between stations, resulting in insufficient accurate monitoring of the immediate position and arrival time of trains within specific sections, making it impossible to issue timely and effective warnings for specific time periods. Furthermore, conventional speed measurement methods employ fixed cameras with narrow shooting angles and blind spots, while radar sensors commonly used to acquire positional information are susceptible to environmental influences, have low resolution, and limited information acquisition capabilities. Moreover, methods that directly extract feature points without processing image distortion after identifying images using computer vision require further improvement in image accuracy. Unlike methods that predict speed sections by determining the train type and calibrate the calculated train speed in conjunction with important railway information such as railway display boards and signal status, such single speed measurement methods limit the breadth and depth of information utilization, thus requiring further improvement in the accuracy of speed measurement. [Prior art documents] [Patent Documents]

[0014] [Patent Document 1] Chinese Patent No. CN115050193A [Patent Document 2] Chinese Patent No. CN118072227A [Overview of the project] [Problems that the invention aims to solve]

[0015] The object of the present invention is to solve the above problems to at least some extent, and / or solve at least some of the above problems, specifically to reliably determine when a train is approaching any designated location where there are construction requirements, thereby improving safety. [Means for solving the problem]

[0016] In a first aspect, the present invention provides a train approach control method based on a drone, including the steps of: taking a video image along a railway using a drone equipped with a camera; obtaining and identifying a train image from the video image; determining the speed and position of the train based on the train image; and calculating the remaining time for the train to approach a designated position based on the speed and position of the train.

[0017] With such a configuration, it is possible to accurately grasp the remaining time for the train to approach an arbitrary designated position, and an improvement in the safety along the railway is expected.

[0018] Regarding the train approach control method based on the drone, in a possible embodiment, the step of obtaining the train image in the video image includes preprocessing the video image information transmitted to a ground computer, identifying and framing the video image information including the train image from the preprocessed video image information, and postprocessing the video image information including the train image.

[0019] Regarding the train approach control method based on the drone, in a possible embodiment, in the step of determining the speed and position of the train based on the train image, the method for determining the speed of the train includes extracting image feature points of the train image based on a pre-trained yolo model, determining the running distance of the train corresponding to the train image of the adjacent frame based on the image feature points of the train image of the adjacent frame, and determining the speed of the train based on the running distance of the train.

[0020] For example, by designing a reasonable coordinate transformation algorithm, the two-dimensional image coordinates of the image can be converted into three-dimensional world coordinates. Thereby, the running distance of the train is determined based on the image feature points of the train images of the adjacent frames after coordinate transformation.

[0021] Regarding the train approach control method based on the drone, in one possible embodiment, the method further includes a step of calibrating the speed of the train, and the step includes determining the type of the train based on the reprocessed video image information, and determining the speed of the train by referring to the type of the train.

[0022] Regarding the train approach control method based on the drone, in one possible embodiment, in the step of determining the speed and position of the train based on the train image, the method for determining the position of the train includes flying the drone along the railway to obtain the position coordinates along the railway line, obtaining the position identification information of the drone by the position identification device mounted on the drone, and determining the position of the train based on the position identification information of the drone.

[0023] Regarding the train approach control method based on the drone, in one possible embodiment, the step of calculating the remaining time until the train arrives at the designated position based on the speed and position of the train includes determining the remaining distance from the train to the designated position based on the position of the train and the position of the railway, and determining the remaining time until the train arrives at the designated position based on the remaining distance and the speed of the train.

[0024] Regarding the train approach control method based on the drone, in one possible embodiment, the control method further includes an alarm step of displaying the remaining time on a display device installed at the designated position and / or performing a countdown during the process of the train arriving at the designated position, so as to display and alarm the remaining time.

[0025] Regarding the train approach control method based on the drone, in one possible embodiment, the alarm step further includes a step-by-step alert alarm, and the step further includes a step-by-step alert alarm step of setting different types of alarm information according to the remaining time as the train gradually approaches the designated position.

[0026] In preferred embodiments of the present invention, the following technical effects can be achieved: By monitoring the position and speed of a train in real time, the remaining time until the train arrives at any designated location can be accurately calculated and displayed. By mounting a camera on the drone, it is possible to take pictures from different altitudes and angles, providing greater flexibility and coverage, and enabling more reliable detection of train speed. By determining the type of train using image recognition or the like and estimating the speed section of the train, and combining this with relevant railway information (including, but not limited to, railway display boards and signal status), the calculated speed can be judged and calibrated, the predicted speed of the train can be dynamically adjusted, and the accuracy of speed measurement can be improved. By applying distortion correction to video images captured by the drone, the image quality can be improved, and the accuracy and usability of the images can be enhanced. Furthermore, by combining positioning technology and computer vision to obtain the location information of a target and using a location information API to obtain the distance until the train arrives at the designated location, the cost can be reduced. Furthermore, by installing countdown displays and audible and optical alarms at the construction site, different alarm sounds and warning lights are set up according to the train's approach time, creating a tiered alarm mechanism. This provides workers with more accurate warning information and reduces / avoids the risk of construction work and train operations intersecting.

[0027] In a second embodiment, the present invention provides a computer-readable storage medium including a memory, wherein a plurality of program codes are stored in the memory, and when the program codes are loaded and executed by a processor, a drone-based train approach control method in any of the above embodiments is realized.

[0028] This computer-readable storage medium is understood to possess all the technical effects of the drone-based train approach control method described above, and therefore its explanation is omitted here.

[0029] Those skilled in the art will understand that implementing all or part of the flow in the drone-based train approach control method of the present invention can be achieved by instructing the relevant hardware to execute a computer program, which is stored on a computer-readable storage medium and executed by a processor, to implement the steps of each embodiment of the method described above. The computer program includes, but is not limited to, the computer program code for executing the drone-based train approach control method. For convenience of explanation, only the parts relevant to the present invention are shown. The computer program code may be in source code format, object code format, executable file format, or some intermediate format. The computer-readable storage medium may include any entity or device, medium, USB disk, removable hard disk, magnetic disk, optical disk, computer memory, read-only memory, random access memory, electrical carrier signals, telecommunication signals, software distribution media, etc., capable of storing the computer program code. The contents of the computer-readable storage medium may be increased or decreased as appropriate according to the requirements of law and patent practice; for example, in some jurisdictions, the computer-readable storage medium does not include electrical carrier signals and telecommunication signals.

[0030] In a third embodiment, the present invention provides a computer device including a memory and a processor, wherein a plurality of program codes are stored in the memory, and the program codes are loaded and executed by the processor to realize a drone-based train approach control method in any of the above embodiments.

[0031] It is understood that this device possesses all the technical effects of the drone-based train approach control methods described above, and therefore, a detailed explanation is omitted here. This device may also be a computer control unit composed of various electronic components.

[0032] The computer device may include a processor, memory, an input / output interface, a communication interface, a display unit, and an input device. The processor, memory, and input / output interface are connected via a system bus, and the communication interface, display unit, and input device are connected to the system bus via the input / output interface. The processor is used to provide calculation and control functions. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system and computer programs. The internal memory provides an environment for executing the operating system and computer programs stored in the non-volatile storage medium. The input / output interface is used to exchange information between the processor and external devices. The communication interface is used to communicate with external terminals by wired or wireless means, and the wireless method can be implemented by Wi-Fi, a cellular network, NFC (Near Field Communication), or other technology. When the computer program is executed by the processor, it implements a thermal management control method for a power battery. The display unit is used to form a visually visible screen and may be a display, a projection device, or a virtual reality imaging device. The display may be a liquid crystal display or an E Ink display, or it may be a key, trackball, or touchpad provided on the casing of the computer device, or it may be an externally connected keyboard, touchpad, or mouse, etc. [Brief explanation of the drawing]

[0033] Preferred embodiments of the present invention will be described below with reference to the drawings. [Figure 1] This figure shows a flow diagram of a drone-based train approach control method according to one embodiment of the present invention. [Figure 2] This figure shows the flow of the image processing step in a drone-based train approach control method according to one embodiment of the present invention. [Figure 3]This figure shows the flow of the train speed detection step in a drone-based train approach control method according to one embodiment of the present invention. [Figure 4] This figure shows the flow of the coordinate transformation step in a drone-based train approach control method according to one embodiment of the present invention. [Figure 5] This figure shows the flow of the train position identification step in a drone-based train approach control method according to one embodiment of the present invention. [Modes for carrying out the invention]

[0034] Preferred embodiments of the present invention will be described below with reference to the drawings. Those skilled in the art should understand that these embodiments are used solely to illustrate the technical principles of the present invention and are not intended to limit the scope of protection of the present invention.

[0035] In the description of this invention, terms indicating direction or positional relationships, such as "center," "up," "down," "left," "right," "vertical," "horizontal," "internal," and "external," are based on the directions or positional relationships shown in the drawings. These terms are merely for explanatory convenience and do not indicate or imply that the device or element has a fixed orientation or must be configured and operate in a fixed orientation. Therefore, they cannot be interpreted as limiting the invention. Furthermore, the terms "first" and "second" are used solely to describe the purpose and cannot be interpreted as indicating or implying relative importance.

[0036] Furthermore, in the description of the present invention, unless otherwise explicitly defined and limited, the terms “installation,” “arrangement,” and “connection” should be understood in a broad sense, for example, a fixed connection, a detachable connection, an integral connection, a direct connection, an indirect connection via an intermediate medium, or a connection between the interiors of two elements. Those skilled in the art will be able to understand the specific meaning of the above terms in the present invention depending on the context.

[0037] Furthermore, while many specific details are described in the following specific embodiments to better illustrate the present invention, those skilled in the art should understand that the present invention can be similarly implemented without these specific details. In order to clarify the spirit of the present invention, some principles and the like that are well known to those skilled in the art are not described in detail in some embodiments.

[0038] The present invention will be described below with reference to at least parts of Figures 1 to 5.

[0039] As shown in Figure 1, in one possible embodiment, a drone-based train approach control method mainly includes the following steps. S100: A drone equipped with a camera is used to ascend and capture video images of the railway line. S200: The train image is obtained from the aforementioned video image and identified. S300: Determine the speed and position of the train based on the aforementioned train image. S400: Based on the speed and position of the train, calculate the remaining time until the train arrives at the designated location.

[0040] In one possible embodiment, S100 includes using a camera-equipped drone to ascend and capture video images of the railway line, converting them into a further processable digital format (digital signal), transmitting them to a ground computer in an appropriate signal frequency band, and then converting the digital signal into corresponding video image information, which is then stored, identified, and processed.

[0041] Referring mainly to Figure 2, in one possible embodiment, S200 includes the following steps: S210: Preprocesses video image information to be transmitted to the ground computer. S220: Identify (enclose in a frame) video image information containing train images from the pre-processed video image information. S230: Reprocess video image information, including train images.

[0042] In one possible embodiment, S210 specifically includes preprocessing image information transmitted by the drone to a ground computer to remove or suppress unwanted information contained in the image, restore or enhance useful information, obtain a higher-quality image, and facilitate image recognition of the following train. Preprocessing methods include, but are not limited to, performing grayscale processing on the image, achieving noise reduction and detail enhancement through frequency domain analysis, performing normalization on the image, and adjusting the image size to ensure image quality and consistency.

[0043] In one possible embodiment, S220 specifically means, The system takes pre-processed video image data as input, performs target detection using a YOLO model, quickly detects in real time whether a train is present in the video image data, and dynamically adjusts the drone's actions and the computer's processing tasks through feedback to conserve computational resources and reduce prediction time. If a train is identified, the ground computer will signal the drone to reprocess the image to obtain a more easily analyzeable image, frame the image information of the processed train's position, track the train image, and wait while the drone performs further analysis. This includes the following: if a train is not identified, the ground computer signals the drone to continue filming along the railway line, and the ground computer does not perform any additional image processing.

[0044] In one possible embodiment, S230 specifically means, After identifying the existence of video image information of the train, image enhancement is performed on the pre-processed image information, which has the train's position framed using Kalman filtering, in order to improve the quality of the image information and make it clearer, brighter, and easier to analyze. This includes performing a perspective transformation on the image information after Kalman filtering using a depth estimation model to remove perspective distortion from the image information, calculating the camera intrinsic parameter matrix and distortion coefficient using the checkerboard calibration method, and then processing the image distortion using OpenCV by combining the above parameters to obtain a reprocessed image.

[0045] In one possible embodiment, calculating the train speed based on the train image in S300 specifically includes the following steps: S310: Using a pre-trained YOLO model, image feature points are extracted from train images. For example, image feature points in train images can be automatically detected and extracted based on changes in the gray value of pixel points in video image information. S320: Based on the image feature points of the train image in the adjacent frame, the travel distance of the train corresponding to the train image in the adjacent frame is determined. S330: Determine the train's speed based on the distance traveled.

[0046] In this case, the usual method for implementing S320 is to design a coordinate transformation algorithm to convert 2D image coordinates to 3D world coordinates, and then determine the train's travel distance by transforming the coordinates of the train image in the adjacent frame.

[0047] However, directly determining the train speed without judgment or calibration may result in low accuracy. Therefore, referring to Figure 3, in one possible embodiment, the drone-based train approach control method further includes S500 for calibrating the train speed.

[0048] The S500 includes the following steps: S510: Determine the train type based on the reprocessed video image information. S510: Determine the train type based on the reprocessed video image information.

[0049] In one possible embodiment, the characteristics of the train are further analyzed based on the framed train image in the video data, and the type of train is determined by feature matching. Referring to Table 1, trains can be broadly divided into three types: high-speed trains with operating speeds of 300-350 km / h, high-speed rail trains with operating speeds of 200-250 km / h, and local trains with operating speeds of 120-160 km / h. The shape and color characteristics of the train are extracted and matched with a pre-built train feature database to determine the type of train, and the operating speed range of the train is obtained to be used as a reference for speed detection. Specifically, based on feature matching and historical data, speed estimation can quickly obtain speed reference values ​​corresponding to actual application scenarios. Furthermore, the correspondence shown in Table 1 is merely illustrative, and those skilled in the art can determine the mapping relationship between train type and train video data according to actual needs, adapt the present invention to determine multiple types of trains running on multiple lines, and improve the universality of type determination.

[0050] [Table 1]

[0051] In one possible embodiment, the method for determining the speed of the train is, for example, specifically, First, the train image is subjected to a coordinate transformation, and combined with the drone's position information, the pixel coordinates are converted to the actual coordinates on the road. Here, pixel coordinates refer to the position of a pixel point in an image. Then, feature points are extracted from the image, train images within adjacent frames of the video image information are tracked to determine the train's distance traveled, the train's speed is calculated based on this analysis, and an estimated value of the train's speed is obtained by averaging all instantaneous speeds within, for example, 1 second. This includes determining the train's speed range again based on the obtained train speed range, combining it with identified railway information (for example, information such as speed ranges on railway information boards), determining whether the calculated speed is within this range, and calibrating the train's speed by combining it with railway information (for example, the status of signals).

[0052] In one possible embodiment, calculating the position of the train based on the train image in S300 specifically includes the following steps: S340: Fly the drone along the railway line and obtain the position coordinates of the area along the railway. S350: The drone's location information is acquired by a positioning device mounted on the drone. S360: Determine the train's position based on drone location information.

[0053] In one possible embodiment, S400 includes the following steps: S401: Based on the train's position and the railway's position information, the remaining distance to the designated location is determined. S402: The remaining time until the train arrives at the designated position is determined based on the remaining distance and the speed of the train.

[0054] Furthermore, when performing coordinate transformations on train images, it is necessary to convert the coordinates of the train images from pixel coordinate system coordinates to world coordinate system coordinates. Referring to Figure 4, the coordinate transformation process mainly includes the following:

[0055] 1. Pixel Coordinate System → Camera Coordinate System: Because there is a scaling and translation relationship between the camera coordinate system and the pixel coordinate system, if the coordinates of the train image in the camera coordinate system are (x1, y1, z1) and the pixel coordinates of the train image are (u, v), then the pixel coordinate system can be transformed to the camera coordinate system according to the following equation (1) using the camera intrinsic parameter matrix K and the image depth H. Here, in the camera coordinate system, the optical center of the camera is the origin, the x and y axes are parallel to the X and Y axes of the train image, respectively, and the z axis is the camera optical axis. The camera intrinsic parameter matrix is ​​a mathematical model for transforming 3D camera coordinates to 2D homogeneous image coordinates, and the transformation from pixel coordinates to camera coordinates is achieved by multiplying by the inverse matrix of the camera intrinsic parameter matrix. Here, the image depth represents the distance information from the camera to objects in the image and is used to process the perspective transformation of the image. JPEG2026106433000003.jpg2773

[0056] 2. Camera Coordinate System → Drone Coordinate System: Although there is no translation relationship to convert the camera coordinate system to the drone coordinate system, the camera coordinate system satisfies the right-hand rule and the drone coordinate system satisfies the left-hand rule. Therefore, if the coordinates of the train in the drone coordinate system are (x2, y2, z2), the camera coordinate system can be converted to the drone coordinate system according to equation (2) below. Here, in the drone coordinate system, the center of gravity of the drone is the origin, the x-axis is the direction of the nose, the y-axis is the direction of the wings, and the z-axis is vertically downward. JPEG2026106433000004.jpg2578

[0057] 3. Drone Coordinate System → World Coordinate System: As the drone flies, the difference in error between the drone coordinate system and the world coordinate system is due to the drone's attitude. For example, the drone's attitude information R and its coordinates (x0, y0, z0) in the world coordinate system can be obtained using a visual odometer. If the coordinates of the train in the world coordinate system are (x3, y3, z3), then the drone coordinate system can be converted to the world coordinate system according to equation (3) below. Here, typically, the x-axis of the world coordinate system points north, the y-axis points east, and the z-axis points towards the ground. JPEG2026106433000005.jpg2581

[0058] In this way, based on the NED coordinates (coordinates in the world coordinate system) of the train obtained by the above coordinate transformation, the longitude and latitude coordinates of the train are obtained by matching the linear correspondence between the drone's longitude and latitude / altitude information and the drone's NED coordinates. Referring to Figure 5, the railway position information and the train position information are integrated and processed in a position information API, and the actual distance between two points with known longitude and latitude can be calculated using the geodesic method or similar. For example, by sequentially processing the railway position information (each designated position) collected in advance using this method, and setting the construction area as the starting point of the calculation and the train as the end point of the calculation, the remaining distance of the train to each designated position can be obtained.

[0059] Based on the remaining distance to the designated location, the train's speed is adjusted, and the remaining time for the train to arrive at the designated location is calculated based on a pre-built train approach time prediction model, using image information from multiple frames and speed information from multiple trains.

[0060] In one possible embodiment, the drone-based train approach control method further includes an alarm step S600.

[0061] The S600 includes the following steps: S610: Displays the remaining time, issues an alarm, and performs a countdown. Specifically, based on the remaining time until the train arrives at the designated location, as determined by S400, the remaining time can be displayed in real time on a display device such as a countdown display.

[0062] In dark environments, workers may not be able to clearly see the information on the countdown display; therefore, in one possible embodiment, S600 further includes the following steps: S620: Staged warning system. Specifically, as the train approaches a designated location, different types of warning information (e.g., distinguishable alarm sounds and / or warning lights) are set according to the remaining time. This effectively alerts workers to take appropriate safety measures as the train approaches, and enables precise warnings.

[0063] Furthermore, a multi-stage warning mechanism such as multi-voice recognition / multi-color recognition alarms can be used. For example, by using an audible and optical alarm, different alarm sounds and different colored warning lights can be set according to the approach time, providing intuitive warning information so that workers can accurately grasp the dynamic information of the train even when the countdown display screen is not visible.

[0064] Furthermore, if workers are scattered across different locations in the construction area, they may not be able to see the countdown display for approaching trains and therefore may not be able to respond in a timely manner. Thus, in one possible embodiment, S600 further includes the following steps. As S630, it transmits alarm information to one or more related terminals.

[0065] In this way, a terminal device and program module compatible with the train approach countdown system according to the present invention can be placed at the construction site. The terminal device displays a real-time countdown of the remaining time until the train arrives at the construction area, so workers can accurately grasp the dynamic information of the train, prepare for avoidance in advance, and ensure safety, no matter where they are, by connecting to the system terminal device.

[0066] For example, based on the alarm mechanism described above, if a train appears in the monitoring area and is about to pass through the construction area, the alarm will automatically activate and emit an alarm, with the alarm sound changing according to the time of the train's approach. This eliminates the need for workers to monitor the train's approach countdown on a display in real time, allowing them to take appropriate action according to the train's approach.

[0067] Thus, it can be seen that preferred embodiments of the present invention have the following advantages.

[0068] (1) By using drones equipped with high-definition cameras to monitor train positions and speeds in real time, and ensuring comprehensive and accurate monitoring through the drones' flexibility and wide field of view coverage, it can flexibly adapt to various railway environments and capture images and videos of the trackside in real time.

[0069] (2) By applying distortion correction to video images captured by a camera mounted on a drone, image quality can be improved, enhancing the accuracy and usability of the images.

[0070] (3) By referring to train types and related railway information, the accuracy of train speed predictions can be further improved. Based on the identified train types and railway information such as the status of railway signs / signals, the meaning of different railway information such as the status of signs / signals is determined, and a rough range of train operating speeds is predicted. This provides a reasonable and referential initial value for speed estimation, and then the calculated speed is judged and corrected, the speed prediction range is subdivided, the accuracy of speed calculations is improved, and it is suitable for actual application scenarios and possesses universality.

[0071] (4) The alarm system for approaching trains does not rely on fixed on-board or ground equipment such as infrared or lasers, and the alarm mechanism avoids relying on existing hardware, simplifying equipment placement and maintenance.

[0072] (5) By using drones to collect video image data and processing the data close to the data source, intelligent analysis of the video content can be achieved. Real-time processing is performed by edge computing devices, i.e., computers located close to the data source, preventing problems such as network delay, network congestion, and degraded service quality caused by the distance between the terminal device (drone) and the server (ground computer).

[0073] (6) By combining drone positioning technology with computer vision technology, it becomes possible to eliminate reliance on radar sensors and other devices, preventing susceptibility to environmental influences, low resolution, and low prediction accuracy due to limited information acquisition capabilities.

[0074] (7) A countdown system is employed to issue advance warnings of train arrivals, alerting workers and ensuring safety. Preferably, different alarm sounds and different colored warning lights are installed according to the approach time, allowing for precise evacuation instructions and realizing an automated, step-by-step warning and response mechanism.

[0075] In this way, in a preferred embodiment of the present invention, a drone is used to capture video images along a railway line. Due to the high flexibility of the drone, it is possible to capture video images along the railway line even in complex terrain, overcoming the limitations of conventional fixed cameras, which have a limited field of view and are difficult to adjust, thus ensuring comprehensive and flexible monitoring range. By performing image processing and analysis using computing vision technology, a large amount of video / image data is processed at high speed, while achieving higher accuracy and precision. At the same time, by performing image processing operations such as distortion correction and noise reduction on images captured by the camera using the camera's intrinsic parameter matrix and distortion coefficient, the accuracy of image recognition is effectively improved. Furthermore, by determining the train's operating speed section in conjunction with important railway information such as train type and the status of railway signs and signals, and by performing judgment calibration on the calculated train speed, the accuracy of the determined train speed is improved. In addition, by combining positioning technology and computer vision technology, the actual position of the train can be obtained from the drone's position, improving prediction accuracy.

[0076] In the embodiments described above, each step was explained in a fixed order. However, those skilled in the art should understand that, in order to achieve the effects of the present invention, it is not necessary to perform the different steps in this order; they may be performed simultaneously, in other orders, or specific steps may be added, substituted, or omitted.

[0077] While the above-described train approach control method based on a drone is explained as an example, those skilled in the art will understand that the present invention is not limited thereto. In fact, users can flexibly adjust the relevant steps and elements such as parameters within those steps according to the actual application scenario and other circumstances.

[0078] The above description of the technical solutions of the present invention has been based on preferred embodiments shown in the drawings, but it will be readily apparent to those skilled in the art that the scope of protection of the present invention is not limited to these specific embodiments. Those skilled in the art can make equivalent modifications or substitutions to features of related technologies without departing from the principles of the present invention, and all such modified or substituted technical solutions are within the scope of protection of the present invention.

Claims

1. The steps include raising a camera-equipped drone to capture video images of the railway line, The steps include: acquiring and identifying a train image from the aforementioned video image, The steps include determining the speed and position of the train based on the aforementioned train image, A drone-based train approach control method, comprising the step of calculating the remaining time until the train arrives at a designated location based on the speed and position of the aforementioned train.

2. The step of acquiring and identifying a train image from the video image is: Preprocessing video image information transmitted to a ground computer, Identifying video image information, including train images, from pre-processed video image information, A drone-based train approach control method according to claim 1, comprising reprocessing video image information including train images.

3. In determining the speed and position of a train based on the aforementioned train image, the method for determining the speed of the train is as follows: Using a pre-trained YOLO model, we extract image feature points from train images, Based on the image feature points of the train image in the adjacent frame, the distance traveled by the train corresponding to the train image in the adjacent frame is determined, A drone-based train approach control method according to claim 1, comprising determining the speed of a train based on the distance traveled by the train.

4. The control method further includes a step of calibrating the speed of the train, the step of which is Based on the reprocessed video image information, the type of train will be determined, A drone-based train approach control method according to claim 3, comprising determining the type of train based on reprocessed video image information.

5. In determining the speed and position of the train based on the aforementioned train image, the method for determining the train's position is: The steps include: flying a drone along the railway line and obtaining the position coordinates of the area along the railway line; The process involves obtaining the drone's location information using a location tracking device mounted on the drone, and A drone-based train approach control method according to claim 1, comprising the step of determining the position of a train based on drone positioning information.

6. Calculating the remaining time until the train arrives at the designated location based on the speed and position of the aforementioned train is: The remaining distance from the train to the designated location is determined based on the train's position and the railway's position, A drone-based train approach control method according to claim 1, comprising determining the remaining time until the train arrives at a designated position based on the remaining distance and the speed of the train.

7. The control method described above is A drone-based train approach control method according to claim 1, further comprising an alarm step of displaying the remaining time until the train arrives at a designated location on a display device installed at the designated location, and / or displaying the remaining time and providing an alarm so as the train arrives at the designated location, a countdown is performed.

8. The warning step further includes a stepwise alert warning step that sets different types of warning information according to the remaining time as the train gradually approaches a designated position, the drone-based train approach control method according to claim 7.

9. A computer-readable storage medium including memory, wherein a plurality of program codes are stored in the memory, and when the program codes are loaded and executed by a processor, a train approach control method based on a drone according to any one of claims 1 to 8 is realized.

10. A computer device comprising memory and a processor, wherein the memory stores a plurality of program codes, and when the program codes are loaded and executed by the processor, it realizes a train approach control method based on a drone according to any one of claims 1 to 8.

Images