Train approaching control method based on unmanned aerial vehicle, medium and computer equipment

Abstract

The embodiment of the invention relates to the technical field of railway safety, and particularly provides a train approaching control method based on an unmanned aerial vehicle, a computer readable storage medium and computer equipment, and the train approaching control method based on the unmanned aerial vehicle comprises the following steps: adopting the unmanned aerial vehicle carrying a camera to lift off to shoot video images along a railway; acquiring and identifying a train image in the video image; determining the speed and the position of the train according to the train image; and according to the speed and the position of the train, the remaining time of the train arriving at the designated position is calculated. With this configuration, it is possible to accurately determine the remaining time for a train to arrive at any designated position having construction requirements, and it is hopeful to improve the safety along the railway.

Description

Technical Field

[0001] The present invention relates to the field of railway safety technology, and specifically provides a train approach control method based on an unmanned aerial vehicle, a computer-readable storage medium, and a computer device. Background Art

[0002] With the continuous construction of infrastructure in the railway system and the rapid expansion of the railway network, the demand for maintenance of the railway system is growing. However, due to the complex construction environment of railway lines, such as the long length of trains and the uninterrupted approach time within 24 hours, there are often risks in construction and maintenance that cannot be ignored. In order to reduce the risk, accurately predicting the approach time of trains and providing timely warning information are crucial to ensure the safety of track maintenance operations.

[0003] Among them, the prediction of the approach time of the train usually depends on the actual position and running speed of the train. Among them, the use of cameras for track speed measurement (the running speed of the train on the track) is currently a widely used and effective method. For example, the principle of track speed measurement is mainly to obtain the running speed of the train by processing the image data captured by the camera through computer vision technology and optical flow method. However, fixed cameras usually have limitations such as small shooting angles and visual blind spots. Among them, radar sensors are usually used to obtain the actual current position of the train. However, the working reliability of radar sensors is easily affected by the environment, and they have limitations such as low resolution and limited information acquisition capabilities.

[0004] Among them, the current strategy for giving early warning information is usually: when the train approaches the construction area of ​​track maintenance operations, infrared detection technology, Zigbee Internet of Things communication technology, etc. are used to determine the information of the train approaching the construction area, and the relevant real-time broadcast is given to the construction personnel. Most of the existing train monitoring systems rely on fixed monitoring equipment. Accordingly, corresponding ground devices are often required to be installed at the construction site, and different installation strategies are designed for different environments, weather / climates, etc. These monitoring devices are often expensive, and in some complex environments, they are difficult to fully cover, are easily affected by the environment, and have limited information acquisition capabilities.

[0005] Although technical means provide data support, the existing train approach alarm mechanism is insufficient in information acquisition, especially in the background or construction site, it is difficult to obtain the location information of the train in real time, and there is a problem of information lag to a certain extent. Information lag will not only affect the safety monitoring of the train, but may also increase safety hazards. In this context, it is particularly important to adopt a safety warning mode of "human defense as the main and technical defense as the auxiliary", such as human analysis and decision-making to make up for the limitations of technical means in the scene. For example, guards (station guards, on-site guards) are usually deployed at platforms, construction areas and other locations. When the train approaches the construction area, the station guard notifies the on-site guard to give construction evacuation instructions. However, manual operation is prone to problems such as false alarms, delayed reports and missed reports.

[0006] In order to achieve more accurate train approach alarm, the present invention combines positioning technology with computer vision to obtain the location information of the target, and integrates the location information into the location service API to obtain the distance of the train from the specified location. The time when the train approaches the specified location can be calculated in combination with the running speed of the train. The image acquisition of the train is carried out by using a drone equipped with a camera, which can be shot from different heights and angles, thereby achieving better flexibility and coverage.

[0007] Traditional train approach time prediction only estimates the travel time between stations, but lacks accurate monitoring of the train's instant position and arrival time in a specific section. However, railway maintenance may occur in any area between stations. In order to achieve real-time warning and remind construction personnel to evacuate in time, it is necessary to accurately predict the remaining time for the train to arrive at the construction area. However, since the distance between stations is usually far, directly using the train travel time between stations to estimate the remaining time will result in large errors and it is difficult to achieve accurate prediction. Therefore, it is particularly important to explore the prediction method of the train approach time in non-station areas. In the present invention, based on computer vision technology, the running speed of the train and the current position of the train are monitored in real time, and the time for the train to approach any position is accurately predicted. On this basis, different alarm sounds / warning lights can be issued according to the different approaching times of the train, and a staged warning mechanism can be combined to achieve real-time warning for the approaching train.

[0008] For example, the Chinese invention patent application (CN115050193A) discloses a vehicle speed measurement method and system based on road monitoring video images, wherein the method includes the following steps: obtaining video image data of the road surface to be monitored; obtaining several frames of video image data selected by the user terminal; obtaining the center pixel points of the same vehicle in two frames of images respectively, and obtaining the vehicle pixel coordinate data of the center pixel points; inputting the vehicle pixel coordinate data into the conversion algorithm model and outputting the corresponding vehicle longitude and latitude coordinate data; obtaining the interval between the two frames of images, and calculating the speed data of the vehicle within the interval based on the changes in the longitude and latitude coordinate data of the same vehicle in the two frames of images. However, in this document:

[0009] (1) Since the position of the monitoring equipment used to monitor the road surface is relatively fixed, it is impossible to shoot videos of vehicles from different heights / angles, and the viewing angle is single and lacks flexibility.

[0010] (2) The conversion from pixel coordinates to longitude and latitude coordinates involves complex geometric transformations, perspective transformations, etc. The accuracy of coordinate conversion through linear models needs to be further improved.

[0011] (3) Since it is often impossible to obtain clear identification marks between two frames of images to determine the travel distance of the train, the method of calculating the speed only through two frames of images has defects such as unstable results and large errors. In fact, this solution is not suitable for long transportation tools such as trains.

[0012] For example, the Chinese invention patent application (CN118072227A) proposes a rail transit train speed measurement method based on knowledge distillation, which includes the following steps: Step S1: Collect train travel video, obtain feature points from the video frame image through the FFT algorithm, and input the feature points and image fusion into the student neural network; the student neural network includes the yolo student network and the Transformer network; Step S2: The feature point and image fusion input data in step S1 are detected by the yolo student network; Step S3: The feature points detected by the yolo student network in step S2 are input into the Transformer network as sequence data, and the sequence matching information is obtained through the Transformer network; Step S4: The student neural network calculates the speed of the output train based on the sequence matching information obtained in step S3 and the video frame rate; the student neural network is trained based on the teacher neural network; the teacher neural network integrates the yolo teacher network and the Transformer network, and the yolo teacher network has a self-attention mechanism added to strengthen the optical flow information between the captured frames; the teacher neural network is trained with multi-scene data sets to improve generalization ability; on the basis of the teacher neural network, the Transformer network is frozen and the yolo teacher network is simplified to the yolo student network to obtain the student neural network; the student neural network is trained with a single scene data set, and the knowledge of the teacher neural network is transferred to the student neural network based on knowledge distillation. However, in this document:

[0013] (1) The FFT algorithm is directly used to process the video frame image to extract feature points, without considering the distortion problem of the video image taken when the camera is zoomed, thus limiting its application scope.

[0014] (2) The train speed is calculated only by the optical flow method, without combining the railway train data to preliminarily predict the train speed range and calibrate the calculated speed, resulting in low speed accuracy.

[0015] (3) The student network may overfit when trained on a specific dataset, which may lead to poor performance in practical applications.

[0016] (4) The FFT algorithm is mainly used for frequency domain analysis and may not be able to fully capture the important time domain characteristics of train motion, especially the dynamic information of trains that are changing rapidly (running at high speed).

[0017] It can be seen that since the current prediction of train approach time only estimates the travel time between stations, there is a lack of accurate monitoring of the train's instant position and arrival time in a specific section, and it is impossible to make timely and effective warnings for specific time periods. In addition, the fixed camera used in the traditional speed measurement method has a small shooting angle and a visual blind spot. The radar sensor commonly used to obtain the listed location information is easily affected by the environment, has low resolution and limited information acquisition capabilities. In addition, the accuracy of the image needs to be further improved in the method of directly extracting feature points without processing the image distortion after computer vision recognition of the image. The single speed measurement method that does not "judge the train type to predict the train speed range, and combine the key railway information such as railway signs and signal light status to judge and calibrate the calculated speed" limits the breadth and depth of information utilization, and the speed measurement accuracy needs to be further improved. Summary of the invention

[0018] The present invention aims to solve the above technical problems and / or solve at least part of the above technical problems to at least a certain extent, specifically, how to reliably determine that a train is approaching any designated location with construction requirements to improve safety.

[0019] In a first aspect, the present invention provides a method for controlling a train approach based on a drone, the method comprising the following steps: using a drone equipped with a camera to take off and shoot video images along the railway; acquiring and identifying train images in the video images; determining the speed and position of the train based on the train images; and calculating the remaining time for the train to arrive at a designated location based on the speed and position of the train.

[0020] This structure will make it possible to accurately determine the remaining time for a train to arrive at any specified location, thereby improving safety along the railway.

[0021] For the above-mentioned UAV-based train approach control method, in a possible implementation manner, the step of "obtaining a train image in the video image" includes: preprocessing the video image information transmitted to the ground computer; identifying (such as framing) the video image information containing the train image from the preprocessed video image information; and reprocessing the video image information containing the train image.

[0022] For the above-mentioned UAV-based train approach control method, in a possible implementation manner, in the “determining the speed and position of the train based on the train image”, the method for determining the speed of the train includes: extracting image feature points in the train image through a pre-trained YOLO model; determining the travel distance of the train corresponding to the train image of adjacent frames based on the image feature points of the train images of adjacent frames; and determining the speed of the train based on the travel distance of the train.

[0023] For example, the two-dimensional image coordinates of the image can be converted into three-dimensional world coordinates by designing a reasonable coordinate conversion algorithm. On this basis, the travel distance of the train can be determined according to the image feature points of the train images of adjacent frames after the coordinate conversion.

[0024] For the above-mentioned UAV-based train approach control method, in a possible implementation, the control method also includes the following steps: calibrating the speed of the train, which step includes: determining the type of the train based on the reprocessed video image information; and determining the speed of the train with reference to the type of the train.

[0025] With respect to the above-mentioned UAV-based train approach control method, in a possible implementation manner, in the “determining the speed and position of the train based on the train image”, the method for determining the position of the train includes: making the UAV fly along the railway to obtain the position coordinates along the railway; obtaining the positioning information of the UAV through a positioning device carried by the UAV; and determining the position of the train based on the positioning information of the UAV.

[0026] For the above-mentioned UAV-based train approach control method, in a possible implementation manner, the "calculating the remaining time for the train to arrive at the designated location based on the speed and position of the train" includes: determining the remaining distance of the train from the designated location based on the position of the train and the position of the railway; determining the remaining time for the train to arrive at the designated location based on the remaining distance and the speed of the train.

[0027] For the above-mentioned UAV-based train approach control method, in a possible implementation, the control method also includes: an early warning step, which includes: a remaining time display early warning, which is used to: display the remaining time for the train to arrive at the designated location on a display device at the designated location; and / or count down during the process of the train arriving at the designated location.

[0028] For the above-mentioned UAV-based train approach control method, in a possible implementation, the warning step also includes: a segmented reminder warning, which is used to: as the train gradually approaches the designated position, different types of reminder information are set according to the remaining time.

[0029] Technical effects:

[0030] In a preferred embodiment of the present invention, by monitoring the position and speed of the train in real time, the remaining time for the train to arrive at any designated location is accurately calculated and displayed. By using a drone equipped with a camera, shooting can be performed from different heights and angles, with better flexibility and coverage, so as to more reliably monitor the speed of the train. The type of train is judged by image recognition and other methods to predict the speed range of the train, and the calculated speed is judged and calibrated in combination with relevant railway information (including but not limited to railway notice boards, signal light status, etc.), and the predicted speed of the train is dynamically adjusted to improve the speed measurement accuracy. By performing distortion correction on the video images taken by the drone, the image quality is enhanced, and the accuracy and availability of the image are improved. In addition, the positioning technology is combined with computer vision to obtain the location information of the target, and then the distance of the train to the designated location is obtained through the location service API, which is lower in cost. In addition, through the countdown display screen and sound and light alarm configured at the construction site, different alarm sounds / warning lights are set according to different train approach times, a phased early warning mechanism is implemented, and more targeted early warning information is provided to the staff, thereby reducing / avoiding the risk of construction and maintenance operations intersecting with train operations.

[0031] In a second aspect, the present invention provides a computer-readable storage medium, which includes a memory, wherein the memory is suitable for storing multiple program codes, and the program codes are suitable for being loaded and run by a processor to execute any of the aforementioned UAV-based train approach control methods.

[0032] It can be understood that the computer-readable storage medium has all the technical effects of the UAV-based train approach control method described in any of the above-mentioned items, which will not be repeated here.

[0033] It is understood by those skilled in the art that the present invention implements all or part of the processes in its approach control method for trains based on drones, which can be completed by instructing related hardware through a computer program, and the computer program can be stored in a computer-readable storage medium. When the computer program is executed by a processor, the steps of each of the above-mentioned method embodiments can be implemented. Among them, the computer program includes computer program code, and it can be understood that the program code includes but is not limited to the program code for executing the above-mentioned approach control method for trains based on drones. For the convenience of explanation, only the parts related to the present invention are shown. The computer program code can be in source code form, object code form, executable file or some intermediate form, etc. The computer-readable storage medium may include: any entity or device, medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory, random access memory, electric carrier signal, telecommunication signal and software distribution medium, etc. that can carry the computer program code. It should be noted that the content contained in the computer-readable storage medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable storage media do not include electric carrier signals and telecommunication signals.

[0034] In a third aspect, the present invention provides a computer device comprising a memory and a processor, wherein the memory is suitable for storing a plurality of program codes, and the program codes are suitable for being loaded and run by the processor to execute any of the aforementioned UAV-based train approach control methods.

[0035] It is understandable that the device has all the technical effects of the train approach control method based on drone described in any of the above items, which will not be repeated here. The device can be a computer-controlled device formed by various electronic devices.

[0036] The computer device may include a processor, a memory, an input / output interface, a communication interface, a display unit and an input device. The processor, the memory and the input / output interface are connected via a system bus, and the communication interface, the display unit and the input device are connected to the system bus via the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and the external device. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be implemented through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, a thermal management control method for a power battery is implemented. The display unit of the computer device is used to form a visually visible picture, which can be a display screen, a projection device or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad set on the computer device shell, or an external keyboard, touchpad or mouse. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] The preferred embodiments of the present invention are described below with reference to the accompanying drawings, in which:

[0038] Figure 1 A flow chart of a train approach control method based on a drone according to an embodiment of the present invention is shown.

[0039] Figure 2 A schematic diagram showing the flow of image processing in a train approach control method based on a drone according to an embodiment of the present invention;

[0040] Figure 3 A schematic diagram showing a flow chart of train speed monitoring in a train approach control method based on a drone according to an embodiment of the present invention;

[0041] Figure 4 A schematic diagram showing a flow chart of coordinate conversion in a train approach control method based on a drone according to an embodiment of the present invention;

[0042] Figure 5 A schematic diagram of the flow of train position monitoring in a train approach control method based on a drone according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0043] The preferred embodiments of the present invention are described below with reference to the accompanying drawings. It should be understood by those skilled in the art that these embodiments are only used to explain the technical principles of the present invention and are not intended to limit the protection scope of the present invention.

[0044] It should be noted that in the description of the present invention, the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer" and the like indicating directions or positional relationships are based on the directions or positional relationships shown in the drawings, which are only for the convenience of description, and do not indicate or imply that the device or element must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first" and "second" are used for descriptive purposes only, and cannot be understood as indicating or implying relative importance.

[0045] In addition, it should be noted that in the description of the present invention, unless otherwise clearly specified and limited, the terms "installation", "setting", and "connection" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a direct connection, an indirect connection through an intermediate medium, or the internal connection of two components. For those skilled in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0046] In addition, in order to better illustrate the present invention, many specific details are given in the specific embodiments below, and those skilled in the art should understand that the present invention can also be implemented without certain specific details. In some examples, principles well known to those skilled in the art are not described in detail in order to highlight the main purpose of the present invention.

[0047] Refer to the following Figures 1 to 5 The present invention is described with reference to at least a part of the present invention.

[0048] like Figure 1 As shown, in a possible implementation manner, the train approach control method based on a drone mainly includes the following steps:

[0049] S100, using a drone equipped with a camera to take off and capture video images along the railway line;

[0050] S200, acquiring and identifying a train image in the video image;

[0051] S300, determining the speed and position of the train according to the train image;

[0052] S400: Calculate the remaining time for the train to arrive at a designated location according to the speed and position of the train.

[0053] In a possible implementation, S100, i.e., the “taking a video image of a railway line by launching a drone equipped with a camera” includes:

[0054] After taking the video images of the railway line by using a drone equipped with a camera, the video images are converted into a digital format (digital signal) for further processing and transmitted to a ground computer at a suitable signal frequency band. The digital signal is then converted into corresponding video image information, which is then stored, identified and processed.

[0055] Main reference Figure 2 In a possible implementation manner, S200, namely, the step of “obtaining a train image in the video image” includes:

[0056] S210, preprocessing the video image information transmitted to the ground computer;

[0057] S220, identifying (framing out) video image information including a train image from the pre-processed video image information;

[0058] S230: reprocess the video image information including the train image.

[0059] In a possible implementation manner, S210, i.e., the “preprocessing the video image information transmitted to the ground computer” specifically includes:

[0060] By preprocessing the image information transmitted from the drone to the ground computer, irrelevant information in the image can be eliminated or weakened, useful information can be restored or enhanced, and a better and higher quality image can be obtained for subsequent train image recognition. For example, the preprocessing methods may include but are not limited to: grayscale processing of the image, noise reduction and detail enhancement through frequency domain analysis, normalization of the image, adjustment of the image size, etc. to ensure the quality and consistency of the image.

[0061] In a possible implementation manner, S220, namely, “framing the video image information including the train image from the pre-processed video image information” specifically includes:

[0062] Input the pre-processed video image information, use the YOLO model for target detection, and quickly detect whether there is a train in the video image information in real time. In order to further save computing resources and reduce prediction time, the behavior of the drone and the processing tasks of the computer are dynamically adjusted through feedback. If a train is identified, the ground computer returns a signal to the drone. This return signal is equivalent to telling the drone to wait for the ground computer to reprocess the image to obtain an image that is easier to analyze, frame the image information of the processed train position, track the train image, and perform subsequent analysis. If the train is not identified, the ground computer returns a signal to the drone. This return signal is equivalent to telling the drone to continue shooting video images along the railway, and the ground computer does not perform additional image processing.

[0063] In a possible implementation manner, S230, i.e., the “reprocessing the video image information including the train image” specifically includes:

[0064] After identifying the video image information of the train, the Kalman filter is used to enhance the pre-processed image information that frames the train position to improve the quality of the image information, making it clearer, brighter and easier to analyze. In addition, the depth estimation model is used to perform perspective transformation on the image information processed by the Kalman filter to eliminate the perspective distortion in the image information. The camera intrinsic parameter matrix and distortion coefficient are calculated by the checkerboard calibration method. The image distortion is processed using openCV combined with the above parameters to obtain the reprocessed image.

[0065] In a possible implementation, S300, i.e., in the step of “determining the speed and position of the train according to the train image”, “calculating the speed of the train according to the train image” specifically includes:

[0066] S310, using a pre-trained YOLO model to extract image feature points in the train image. For example, the image feature points in the train image can be automatically detected and extracted based on the change of the gray value of the pixel in the video image information;

[0067] S320, determining the travel distance of the train corresponding to the train image of the adjacent frame according to the image feature points of the train image of the adjacent frame;

[0068] S330: Determine the speed of the train according to the travel distance of the train.

[0069] The implementation method of S320 is usually: by designing a coordinate conversion algorithm, the two-dimensional image coordinates are converted into three-dimensional world coordinates, and the travel distance of the train is determined according to the coordinate transformation of the train images of adjacent frames.

[0070] If the train speed is determined directly without judgment or calibration, the accuracy may be low. Figure 3 In a possible implementation, the UAV-based train approach control method further includes:

[0071] S500, calibrating the speed of the train, this step includes:

[0072] S510, determining the type of the train according to the reprocessed video image information;

[0073] S520: Determine the speed of the train by referring to the type of the train.

[0074] In a possible implementation, according to the train image framed in the video image data, the features of the train are further analyzed, and the category of the train is determined by feature matching. Referring to Table 1, the train can be roughly divided into three categories: high-speed rail with an operating speed of 300-350km / h, motor train with an operating speed of 200-250km / h, and ordinary railway train with an operating speed of 120-160km / h. By extracting the shape features and color features of the train, matching with the pre-built feature database of the train, the category of the train is determined, thereby obtaining the operating speed range of the train, providing a reference for speed monitoring. Specifically, based on feature matching and historical data, the speed reference value corresponding to the real-time application scenario can be quickly obtained through speed estimation. The corresponding relationship described in Table 1 is only an exemplary description. Those skilled in the art can determine the mapping relationship between the category of the train and the video image data of the train according to actual needs, so that the present invention can adapt to determine various types of trains running on various lines and improve the universality of category determination. Table 1 Correspondence between train types, speed ranges and train characteristics

[0075] In a possible implementation, the method for determining the speed of a train specifically includes: first, coordinate conversion of the train image is performed, and the pixel coordinates are converted into actual coordinates on the road in combination with the position information of the drone. Among them, the pixel coordinates are the positions of the pixel points in the image. Then, the image feature points are extracted, and the train images in adjacent frames of the video image information are tracked to obtain the distance traveled by the train. The train speed is calculated based on this analysis. For example, by taking the average of all instantaneous speeds within 1s, an estimated value of the train speed is obtained. On the basis of the obtained speed range of the train, the speed range of the train is determined again in combination with the identified railway billboards (such as the information about the speed range on the railway billboards), and it is determined whether the calculated speed is within this range. The speed of the train can also be judged and calibrated in combination with railway information (such as the status of signal lights, etc.).

[0076] In a possible implementation, S300, i.e., in the step of “determining the speed and position of the train according to the train image”, “calculating the position of the train according to the train image” specifically includes:

[0077] S340, flying the UAV along the railway to obtain the position coordinates along the railway;

[0078] S350, obtaining positioning information of the drone through a positioning device carried on the drone;

[0079] S360. Determine the location of the train based on the positioning information of the drone.

[0080] In a possible implementation, S400, namely, "calculating the remaining time for the train to arrive at the designated location according to the speed and position of the train" includes:

[0081] S401, determining the remaining distance of the train from the designated location according to the location information of the train and the railway;

[0082] S402: Determine the remaining time for the train to arrive at the designated location according to the remaining distance and the speed of the train.

[0083] When performing coordinate transformation on the train image, the coordinates of the train image need to be transformed from the coordinates in the pixel coordinate system to the coordinates in the world coordinate system. Figure 4 , the coordinate transformation process mainly includes:

[0084] 1. Pixel coordinate system → camera coordinate system: Since there is a scaling and translation relationship between the camera coordinate system and the pixel coordinate system, it is assumed that the coordinates of the train image in the camera coordinate system are (x1, y1, z1) and the pixel coordinates of the train image are (u, v). Then, according to the following formula (1), the pixel coordinate system can be converted to the camera coordinate system through the camera intrinsic parameter matrix K and the image depth H. Among them, the camera coordinate system takes the optical center of the camera as the origin, the x-axis and the y-axis are parallel to the X-axis and Y-axis of the train image respectively, and the z-axis is the camera optical axis. The camera intrinsic parameter matrix is ​​a mathematical model used to transform 3D camera coordinates to 2D homogeneous image coordinates. Here, the conversion from pixel coordinates to camera coordinates is achieved by multiplying the inverse of the camera intrinsic parameter matrix. The image depth represents the distance information from the camera to the object in the image, which is used to process the perspective transformation of the image.

[0085]

[0086] 2. Camera coordinate system → UAV coordinate system: There is no translation relationship when converting the camera coordinate system to the UAV coordinate system. However, the camera coordinate system satisfies the right-hand rule, and the UAV coordinate system satisfies the left-hand rule. Assuming that the coordinates of the train in the UAV coordinate system are (x2, y2, z2), the camera coordinate system can be converted to the UAV coordinate system according to the following formula (2). Among them, the UAV coordinate system takes the center of gravity of the UAV as the origin, the direction of the nose is the x-axis, the direction of the wing is the y-axis, and the vertical downward is the z-axis.

[0087]

[0088] 3. UAV coordinate system → world coordinate system: As the UAV flies, the difference between the UAV coordinate system and the world coordinate system is the UAV's posture relationship. For example, the UAV's posture information R and the coordinates of the UAV in the world coordinate system (x0, y0, z0) can be obtained through the visual odometer. Assuming that the coordinates of the train in the world coordinate system are (x3, y3, z3), the UAV coordinate system can be converted to the world coordinate system according to the following formula (3). In general, the x-axis of the world coordinate system points to the north, the y-axis points to the east, and the z-axis points to the ground.

[0089]

[0090] In this way, based on the NED coordinates of the train (coordinates in the world coordinate system) obtained by the above coordinate transformation, and combined with the linear correspondence between the longitude and latitude / altitude information of the drone and the NED coordinates of the drone, the longitude and latitude coordinates of the train are obtained. Figure 5 For example, the location information of the railway and the location information of the train can be integrated into the location service API for processing. For example, the actual distance between two points with known longitude and latitude can be calculated by using the geodesic method. For example, this method can be used to process the pre-collected railway location information (each designated location) in sequence, and the construction area is used as the calculation starting point and the train is used as the calculation end point, so as to obtain the remaining distance of the train from each designated location.

[0091] Based on the determined remaining distance of the train from the designated location and the speed of the train, the remaining time for the train to arrive at the designated location can be calculated based on multi-frame image information and speed information of multiple trains, such as based on a pre-built prediction model of the train approach time.

[0092] In a possible implementation, the control method for approaching a train based on a drone further includes:

[0093] S600, early warning step. This step includes:

[0094] S610: Display a warning and count down the remaining time.

[0095] Specifically, based on the remaining time for the train to arrive at the designated location determined according to S400, the remaining time can be displayed in real time on a display device such as a countdown display screen.

[0096] Since construction workers may not be able to clearly see the information on the countdown display screen in a dark environment, in a possible implementation, S600 also includes:

[0097] S620, segmented reminder warning.

[0098] Specifically, as the train approaches the designated location, different types of reminder information (such as distinguishable alarm sounds and / or warning lights) are set according to the remaining time. In this way, the staff can be effectively reminded to take corresponding safety measures as the train approaches, thereby achieving accurate early warning.

[0099] Therefore, a segmented warning mechanism such as multi-sound recognition / multi-color recognition warning can be adopted. For example, an audible and visual alarm can be used to set different alarm sounds and warning lights of different colors according to different approach times to provide intuitive warning information, helping construction workers to accurately understand the dynamic information of the train even if they cannot see the countdown display screen.

[0100] For example, when construction workers are scattered in different locations in the construction area, they may not be able to see the countdown display screen of the approaching train, and therefore cannot take timely countermeasures. In one possible implementation. In one possible implementation, S600 also includes:

[0101] S630: Send the warning information to one or more associated terminals.

[0102] In this way, as long as the terminal device and program module compatible with the train approach countdown system of the present invention are deployed at the construction site, the terminal device will display the countdown of the remaining time for the train to arrive at the construction area in real time, so that no matter where the construction workers are, as long as they are connected to the system terminal device, they can accurately grasp the dynamic information of the train, make avoidance preparations in advance, and ensure safety.

[0103] For example, based on the above-mentioned early warning mechanism, once a train appears in the monitoring area and is about to pass through the construction area, the alarm will be automatically triggered to sound an alarm, and the alarm sound will vary according to the approaching time of the train. For example, construction personnel can take corresponding measures according to the approaching situation of the train without having to pay real-time attention to the train approaching countdown on the display screen.

[0104] It can be seen that in the preferred embodiment of the invention, the following advantages are achieved:

[0105] 1. Use drones equipped with high-definition cameras to monitor the position and speed of trains in real time. The flexibility and wide viewing angle coverage of drones ensure the comprehensiveness and accuracy of monitoring. They can flexibly adapt to various railway environments and can capture images and videos along the track in real time.

[0106] 2. Perform distortion processing on the video images taken by the camera on the drone to enhance the image quality and improve the accuracy and availability of the image.

[0107] 3. By referring to the type of train and related railway information, the accuracy of train speed estimation is further improved. For example, the approximate range of train operating speed can be estimated by identifying the type of train and railway information such as railway signboards / signal light status, and determining the meaning of railway information such as different signboards / signal light status, thereby providing a reasonable reference initial value for speed estimation. On this basis, the calculated speed is judged and calibrated, the estimated range of speed is refined, and the accuracy of speed calculation is improved. It is suitable for real-time application scenarios and has universal applicability.

[0108] 4. The alarm for approaching trains no longer relies on fixed onboard equipment or ground equipment such as infrared and laser, avoiding the early warning mechanism's reliance on existing hardware and simplifying equipment deployment and maintenance.

[0109] 5. Use drones to collect video image data, perform data processing at a location close to the data source to achieve intelligent analysis of the video content, and perform real-time processing through edge computing devices, that is, computers deployed in places close to the data source, to avoid network delays, network congestion, and reduced service quality caused by the distance between the terminal device (drone) and the server (ground computer).

[0110] 6. By combining drone positioning technology with computer vision technology, it no longer relies on radar sensors, etc., avoiding their susceptibility to environmental influences, low resolution and limited information acquisition capabilities that lead to low prediction accuracy.

[0111] 7. Use a countdown method to issue a train approach alarm in advance to remind construction workers and ensure their safety. Preferably, different alarm sounds and warning lights of different colors can be set according to the different times of approach, accurately reminding evacuation and realizing an automated phased early warning and response mechanism.

[0112] Thus, in the preferred embodiment of the present invention, the video images along the railway are taken by drones. The high flexibility of drones enables them to take video images along the railway and cover complex terrain, overcoming the limitations of the limited field of view and difficulty in adjustment of traditional fixed cameras, and ensuring the comprehensiveness and flexibility of the monitoring range. The use of computer vision technology for image processing and analysis can quickly process a large amount of image / video data with higher accuracy and precision. At the same time, the image taken by the camera is subjected to image processing operations such as distortion correction and noise removal through the camera internal parameter matrix and distortion coefficient, which effectively improves the accuracy of image recognition. In addition, the train operating speed range is determined by the type of train and combined with key railway information such as railway signs and signal light status, and the calculated train speed is judged and calibrated, thereby improving the accuracy of the determined train speed. At the same time, the positioning technology is combined with computer vision technology to achieve the actual position of the train through the position of the drone, thereby improving the prediction accuracy.

[0113] It should be pointed out that although the various steps in the above embodiments are described in a specific order, those skilled in the art can understand that in order to achieve the effects of the present invention, different steps do not have to be performed in such an order. They can be performed simultaneously or in other orders, and some steps can also be added, replaced or omitted.

[0114] It should be noted that although the train approach control method based on drones constructed in the above specific manner is introduced as an example, those skilled in the art will understand that the present invention should not be limited thereto. In fact, users can flexibly adjust the relevant steps and parameters in the steps according to the actual application scenarios and other situations.

[0115] So far, the technical solutions of the present invention have been described in conjunction with the preferred embodiments shown in the accompanying drawings. However, it is easy for those skilled in the art to understand that the protection scope of the present invention is obviously not limited to these specific embodiments. Without departing from the principle of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will fall within the protection scope of the present invention.

Claims

1. A train approach control method based on drone, characterized in that: The control method comprises the following steps: Use drones equipped with cameras to take off and capture video images along the railway line; Acquire and identify a train image in the video image; Determining the speed and position of the train according to the train image; The remaining time for the train to arrive at the designated location is calculated based on the speed and location of the train.

2. The method for controlling train approach based on a UAV according to claim 1, characterized in that: The step of "obtaining and identifying the train image in the video image" includes: Pre-process the video image information transmitted to the ground computer; Identifying video image information containing a train image from the pre-processed video image information; The video image information including the train image is reprocessed.

3. The method for controlling train approach based on drone according to claim 1, characterized in that: In the “determining the speed and position of the train according to the train image”, the method for determining the speed of the train includes: The yolo model is used to extract image feature points in the train image; Determine the travel distance of the train corresponding to the train image of the adjacent frame according to the image feature points of the train image of the adjacent frame; Determine the speed of the train based on the distance it travels.

4. The method for controlling train approach based on drone according to claim 3, characterized in that: The control method further comprises the following steps: The speed of the train is calibrated, the step comprising: Determine the type of the train based on the reprocessed video image information; The speed of the train is determined with reference to the class of the train.

5. The method for controlling train approach based on drone according to claim 1, characterized in that: In the “determining the speed and position of the train according to the train image”, the method for determining the position of the train includes: Make the drone fly along the railway and obtain the location coordinates along the railway; Obtaining the positioning information of the drone through a positioning device mounted on the drone; The location of the train is determined based on the positioning information of the drone.

6. The method for controlling train approach based on drone according to claim 1, characterized in that: The "calculating the remaining time for the train to arrive at the designated location based on the speed and position of the train" includes: Determine the remaining distance of the train from the designated location based on the location of the train and the location information of the railway; The remaining time for the train to arrive at the designated location is determined according to the remaining distance and the speed of the train.

7. The method for controlling train approach based on UAV according to claim 1, characterized in that: The control method further comprises: Early warning steps, which include: The remaining time displays an early warning, which is used to: displaying the remaining time for the train to arrive at the designated location on a display device at the designated location; and / or Countdown is performed while the train arrives at the designated location.

8. The method for controlling train approach based on unmanned aerial vehicle according to claim 7, characterized in that: The early warning step also includes: Segmented reminder warning, which is used for: As the train gradually approaches the designated location, different types of reminder messages are set according to the remaining time.

9. A computer-readable storage medium, the storage medium comprising a memory, the memory being suitable for storing a plurality of program codes, characterized in that: The program code is suitable for being loaded and run by a processor to execute the UAV-based train approach control method according to any one of claims 1 to 8.

10. A computer device, comprising a memory and a processor, wherein the memory is suitable for storing a plurality of program codes, wherein: The program code is suitable for being loaded and run by the processor to execute the UAV-based train approach control method according to any one of claims 1 to 8.

Images