Train approach control method based on unmanned aerial vehicle, medium and computer device

Abstract

The embodiment of the present application relates to the field of railway safety technology, and specifically provides a train approaching control method based on a UAV, a computer readable storage medium and a computer device, wherein the train approaching control method based on the UAV comprises the following steps: a UAV carrying a camera is used to take off and take video images along a railway; train images in the video images are acquired and recognized; the speed and position of the train are determined according to the train images; and the remaining time for the train to reach a specified position is calculated according to the speed and position of the train. Through such a configuration, the remaining time for the train to reach any specified position with construction requirements can be accurately determined, and based on this, the safety along the railway can be improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of railway safety technology, and particularly provides a train approaching control method based on a UAV, a computer readable storage medium and a computer device. BACKGROUND

[0002] With the continuous construction of infrastructure in the railway system and the rapid expansion of the railway network, the demand for maintenance of the railway system is increasing. However, due to the complex construction environment of the railway line, such as the long length of the train and the uninterrupted approaching time within 24 hours, the construction and maintenance often have risks that cannot be ignored. In order to reduce the risk, accurately predicting the approaching time of the train and giving timely warning information are crucial to ensure the safety of track maintenance work.

[0003] Among them, the approaching time prediction of the train usually depends on the actual position, running speed and other information of the train currently in. Among them, the track speed (the running speed of the train running on the track) is currently widely used and effective method, such as the principle of track speed is mainly through computer vision technology and optical flow method to process the image data captured by the camera to obtain the running speed of the train. However, the fixed camera usually has the limitations of small shooting angle, visual blind area and the like. Among them, radar sensors are usually used to obtain the actual position of the train currently in, but the working reliability of the radar sensor is easily affected by the environment, and it has the limitations of low resolution and limited information acquisition capability.

[0004] Among them, the current strategy for giving warning information is usually: when the train approaches the construction area of track maintenance work, the infrared detection technology, Zigbee Internet of Things communication technology and the like are used to determine the information that the train approaches the construction area, and the relevant real-time broadcast is given to the construction personnel. The existing train monitoring system mostly depends on fixed monitoring equipment, accordingly, in the construction site, the corresponding ground device needs to be installed, and different installation strategies are designed for different environments, weather / climate and the like. These monitoring equipment often has high cost, and in some complex environments, there are problems of difficult to fully cover, easily affected by the environment, limited information acquisition capability and the like.

[0005] Although technical means provide data support, due to the existing train approaching alarm mechanism has the problem of information lag to a certain extent, especially in the background or construction site, it is difficult to obtain the position information of the train in real time. Information lag not only affects the safety monitoring of the train, but also may increase the safety hazard. Under this background, it is particularly important to adopt the safety warning mode of "people defense first, technical defense second", such as using human analysis and decision-making to make up for the limitations of technical means in the scene. For example, usually configure protection officers (station protection officers, site protection officers) at positions such as platforms and construction areas. When the train approaches the construction area, the station protection officer notifies the site protection officer to make construction evacuation instructions. However, the manual operation method is prone to false reports, delayed reports and missed reports and other problems.

[0006] To realize more accurate train approaching alarm, the present application combines positioning technology with computer vision to obtain the position information of the target, and obtains the distance of the train from the specified position by integrating the position information in the location service API. Combined with the running speed of the train, the time of the train approaching the specified position can be calculated. The image acquisition of the train is carried out by the way of unmanned aerial vehicle carrying camera, which can shoot from different heights and angles, so as to realize better flexibility and coverage.

[0007] The traditional train approaching time prediction only estimates the travel time between stations, but lacks accurate monitoring of the real-time position and arrival time of the train in a specific section. However, railway maintenance may occur in any area between stations. To realize real-time warning and remind construction personnel to evacuate in time, it is necessary to accurately predict the remaining time of the train arriving at the construction area. However, due to the long distance between stations, directly using the train travel time between stations to estimate the remaining time will result in large errors, making it difficult to achieve accurate prediction. Therefore, it is particularly important to explore the prediction method of train approaching time in non-station area. In the present application, the running speed of the train and the current position of the train are monitored in real time based on computer vision technology, and the time of the train approaching any position is accurately predicted. On this basis, different alarm sounds / warning lights can be issued according to the different approaching times of the train, and a phased warning mechanism is combined to realize real-time warning for the train approaching.

[0008] As disclosed in Chinese patent application (CN115050193A), a vehicle speed measurement method and system based on road monitoring video images are provided, wherein the method comprises the steps of: acquiring video image data of a road to be monitored; acquiring a plurality of image frames of the video image data selected by a user terminal; acquiring the center pixel points of the same vehicle in the two image frames respectively, and acquiring the vehicle pixel coordinate data of the center pixel points; inputting the vehicle pixel coordinate data into a conversion algorithm model and outputting the corresponding vehicle latitude and longitude coordinate data; acquiring the interval time length of the two image frames, and calculating the speed data of the vehicle within the interval time length based on the change of the latitude and longitude coordinate data of the same vehicle in the two image frames. However, in this document:

[0009] (1) Since the position of the monitoring device for monitoring the road is relatively fixed, the video of the vehicle cannot be shot from different heights / angles, and the perspective is single and lacks flexibility.

[0010] (2) The conversion from pixel coordinates to latitude and longitude coordinates involves complex geometric transformation, perspective transformation, etc., and the accuracy of the coordinate conversion by the linear model needs to be further improved.

[0011] (3) Since the two image frames cannot obtain clear identification marks to judge the running amount of the train, the method of calculating the speed only by the two image frames has defects such as unstable results and large errors, and in fact, this scheme is not suitable for the long vehicle such as train.

[0012] As China's invention patent application (CN118072227A) proposes a track traffic train speed measurement method based on knowledge distillation, including the following steps: step S1: collect train running video, and obtain feature points by FFT algorithm on video frame image, and input the feature points and image fusion into the student neural network; the student neural network includes yolo student network and Transformer network; step S2: the feature points and image fusion input data in step S1 are detected by yolo student network; step S3: the feature points detected by yolo student network in step S2 are input into Transformer network as sequence data, and sequence matching information is obtained by Transformer network; step S4: the student neural network combines the sequence matching information obtained in step S3 with the video frame rate to output the speed of the train by calculation; the student neural network is trained based on the teacher neural network; the teacher neural network combines yolo teacher network and Transformer network, and the self-attention mechanism is added in yolo teacher network to strengthen the capture of optical flow information between frames; the teacher neural network is trained with multi-scene data set to improve the generalization ability; the Transformer network is frozen based on the teacher neural network, and the yolo teacher network is simplified to yolo student network to obtain the student neural network; the student neural network is trained with single scene data set, and the knowledge of the teacher neural network is transferred to the student neural network based on knowledge distillation. However, in this document:

[0013] (1) Directly use FFT algorithm to process video frame image to extract feature points, without considering the distortion problem of video image shot by camera when zooming, so as to limit its application range.

[0014] (2) Only calculate the speed of the train by optical flow method, without combining the preliminary prediction of the speed range of the train with the railway train data and calibrating the calculated speed, so as to obtain low speed precision.

[0015] (3) The student network may appear overfitting phenomenon when training on specific data set, which may lead to poor performance in actual application.

[0016] (4) FFT algorithm is mainly used for frequency domain analysis, which may not fully capture important time domain features in train movement, especially dynamic information of train in rapid change (high speed running).

[0017] It can be seen that, due to the fact that the current train approaching time prediction only estimates the travel time between stations, there is a lack of accurate monitoring of the real-time position and arrival time of the train within a specific section, and it is impossible to make timely and effective warnings for a specific period. In addition, the fixed camera shooting angle used in the traditional speed measurement method is small, and there are visual blind areas. The radar sensor commonly used to obtain train position information is easily affected by the environment, has low resolution and limited information acquisition capability. In addition, in the way of using computer vision to identify images without processing image distortion, the accuracy of the image needs to be further improved. The single speed measurement method of not "judging the train category to predict the train speed range, and combining key railway information such as railway signs and signal lights to judge and calibrate the calculated speed" limits the breadth and depth of information utilization, and the speed measurement accuracy needs to be further improved. SUMMARY

[0018] The present application aims to at least partially solve the above technical problems and / or at least part of the above technical problems, specifically, how to reliably determine the train approaching any specified location with construction needs to improve safety.

[0019] In a first aspect, the present application provides a UAV-based train approaching control method, which comprises the following steps: using a UAV equipped with a camera to take video images along the railway; obtaining and identifying train images in the video images; determining the speed and position of the train according to the train images; calculating the remaining time for the train to arrive at the specified location according to the speed and position of the train.

[0020] Through such a configuration, it is possible to accurately determine the remaining time for the train to arrive at any specified location, and based on this, it is expected to improve the safety along the railway.

[0021] For the above-mentioned UAV-based train approaching control method, in one possible implementation, the step of "obtaining train images in the video images" includes: pre-processing video image information transmitted to the ground computer; identifying (such as framing) video image information containing train images from the pre-processed video image information; re-processing the video image information containing the train images.

[0022] For the above-mentioned UAV-based train approaching control method, in one possible implementation, the determination of the speed of the train in the "determination of the speed and position of the train according to the train images" includes: extracting image feature points in the train images through a pre-trained yolo model; determining the travel distance of the train corresponding to the adjacent frames of the train images according to the image feature points of the adjacent frames of the train images; determining the speed of the train according to the travel distance of the train.

[0023] The two-dimensional image coordinates of the image can be converted into three-dimensional world coordinates by a reasonable coordinate conversion algorithm. On this basis, the running distance of the train is determined according to the image feature points of the train image of the adjacent frame after coordinate conversion.

[0024] For the above-mentioned unmanned aerial vehicle-based train approaching control method, in a possible implementation, the control method further includes the step of calibrating the speed of the train, which includes: determining the category of the train according to the reprocessed video image information; and determining the speed of the train by referring to the category of the train.

[0025] For the above-mentioned unmanned aerial vehicle-based train approaching control method, in a possible implementation, the determination of the position of the train in the step of determining the speed and position of the train according to the train image includes: flying the unmanned aerial vehicle along the railway to obtain the position coordinates along the railway; obtaining the positioning information of the unmanned aerial vehicle by the positioning device carried on the unmanned aerial vehicle; and determining the position of the train according to the positioning information of the unmanned aerial vehicle.

[0026] For the above-mentioned unmanned aerial vehicle-based train approaching control method, in a possible implementation, the step of calculating the remaining time for the train to reach the specified position according to the speed and position of the train includes: determining the remaining distance of the train from the specified position according to the position of the train and the position of the railway; and determining the remaining time for the train to reach the specified position according to the remaining distance and the speed of the train.

[0027] For the above-mentioned unmanned aerial vehicle-based train approaching control method, in a possible implementation, the control method further includes a warning step, which includes: a remaining time display warning for displaying the remaining time for the train to reach the specified position on the display device at the specified position; and / or performing countdown during the process of the train reaching the specified position.

[0028] For the above-mentioned unmanned aerial vehicle-based train approaching control method, in a possible implementation, the warning step further includes a segmented reminder warning for setting different types of reminder information according to the different remaining times as the train gradually approaches the specified position.

[0029] Technical effects:

[0030] In the preferred embodiment of the present application, by monitoring the position and speed of the train in real time, the remaining time for the train to arrive at any designated location is accurately calculated and displayed. By mounting a camera on the UAV, it can take pictures from different heights and angles, providing better flexibility and coverage, and thus more reliably monitoring the speed of the train. By judging the type of train through image recognition and other means to predict the speed range of the train, and combining relevant railway information (including but not limited to railway signs, signal light status, etc.) to judge and calibrate the calculated speed, the predicted speed of the train is dynamically adjusted to improve the speed measurement accuracy. By rectifying the distortion of the video images taken by the UAV, the image quality is enhanced, and the accuracy and usability of the images are improved. In addition, by combining positioning technology with computer vision to obtain the position information of the target, and then obtaining the distance of the train to the designated location through the location service API, the cost is lower. In addition, through the countdown display screen and the sound and light alarm configured on the construction site, different alarm sounds / alert lights are set according to different train approaching times, a phased warning mechanism is realized, more targeted warning information is provided for the workers, and the risk of the construction and maintenance work crossing with the train operation is reduced / avoided.

[0031] In the second aspect, the present application provides a computer readable storage medium, the storage medium comprising a memory, the memory being adapted to store a plurality of program codes, the program codes being adapted to be loaded and run by a processor to execute the UAV-based train approaching control method of any one of the preceding aspects.

[0032] It can be understood that the computer readable storage medium has all the technical effects of the UAV-based train approaching control method of any one of the preceding aspects, which will not be repeated here.

[0033] Those skilled in the art can understand that the present application can realize all or part of the process in the train approaching control method based on the unmanned aerial vehicle, and the process can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer readable storage medium, and the computer program can realize the steps of each method embodiment when executed by a processor. The computer program includes computer program code, and the program code includes but is not limited to the program code for executing the train approaching control method based on the unmanned aerial vehicle. For the convenience of description, only the relevant part is shown. The computer program code can be in the form of source code, object code, executable file or some intermediate form. The computer readable storage medium can include any entity or device, medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory, random access memory, electrical carrier signal, telecommunication signal and software distribution medium, etc. that can carry the computer program code. It should be noted that the content included in the computer readable storage medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction, for example, in some jurisdictions, according to the legislation and patent practice, the computer readable storage medium does not include electrical carrier signals and telecommunication signals.

[0034] In a third aspect, the present application provides a computer device, comprising a memory and a processor, the memory is adapted to store a plurality of program codes, the program codes are adapted to be loaded and run by the processor to execute the train approaching control method based on the unmanned aerial vehicle according to any one of the preceding aspects.

[0035] It can be understood that the device has all the technical effects of the train approaching control method based on the unmanned aerial vehicle according to any one of the preceding aspects, which will not be repeated here. The device can be a computer control device formed by various electronic devices.

[0036] The computer device can include a processor, a memory, an input / output interface, a communication interface, a display unit and an input device. Among them, the processor, the memory and the input / output interface are connected through a system bus, the communication interface, the display unit and the input device are connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capability. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operating system and the computer program in the non-volatile storage medium to run. The input / output interface of the computer device is used to exchange information between the processor and external devices. The communication interface of the computer device is used to communicate with external terminals in a wired or wireless manner. The wireless manner can be realized through WIFI, mobile cellular network, NFC (near field communication) or other technologies. The computer program is executed by the processor to realize a kind of thermal management control method of power battery. The display unit of the computer device is used to form visual visible picture, can be display screen, projection device or virtual reality imaging device. The display screen can be liquid crystal display screen or electronic ink display screen, the input device of the computer device can be the touch layer covered on display screen, it can also be the key, trackball or touchpad arranged on the shell of computer device, it can also be external keyboard, touchpad or mouse etc. BRIEF DESCRIPTION OF DRAWINGS

[0037] The preferred embodiments of the present application will be described below with reference to the accompanying drawings, in which:

[0038] Figure 1 A flowchart of a train approaching control method based on a UAV according to an embodiment of the present application is shown.

[0039] Figure 2 A flowchart of image processing in a train approaching control method based on a UAV according to an embodiment of the present application is shown.

[0040] Figure 3 A flowchart of train speed monitoring in a train approaching control method based on a UAV according to an embodiment of the present application is shown.

[0041] Figure 4 A flowchart of coordinate conversion in a train approaching control method based on a UAV according to an embodiment of the present application is shown.

[0042] Figure 5 A flowchart of train position monitoring in a train approaching control method based on a UAV according to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0043] The preferred embodiments of the present application will be described below with reference to the accompanying drawings. Those skilled in the art will understand that these embodiments are only used to explain the technical principles of the present application, and are not intended to limit the protection scope of the present application.

[0044] It should be noted that in the description of the present application, the terms indicating the direction or positional relationship of "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer" and the like are based on the direction or positional relationship shown in the drawings, which is only for the convenience of description, and does not indicate or imply that the device or element must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as a limitation on the present application. In addition, the terms "first", "second" are only for the purpose of description, and cannot be understood as indicating or implying relative importance.

[0045] In addition, it should also be noted that in the description of the present application, unless otherwise explicitly specified and limited, the terms "mounting", "setting", "connecting" should be understood broadly, for example, it can be fixedly connected, or it can be detachably connected, or integrally connected; it can be directly connected, or indirectly connected through an intermediate medium, or it can be the internal communication of two elements. For those skilled in the art, the specific meaning of the above terms in the present application can be understood according to the specific circumstances.

[0046] In addition, in order to better illustrate the present application, a large number of specific details are given in the specific embodiments below, and those skilled in the art should understand that the present application can also be implemented without some specific details. In some examples, principles and the like well known to those skilled in the art are not described in detail in order to highlight the main idea of the present application.

[0047] The present application will be described below with reference to at least part of the accompanying drawings. Figures 1 to 5

[0048] As shown in Figure 1 In one possible embodiment, the unmanned aerial vehicle-based train approach control method mainly includes the following steps:

[0049] S100, using an unmanned aerial vehicle carrying a camera to take off and shoot video images along the railway;

[0050] S200, acquiring and recognizing train images in the video images;

[0051] S300, determining the speed and position of the train according to the train images;

[0052] S400, calculating the remaining time for the train to arrive at the specified position according to the speed and position of the train.

[0053] ​In a possible implementation, S100, i.e., the "using the unmanned aerial vehicle with the camera to take the video images along the railway line" includes:

[0054] After the unmanned aerial vehicle with the camera is used to take the video images along the railway line, the video images are converted into digital format (digital signals) for further processing, transmitted to the ground computer in a suitable signal frequency band, and then converted into corresponding video image information, and stored, identified and processed.

[0055] Referring mainly to Figure 2 In a possible implementation, S200, i.e., the "obtaining the train image in the video image" includes:

[0056] S210, pre-processing the video image information transmitted to the ground computer;

[0057] S220, identifying (framing) the video image information containing the train image from the pre-processed video image information;

[0058] S230, re-processing the video image information containing the train image.

[0059] In a possible implementation, S210, i.e., the "pre-processing the video image information transmitted to the ground computer" specifically includes:

[0060] By pre-processing the image information transmitted to the ground computer by the unmanned aerial vehicle, the irrelevant information in the image is eliminated or weakened, and the useful information is restored or enhanced, so that the image with better effect and higher quality is obtained, so as to facilitate the subsequent image recognition of the train. For example, the pre-processing mode can include but is not limited to: gray processing of the image, noise reduction and detail enhancement through frequency domain analysis processing, normalization processing of the image, adjustment of the image size, etc. to ensure the quality and consistency of the image.

[0061] In a possible implementation, S220, i.e., "framing the video image information containing the train image from the pre-processed video image information" specifically includes:

[0062] The pre-processed video image information is input, and a YOLO model is used for target detection to quickly detect whether a train exists in the video image information in real time. To further save computing resources and reduce prediction time, the behavior of the UAV and the processing task of the computer are dynamically adjusted through feedback. If the train is identified, the ground computer returns a signal to the UAV, and the return signal is equivalent to telling the UAV to wait for the ground computer to re-process the image to obtain an image that is more convenient for analysis, frame the image information of the position of the train after processing, track the train image, and perform subsequent analysis. If the train is not identified, the ground computer returns a signal to the UAV, and the return signal is equivalent to telling the UAV to continue shooting video images along the railway, and the ground computer does not perform additional image processing.

[0063] In a possible implementation, S230, i.e., the re-processing of the video image information containing the train image, specifically includes:

[0064] After the video image information in which the train exists is identified, the pre-processed image information in which the position of the train is framed is subjected to image enhancement through Kalman filtering to improve the quality of the image information, so that the image information is clearer, brighter, and easier to analyze. In addition, the image information processed through Kalman filtering is subjected to perspective transformation through a depth estimation model to eliminate perspective distortion in the image information. The camera intrinsic matrix and the distortion coefficient are calculated through a checkerboard calibration method, and the image distortion is processed using openCV in combination with the above parameters to obtain the re-processed image.

[0065] In a possible implementation, S300, i.e., the determination of the speed and position of the train according to the train image, specifically includes:

[0066] S310, a pre-trained YOLO model is used to extract image feature points in the train image. For example, the image feature points in the train image can be automatically detected and extracted according to the change in the gray value of the pixel points in the video image information.

[0067] S320, the driving distance of the train corresponding to the train image of the adjacent frame is determined according to the image feature points of the train image of the adjacent frame.

[0068] S330, the speed of the train is determined according to the driving distance of the train.

[0069] The implementation of S320 is generally as follows: a coordinate conversion algorithm is designed to convert two-dimensional image coordinates into three-dimensional world coordinates, and the driving distance of the train is determined according to the coordinate transformation of the train image of the adjacent frame.

[0070] If the speed of the train is directly determined without judgment and calibration, there may be a deficiency of low precision. Therefore, the speed of the train is determined by referring to the type of the train Figure 3 In a possible implementation, the unmanned aerial vehicle-based train approaching control method further includes:

[0071] S500, calibrating the speed of the train, including:

[0072] S510, determining the type of the train according to the reprocessed video image information;

[0073] S520, determining the speed of the train by referring to the type of the train.

[0074] In a possible implementation, the features of the train are further analyzed according to the train image framed in the video image data, and the type of the train is determined through feature matching. Referring to Table 1, the train can be roughly classified into three types, i.e., high-speed rail with an operating speed of 300-350 km / h, motor train with an operating speed of 200-250 km / h, and ordinary railway train with an operating speed of 120-160 km / h. By extracting the shape features and color features of the train, the type of the train is determined by matching with a pre-constructed feature database of the train, so as to obtain the operating speed range of the train, thereby providing a reference for speed monitoring. Specifically, based on feature matching and historical data, the speed reference value corresponding to the real-time application scenario can be quickly obtained through speed estimation. The correspondence described in Table 1 is only an exemplary description, and a person skilled in the art can determine the mapping relationship between the type of the train and the train video image data according to actual needs, so that the present application can adapt to determine multiple types of trains running on multiple lines, and improve the universality of type determination.

[0075] Table 1: Correspondence between train type, speed range and train features

[0076]

[0077] In a possible implementation, the manner of determining the speed of the train specifically includes: first, performing coordinate conversion on the train image, and converting pixel coordinates into actual coordinates on the road in combination with position information of the UAV. The pixel coordinates are positions of pixel points in the image. Then, feature points of the image are extracted, the train images in adjacent frames of the video image information are tracked to obtain a distance traveled by the train, and the speed of the train is calculated based on this analysis, for example, by taking an average of all instantaneous speeds in 1s to obtain an estimated value of the speed of the train. On the basis of the obtained speed range of the train, the speed range of the train is determined again in combination with a recognized railway sign (for example, the railway sign has information about the speed range and the like on the sign), and it is judged whether the calculated speed is in the range, and the speed of the train can also be judged and calibrated in combination with railway information (for example, a signal lamp state and the like).

[0078] In a possible implementation, S300, that is, the "determining the speed and position of the train according to the train image" specifically includes:

[0079] S340, causing the UAV to fly along the railway to obtain position coordinates along the railway;

[0080] S350, obtaining position information of the UAV by using a positioning device carried by the UAV;

[0081] S360, determining the position of the train according to the position information of the UAV.

[0082] In a possible implementation, S400, that is, the "calculating the remaining time for the train to arrive at the specified position according to the speed and position of the train" includes:

[0083] S401, determining a remaining distance of the train from the specified position according to the position of the train and position information of the railway;

[0084] S402, determining the remaining time for the train to arrive at the specified position according to the remaining distance and the speed of the train.

[0085] When the coordinate conversion is performed on the train image, the coordinates of the train image need to be converted from pixel coordinates in a pixel coordinate system to coordinates in a world coordinate system. Referring to Figure 4 , the process of coordinate conversion mainly includes:

[0086] 1、Pixel coordinate system→Camera coordinate system: Since there is a scaling and translation relationship between the camera coordinate system and the pixel coordinate system, assuming that the coordinates of the train image in the camera coordinate system are (x1, y1, z1), and the pixel coordinates of the train image are (u, v), the pixel coordinate system can be converted into the camera coordinate system by the camera intrinsic matrix K and the image depth H according to the following formula (1). The camera coordinate system takes the optical center of the camera as the origin, the x-axis and the y-axis are parallel to the X-axis and the Y-axis of the train image respectively, and the z-axis is the optical axis of the camera. The camera intrinsic matrix is a mathematical model used to transform 3D camera coordinates into 2D homogeneous image coordinates. Here, the conversion of pixel coordinates to camera coordinates is realized by multiplying the inverse of the camera intrinsic matrix. The image depth represents the distance information from the camera to the object in the image, and is used to process the perspective transformation of the image.

[0087]

[0088] 2、Camera coordinate system→UAV coordinate system: There is no translation relationship when converting the camera coordinate system into the UAV coordinate system, but the camera coordinate system satisfies the right-hand rule and the UAV coordinate system satisfies the left-hand rule. Assuming that the coordinates of the train in the UAV coordinate system are (x2, y2, z2), the camera coordinate system can be converted into the UAV coordinate system according to the following formula (2). The UAV coordinate system takes the center of gravity of the UAV as the origin, the nose direction as the x-axis, the wing direction as the y-axis, and the vertical downward direction as the z-axis.

[0089]

[0090] 3、UAV coordinate system→World coordinate system: As the UAV flies, the difference between the UAV coordinate system and the world coordinate system is the pose relationship of the UAV. For example, the pose information R of the UAV and the coordinates (x0, y0, z0) of the UAV in the world coordinate system can be obtained by visual odometry. Assuming that the coordinates of the train in the world coordinate system are (x3, y3, z3), the UAV coordinate system can be converted into the world coordinate system according to the following formula (3). In general cases, the x-axis of the world coordinate system points north, the y-axis points east, and the z-axis points downward.

[0091]

[0092] In this way, the NED coordinates (coordinates in the world coordinate system) of the train obtained based on the above coordinate conversion, combined with the linear correspondence relationship between the latitude / longitude / height information of the UAV and the NED coordinates of the UAV, obtain the latitude / longitude coordinates of the train. For reference Figure 5The position information of the railway can be integrated with the position information of the train for processing in the location service API. The actual distance between two points with known latitude and longitude can be calculated using a geodesic method, for example. Using this method, the pre-collected position information of the railway (various specified positions) can be processed in sequence, and the construction area can be taken as the starting point for calculation and the train as the end point for calculation, so as to obtain the remaining distance of the train from the various specified positions.

[0093] According to the determined remaining distance of the train from the specified position, in combination with the speed of the train, the remaining time of the train to reach the specified position can be calculated according to a pre-constructed prediction model of the train approaching time, based on multiple image information and speed information of multiple trains.

[0094] In a possible implementation, the method for controlling the train approaching based on the unmanned aerial vehicle further comprises:

[0095] S600, a warning step. This step includes:

[0096] S610, display warning and countdown of the remaining time.

[0097] Specifically, the remaining time of the train to reach the specified position determined according to S400 can be displayed in real time on a countdown display screen or other display device.

[0098] Because the construction personnel in a dark environment can not clearly see the information on the countdown display screen. In a possible implementation, S600 further comprises:

[0099] S620, segmented warning.

[0100] Specifically, as the train approaches the specified position, different types of reminder information (such as distinguishable alarm sounds and / or warning lights) are set according to the different remaining times. In this way, the workers can be effectively reminded to take corresponding safety measures as the train approaches, thereby achieving accurate warning.

[0101] Therefore, a segmented warning mechanism such as multi-sound recognition / multi-color recognition warning can be used. For example, an audible and visual alarm can be used, different alarm sounds and warning lights of different colors can be set according to different approaching times, to provide intuitive warning information and help the construction personnel to accurately understand the dynamic information of the train even when they cannot see the countdown display screen.

[0102] For example, when the construction personnel are dispersed at different positions in the construction area, they can not be able to see the countdown display screen of the train approaching, and therefore can not be able to take timely countermeasures. In a possible implementation, S600 further comprises:

[0103] S630, send the early warning information to one or more associated terminals.

[0104] In this way, as long as the terminal device and program module compatible with the train approaching countdown system of the present application are deployed at the construction site. The terminal device will display the remaining time countdown of the train arriving at the construction area in real time. No matter where the construction personnel are located, as long as they are connected to the system terminal device, they can accurately grasp the dynamic information of the train and make preparations in advance to ensure safety.

[0105] Illustratively, based on the above-mentioned early warning mechanism, once a train appears in the monitoring area and is about to pass through the construction area, the alarm will be automatically triggered to issue an alarm, and the alarm sound will change according to the different approaching time of the train. For example, the construction personnel can also take appropriate measures according to the approaching situation of the train without needing to pay attention to the train approaching countdown on the display screen in real time.

[0106] It can be seen that in the preferred embodiment of the application, it has the following advantages:

[0107] 1. Using a drone to carry a high-definition camera to monitor the position and speed of the train in real time. The flexibility and wide visual coverage of the drone ensure the comprehensiveness and accuracy of the monitoring, and can adapt to various railway environments and capture images and videos along the track in real time.

[0108] 2. Distortion processing of the video images captured by the camera carried by the drone to enhance image quality and improve image accuracy and usability.

[0109] 3. By referring to the type of train, relevant railway information, etc., the accuracy of the speed estimation of the train is further improved. For example, by identifying the type of train and the status of the railway sign / signal light, etc. railway information, the meaning of different signs / signal lights, etc. railway information is determined to estimate the approximate range of the operating speed of the train, thereby providing a reasonable reference initial value for speed estimation. On this basis, the calculated speed is judged and calibrated to refine the estimated range of the speed and improve the accuracy of the speed calculation, which is suitable for real-time application scenarios and has universality.

[0110] 4. For the alarm when the train approaches, it is no longer dependent on fixed on-board equipment or ground equipment such as infrared, laser, etc. This avoids the dependence of the early warning mechanism on existing hardware and simplifies the deployment and maintenance of the equipment.

[0111] 5. Video image data is collected by the unmanned aerial vehicle, data processing is performed at a location close to the data source to realize intelligent analysis of the video content, real-time processing is performed by an edge computing device, i.e., a computer deployed at a location close to the data source, to avoid network latency, network congestion, and degradation of service quality caused by the long distance between the terminal device (unmanned aerial vehicle) and the server (ground computer).

[0112] 6. By combining unmanned aerial vehicle positioning technology and computer vision technology, the present application no longer relies on radar sensors and the like, thereby avoiding low prediction accuracy caused by the fact that radar sensors are susceptible to environmental influences, have low resolution, and have limited information acquisition capabilities.

[0113] 7. The countdown mode is used to issue a train arrival warning in advance to remind construction personnel and ensure personnel safety. Preferably, different alarm sounds and color warning lights can be set according to different arrival times to accurately remind evacuation and realize an automatic phased warning and response mechanism.

[0114] In this way, in the preferred embodiment of the present application, the unmanned aerial vehicle captures video images along the railway, the high flexibility of the unmanned aerial vehicle enables it to capture video images along the railway and cover complex terrain, overcoming the limitations of traditional fixed cameras, i.e., limited field of view and difficulty in adjustment, thereby ensuring comprehensive and flexible monitoring. The use of computer vision technology for image processing and analysis enables rapid processing of a large amount of image / video data with higher accuracy and precision. At the same time, the image captured by the camera is subjected to distortion correction, noise removal, and other image processing operations by using the camera intrinsic matrix and distortion coefficient, thereby effectively improving image recognition accuracy. In addition, the train operation speed interval is determined by combining the train type with key railway information such as railway signboards and signal light states, the calculated train speed is judged and calibrated, thereby improving the accuracy of the determined train speed. At the same time, the positioning technology is combined with the computer vision technology to realize the acquisition of the actual position of the train through the position of the unmanned aerial vehicle, thereby improving the prediction accuracy.

[0115] It should be noted that although the steps in the above embodiments are described in a specific order, those skilled in the art can understand that, in order to achieve the effect of the present application, the steps do not necessarily have to be executed in this order, they can be executed simultaneously or in other orders, and some steps can be added, replaced, or omitted.

[0116] It should be noted that although the unmanned aerial vehicle-based train arrival control method constituted in the above specific manner is introduced as an example, those skilled in the art can understand that the present application should not be limited thereto. In fact, users can flexibly adjust the relevant steps and parameters and other elements in the steps according to the actual application scenario and the like.

[0117] The technical scheme of the present application has been described in combination with the preferred embodiments shown in the drawings, but it is easy for those skilled in the art to understand that the protection scope of the present application is obviously not limited to these specific embodiments. Those skilled in the art can make equivalent changes or replacements to the related technical features without departing from the principles of the present application, and the technical schemes after the changes or replacements will all fall within the protection scope of the present application.

Claims

1. A train approach control method based on a drone, characterized by, The control method comprises the following steps: The unmanned aerial vehicle equipped with a camera is used to take off and shoot video images along the railway line; Train images in the video images are acquired and recognized; The speed and position of the train are determined according to the train images; The remaining time for the train to arrive at the specified position is calculated according to the speed and position of the train; In the step of determining the speed and position of the train according to the train images, the determination of the speed of the train comprises: The yolo model is used to extract image feature points in the train images; The driving distance of the train corresponding to the adjacent frames of the train images is determined according to the image feature points of the adjacent frames of the train images; The speed of the train is determined according to the driving distance of the train; The control method further comprises the following steps: The speed of the train is calibrated, which comprises: The category of the train is determined according to the reprocessed video image information; The speed of the train is determined according to the category of the train.

2. The unmanned aerial vehicle-based train approach control method according to claim 1, characterized by, The step of acquiring and recognizing the train images in the video images comprises: The video image information transmitted to the ground computer is preprocessed; The video image information containing the train images is recognized from the preprocessed video image information; The video image information containing the train images is reprocessed. 3.The unmanned aerial vehicle-based train approach control method according to claim 1, wherein, In the step of determining the speed and position of the train according to the train images, the determination of the position of the train comprises: The unmanned aerial vehicle is made to fly along the railway to acquire the position coordinates along the railway line; The positioning information of the unmanned aerial vehicle is acquired by the positioning device carried on the unmanned aerial vehicle; The position of the train is determined according to the positioning information of the unmanned aerial vehicle. 4.The UAV-based train approach control method of claim 1, wherein, The step of calculating the remaining time for the train to arrive at the specified position according to the speed and position of the train comprises: The remaining distance of the train from the specified position is determined according to the position of the train and the position information of the railway; The remaining time for the train to arrive at the specified position is determined according to the remaining distance and the speed of the train. 5.The UAV-based train approach control method of claim 1, wherein, The control method further comprises: The warning step comprises: The remaining time display warning is used to: Display the remaining time for the train to arrive at the specified position on the display device at the specified position; and / or Countdown during the process of the train arriving at the specified position. 6.The UAV-based train approach control method of claim 5, wherein, The warning step further comprises: The segmented reminder warning is used to: Different types of reminder information are set according to the different remaining times as the train gradually arrives at the specified position.

7. A computer-readable storage medium comprising a memory adapted to store a plurality of program codes, characterized in that, The program code is adapted to be loaded and run by the processor to execute the unmanned aerial vehicle-based train approaching control method according to any one of claims 1 to 6.

8. A computer device, said device comprising a memory and a processor, said memory being adapted to store a plurality of program codes, characterized in that, The program code is adapted to be loaded and run by the processor to execute the unmanned aerial vehicle-based train approaching control method according to any one of claims 1 to 6.

Images