Railway tunnel unmanned aerial vehicle auxiliary sensing method, device and system and medium

Abstract

The invention relates to the technical field of unmanned aerial vehicles, and discloses a railway tunnel unmanned aerial vehicle auxiliary sensing method, device and system and a medium. The method comprises the following steps: acquiring environment images of a current position and different angles in a railway tunnel in an operation process of an unmanned aerial vehicle system; splicing the environment images at different angles to obtain a panoramic image of the current position; according to the current position, an unmanned aerial vehicle coordinate system is converted into a railway tunnel coordinate system, and the railway tunnel coordinate system is a linear coordinate system changing along with the track tangential direction in a railway tunnel; and transmitting a target image in a selected angle range in the panoramic image of the current position to the ground, and transmitting the current position and the converted unmanned aerial vehicle coordinate system to the ground. Auxiliary perception can be provided for beyond-visual-range operation of the unmanned aerial vehicle in the railway tunnel, so that operation of the unmanned aerial vehicle conforms to the real situation of the tunnel, and safe and reliable flight is guaranteed. The safety and the working efficiency of the operation of the unmanned aerial vehicle outside the tunnel in the tunnel are obviously improved.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of drone technology, and in particular to a method, device, system and medium for drone-assisted perception in a railway tunnel. Background Art

[0002] As the railway network continues to expand, the number of railway tunnels is also increasing. Traditional tunnel inspection methods, such as manual inspection, are not only inefficient but also pose safety risks. In practice, it is difficult to achieve beyond-visual-range operation of drones, and operators operate drones outside the tunnel to perform operations inside the tunnel, which has poor safety and operating efficiency. Summary of the invention

[0003] The purpose of the present invention is to at least provide a method, device, system and medium for assisting perception of UAVs in railway tunnels, which can at least solve the problem of poor safety and operating efficiency when operators operate UAVs outside the tunnel to perform operations inside the tunnel.

[0004] In order to solve the above technical problems, at least one embodiment of the present application provides a railway tunnel UAV-assisted perception method, including:

[0005] Acquire the current location and environmental images from different angles in the railway tunnel during the operation of the UAV system;

[0006] Obtaining a panoramic image of the current location by stitching together environmental images at different angles;

[0007] For the current position, converting the coordinate system of the drone into a railway tunnel coordinate system, wherein the railway tunnel coordinate system is a linear coordinate system that changes along the tangent direction of the track in the railway tunnel;

[0008] The target image within the selected angle range in the panoramic image of the current position is transmitted to the ground, and the current position and the converted drone coordinate system are transmitted to the ground at the same time.

[0009] In some preferred embodiments, the railway tunnel UAV-assisted perception method further includes:

[0010] Establishing a drone coordinate system, wherein the x-axis of the drone coordinate system is the flight direction of the drone system, the y-axis is the left direction of the flight direction, and the z-axis is the direction vertical to the ground and upward;

[0011] A railway tunnel coordinate system is established, wherein the x-axis of the railway tunnel coordinate system is the direction of the track tangent, the y-axis is the direction of the normal vector of the track tangent, and the z-axis is the direction vertical to the ground and upward.

[0012] In some preferred embodiments, the method further comprises:

[0013] Abstract each track in the railway tunnel into a three-dimensional arc, and establish a three-dimensional curve equation of the arc;

[0014] The three-dimensional curve equation is sampled at equal intervals along the X-axis of the railway tunnel coordinate system to obtain a tunnel section equation at each sampling point, and the tunnel section equations of all sampling points are constructed as a dictionary data set;

[0015] For the current position, the coordinate system of the UAV is converted to the coordinate system of the railway tunnel, including:

[0016] Query the dictionary data set for a target tunnel section equation that matches the current position;

[0017] The normal vector direction of the target tunnel section represented by the target tunnel section equation is determined as the new x-axis of the UAV coordinate system, and the direction parallel to the target tunnel section and perpendicular to the new x-axis is determined as the new y-axis of the UAV coordinate system.

[0018] In some preferred embodiments, obtaining a panoramic image of the current location by stitching together environment images at different angles includes:

[0019] The feature extraction algorithm is used to extract feature points from environmental images at different angles;

[0020] The feature point matching algorithm is used to determine the corresponding relationship between the feature points of environmental images at different angles;

[0021] According to the correspondence relationship between the feature points, the environment images at different angles are aligned and spliced ​​using a homography matrix or a perspective transformation matrix.

[0022] At least one embodiment of the present application provides a railway tunnel drone auxiliary perception device, including:

[0023] An acquisition module is used to acquire the current position and environmental images at different angles in the railway tunnel during the operation of the UAV system;

[0024] A stitching module, used for stitching environmental images at different angles to obtain a panoramic image of the current location;

[0025] A conversion module, for converting the coordinate system of the drone to a railway tunnel coordinate system according to the current position, wherein the railway tunnel coordinate system is a linear coordinate system that changes along the tangent direction of the track in the railway tunnel;

[0026] The transmission module is used to transmit the target image within the selected angle range in the panoramic image of the current position to the ground, and at the same time transmit the converted drone coordinate system to the ground.

[0027] At least one embodiment of the present application provides an airborne device, including:

[0028] at least one processor; and,

[0029] a memory communicatively connected to the at least one processor; wherein,

[0030] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the above-mentioned railway tunnel drone-assisted perception method.

[0031] At least one embodiment of the present application provides a drone system, comprising:

[0032] The drone itself;

[0033] The camera system is mounted on the drone body and is used to capture environmental images from different angles in the railway tunnel;

[0034] The radar unit is mounted on the drone body and is used to measure the distance between the drone body and obstacles;

[0035] The inertial measurement unit is mounted on the drone body and is used to measure the attitude and acceleration of the drone body;

[0036] The above-mentioned airborne equipment is connected to the camera system, the radar unit and the inertial measurement unit;

[0037] An image transmission radio station connected to the airborne device and used to transmit a target image within a selected angle range in the panoramic image of the current position to the ground;

[0038] A digital radio station is connected to the airborne device and is used to transmit the current position and the converted drone coordinate system to the ground.

[0039] At least one embodiment of the present application provides a ground station system, including:

[0040] A ground data processing unit is used to receive the target image, current position and converted drone coordinate system transmitted by the drone system, and is also used to compare the feature points in the target image with the feature points of the pre-constructed three-dimensional map of the railway tunnel to determine the depth of the railway tunnel where the drone body is currently located;

[0041] The three-dimensional map display unit is connected to the ground data processing unit and is used to display the current depth of the drone body in the railway tunnel and the current position in the drone coordinate system in three dimensions using the railway tunnel section contour map and the plane map formed by the railway tunnel plumb line and the rail tangent.

[0042] At least one embodiment of the present application provides a railway tunnel drone-assisted perception system, including:

[0043] The above-mentioned UAV systems;

[0044] The above ground station system;

[0045] The UAV system and the ground station system transmit data via a communication link.

[0046] At least one embodiment of the present application provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the above-mentioned railway tunnel drone-assisted perception method.

[0047] The invention discloses a method, device, system and medium for assisting perception of a railway tunnel drone. During the operation of the drone system, the drone system obtains images of the environment at different angles in the railway tunnel; a panoramic image of the current position is obtained by splicing the environment images at different angles; the drone coordinate system is converted to the railway tunnel coordinate system for the current position, and the railway tunnel coordinate system is a linear coordinate system that changes with the tangent direction of the track in the railway tunnel; the target image within the selected angle range in the panoramic image of the current position is transmitted to the ground, and the current position and the converted drone coordinate system are transmitted to the ground at the same time. After the images at different angles are spliced ​​into a panoramic image, the images within the specified angle range are transmitted back to the ground, and the drone coordinate system is adjusted to the tunnel coordinate system in real time and then controlled and transmitted back to the ground for display, which can provide assisting perception for the drone's beyond-visual-range operation in the railway tunnel, so that the operation of the drone conforms to the actual situation of the tunnel, and provides a guarantee for safe and reliable flight. The safety and efficiency of the operator operating the drone outside the tunnel to operate in the tunnel are significantly improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] One or more embodiments are exemplarily described by the pictures in the corresponding drawings, and these exemplary descriptions do not constitute limitations on the embodiments.

[0049] Figure 1 is a flow chart of a method for UAV-assisted perception in a railway tunnel provided by an embodiment of the present application;

[0050] Figure 2 is a schematic diagram of a railway tunnel drone auxiliary perception device provided by an embodiment of the present application;

[0051] Figure 3 It is a schematic diagram of a railway tunnel drone-assisted perception system provided by an embodiment of the present application. DETAILED DESCRIPTION

[0052] To make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the embodiments of the present application will be described in detail below in conjunction with the accompanying drawings. However, it will be appreciated by those skilled in the art that in the present application, many technical details are proposed in order to enable the reader to better understand the present application. However, even without these technical details and various changes and modifications based on the following embodiments, the technical scheme claimed in the present application can also be implemented. The division of the following embodiments is for the convenience of description, and the specific implementation of the present application should not be construed as any limitation, and the various embodiments can be combined and referenced with each other under the premise of no contradiction.

[0053] In order to solve the technical problem of poor safety and operating efficiency when operators operate drones outside the tunnel to perform operations in the tunnel, the present invention provides a railway tunnel drone-assisted perception method, device, system and medium. The implementation details of a railway tunnel drone-assisted perception method of this embodiment are specifically described below. The following content is only the implementation details provided for ease of understanding and is not necessary for the implementation of this solution.

[0054] Embodiment 1

[0055] The railway tunnel drone-assisted perception method of this embodiment can be applied to electronic devices with communication, computing and data storage capabilities. The specific process can be as follows: Figure 1 As shown, including:

[0056] Step 101: Acquire the current position and environmental images at different angles in the railway tunnel during the operation of the drone system.

[0057] In some specific implementations, the railway tunnel drone-assisted perception method of this embodiment further includes:

[0058] Establish a drone coordinate system. The x-axis of the drone coordinate system is the flight direction of the drone system, the y-axis is the left direction of the flight direction, and the z-axis is the vertical direction upward.

[0059] A railway tunnel coordinate system is established, in which the x-axis is the direction of the track tangent, the y-axis is the direction of the normal vector of the track tangent, and the z-axis is the direction vertical to the ground.

[0060] The current position of the UAV system obtained in step 101 is its position (coordinate) in the UAV coordinate system. In reality, the track in the railway tunnel is not a straight track, but an arc with a certain curvature. Therefore, during the operation of the UAV system in the railway tunnel, at least two coordinate systems are involved - the UAV coordinate system and the railway tunnel coordinate system, among which the railway tunnel coordinate system is a linear coordinate system that changes with the tangent direction of the track in the railway tunnel. Convert the UAV coordinate system to the railway tunnel coordinate system, and then push the push rod of the UAV remote control to change the forward flight state of the UAV system to flying along the railway tunnel.

[0061] Step 102: obtain a panoramic image of the current location by stitching together environmental images from different angles.

[0062] In the specific implementation, image processing technology is used to stitch together environmental images from different angles captured by the camera system to generate a panoramic image, providing operators with 360-degree visual perception.

[0063] In some cases, obtaining a panoramic image of the current location by stitching together environment images at different angles further includes:

[0064] The feature extraction algorithm is used to extract feature points from environmental images at different angles;

[0065] The feature point matching algorithm is used to determine the corresponding relationship between the feature points of environmental images at different angles;

[0066] According to the correspondence between feature points, the homography matrix or perspective transformation matrix is ​​used to align and stitch environment images at different angles.

[0067] By matching the feature points in the feature matrix of the environment images at different angles, and then using the homography matrix or perspective transformation matrix to align the images and stitch the panoramic images, the problem of difficulty in fusion between images can be avoided. Through feature extraction and description, feature matching and optimization, image fusion and boundary processing, the construction of a panoramic image inside the tunnel is realized, providing operators with 360° visual perception. In practice, by comparing the feature points of the panoramic image with the feature points of the three-dimensional map of the railway tunnel, the current depth of the drone body in the railway tunnel can be further determined. According to the distance from the drone body to obstacles such as the tunnel wall and contact network collected by the laser radar, as well as the attitude and acceleration of the drone body, the current position of the drone body in the railway tunnel can be determined, providing an important reference for the safe flight of the drone.

[0068] The current mainstream image stitching algorithm is the image stitching algorithm based on feature points (such as SIFT). This type of algorithm extracts specific features and matches the corresponding feature points in the two images to calculate the projection relationship between the images and the stitching displacement. However, since this type of algorithm detects the entire image when extracting features and performs matching after the detection is completed, there will be phenomena such as wrong matching in this process, so the complete image stitching algorithm also includes the step of eliminating wrong matches (RANSAC can be used to eliminate wrong matches). This stitching algorithm has a large amount of calculation and takes a long time. The drone body needs the visual perception system to provide various required visual information at any time during the movement, so this method will not be able to provide effective information in a timely manner when applied to the drone body.

[0069] The applicant has found through research on image stitching technology that the essence of image stitching is to find the projection relationship between images. In this embodiment, the environmental images at different angles are captured by a camera system composed of multiple cameras. By performing stereoscopic vision calibration on multiple cameras, the relative relationship between each camera is obtained, and the projection relationship between the images captured by each camera can be solved, thereby further realizing image stitching.

[0070] The cylindrical projection model is not only conducive to storage, but also can establish a mapping relationship with the plane to complete the plane display of the cylindrical panoramic image. At the same time, the horizontal 360° cylindrical vision can provide good information about the surrounding space environment, which meets the requirements of the drone's body perception ability. Therefore, the image acquisition method of stitching the environment images of multiple cameras can be approximately regarded as a single camera rotating around the axis.

[0071] In an example, the specific implementation steps are as follows:

[0072] 1) Deploy multiple cameras to ensure that they cover the area to be monitored and that there is a certain overlap between the cameras to facilitate subsequent image stitching. Each camera is equipped with a high-resolution sensor to capture clear and detailed images.

[0073] 2) Preprocess the collected images, including denoising, color correction, contrast enhancement, etc., to improve image quality, perform image distortion correction, and eliminate image distortion caused by camera lens distortion.

[0074] 3) Use advanced feature extraction algorithms (such as SIFT, SURF, ORB, etc.) to extract stable feature point arrays from images, and use feature point matching algorithms (such as brute force matching, FLANN matching, etc.) to find the corresponding relationship between feature points of different images.

[0075] 4) Based on the correspondence between the feature points, the image is aligned to a unified coordinate system using the homography matrix or perspective transformation matrix. During the alignment process, image rotation, scaling, and translation transformations need to be considered.

[0076] 5) Apply image fusion algorithms such as weighted averaging and Laplacian pyramid fusion between the aligned images to eliminate stitching traces and generate a smooth panoramic image. The consistency of image features such as color, brightness, and texture needs to be considered during the fusion process.

[0077] When the drone flies quickly in the tunnel, it can form a 360-degree or larger panoramic view by stitching images from multiple panoramic cameras, providing a wider field of view than a single camera. Advanced image stitching algorithms can achieve seamless stitching between images, eliminate stitching marks, and improve the image's viewing and practicality. It can capture and transmit image data within a specified angle range in real time, achieving efficient and stable transmission. Operators can understand the environment in the tunnel, which helps to detect and deal with abnormal situations in a timely manner.

[0078] Step 103: for the current position, convert the coordinate system of the UAV to the coordinate system of the railway tunnel. The coordinate system of the railway tunnel is a linear coordinate system that changes along the tangent direction of the track in the railway tunnel.

[0079] In some specific implementations, based on establishing the drone coordinate system and the railway tunnel coordinate system, the method of this embodiment further includes:

[0080] Abstract each track in the railway tunnel into a three-dimensional arc and establish the three-dimensional curve equation of the arc; and

[0081] The three-dimensional curve equation is sampled at equal intervals along the X-axis of the railway tunnel coordinate system to obtain the tunnel section equation of each sampling point, and the tunnel section equations of all sampling points are constructed as a dictionary data set.

[0082] In some examples, the SLAM algorithm is used to perform point cloud modeling on the tunnel, and the rails in the tunnel are abstracted into a three-dimensional arc using modeling software to obtain the three-dimensional curve equation. When sampling at equal intervals, 0.5m intervals are used to sample, and the tunnel section equations of all sampling points are obtained.

[0083] Furthermore, for the current position, the coordinate system of the UAV is converted to the coordinate system of the railway tunnel, including:

[0084] Step 103a, querying the dictionary data set for a target tunnel section equation that matches the current position;

[0085] Step 103b, determine the normal vector direction of the target tunnel section represented by the target tunnel section equation as the new x-axis of the UAV coordinate system, and determine the direction parallel to the target tunnel section and perpendicular to the new x-axis as the new y-axis of the UAV coordinate system.

[0086] In one example, the normal vector direction of the target tunnel section can be calculated by a mathematical method.

[0087] By constructing a dictionary data set consisting of tunnel section equations, when the UAV system flies to any position, the corresponding tunnel section equations in the dictionary data set can be queried to determine the new UAV coordinate system, so that the UAV's flight is always along the railway tunnel (the extension direction of the rails), and the control of the UAV is safer and more reliable.

[0088] Step 104: Transmit the target image within the selected angle range in the panoramic image of the current position to the ground, and transmit the current position and the converted drone coordinate system to the ground, so that the operator can intuitively understand the position, altitude, mileage and other information of the drone in the railway tunnel.

[0089] Users can flexibly select images of the desired angle for viewing through the interface of the ground station to meet the monitoring requirements under different scenarios and needs. In actual applications, users can select the angle range of the images they want to display, such as 0-180°, 180°-360°, and transmit the images within the selected angle range back to the ground station or remote server for storage and analysis, which not only meets the user's display needs for the tunnel environment, but also avoids the low transmission efficiency and unstable transmission caused by the transmission of all panoramic images, and the serious time delay that makes it impossible to accurately control the flight of the drone, thus bringing flight safety risks. In some cases, the ground station can also dynamically adjust the resolution and transmission speed of the image according to user needs and bandwidth limitations.

[0090] The method of this embodiment can provide auxiliary perception for the beyond-visual-range operation of the drone in the railway tunnel by stitching images of different angles into a panoramic image and then transmitting the images of the specified angle range back to the ground, and adjusting the drone coordinate system to the tunnel coordinate system in real time and then controlling and transmitting it back to the ground for display. This can make the operation of the drone conform to the actual situation of the tunnel and provide a guarantee for safe and reliable flight. In addition, the depth of the railway tunnel where the drone body is currently located can be determined by comparing the feature points in the target image with the feature points of the pre-constructed three-dimensional map of the railway tunnel, and a three-dimensional display of the depth information can be provided. The safety and efficiency of operators operating drones outside the tunnel to perform operations in the tunnel are significantly improved.

[0091] Embodiment 2

[0092] Another embodiment of the present application relates to a railway tunnel UAV auxiliary perception device. The implementation details of the railway tunnel UAV auxiliary perception device of this embodiment are specifically described below. The following content is only for the convenience of understanding the implementation details, and is not necessary for the implementation of this solution. The schematic diagram of the railway tunnel UAV auxiliary perception device of this embodiment can be as follows Figure 2 As shown, it includes an acquisition module 201, a splicing module 202, a conversion module 203 and a transmission module 204.

[0093] The acquisition module 201 is used to acquire the current position and environmental images at different angles in the railway tunnel during the operation of the UAV system;

[0094] A stitching module 202 is used to stitch environment images at different angles to obtain a panoramic image of the current location;

[0095] The conversion module 203 is used to convert the drone coordinate system into the railway tunnel coordinate system according to the current position, where the railway tunnel coordinate system is a linear coordinate system that changes along the track tangent direction in the railway tunnel;

[0096] The transmission module 204 is used to transmit the target image within the selected angle range in the panoramic image of the current position to the ground, and transmit the converted drone coordinate system to the ground.

[0097] The specific implementation of each module can be found in the above embodiments and will not be described again here.

[0098] All modules involved in this embodiment are logic modules. In practical applications, a logic unit can be a physical unit, a part of a physical unit, or a combination of multiple physical units. In addition, in order to highlight the innovative part of this application, this embodiment does not introduce units that are not closely related to solving the technical problems proposed by this application, but this does not mean that there are no other units in this embodiment.

[0099] Embodiment 3

[0100] Another embodiment of the present application relates to an airborne device, comprising:

[0101] at least one processor; and,

[0102] a memory communicatively connected to at least one processor; wherein,

[0103] The memory stores instructions that can be executed by at least one processor, and the instructions are executed by at least one processor so that the at least one processor can execute the above-mentioned railway tunnel drone-assisted perception method.

[0104] Among them, the memory and the processor are connected in a bus manner, and the bus may include any number of interconnected buses and bridges, and the bus connects various circuits of one or more processors and memories together. The bus can also connect various other circuits such as peripherals, voltage regulators, and power management circuits, which are well known in the art and are therefore not further described herein. The bus interface provides an interface between the bus and the transceiver. The transceiver can be one element or multiple elements, such as multiple receivers and transmitters, providing a unit for communicating with various other devices on a transmission medium. The data processed by the processor is transmitted on a wireless medium via an antenna, and further, the antenna also receives data and transmits the data to the processor.

[0105] The processor is responsible for managing the bus and general processing, and can also provide various functions, including timing, peripheral interfaces, voltage regulation, power management, and other control functions. Memory can be used to store data used by the processor when performing operations.

[0106] Embodiment 4

[0107] Another embodiment of the present application relates to a drone system, comprising:

[0108] The drone itself;

[0109] The camera system is mounted on the drone body and is used to capture environmental images from different angles in the railway tunnel;

[0110] The radar unit is mounted on the drone body and is used to measure the distance between the drone body and obstacles;

[0111] The inertial measurement unit is mounted on the drone body and is used to measure the attitude and acceleration of the drone body;

[0112] The airborne device of the above embodiment is connected to the camera system, the radar unit and the inertial measurement unit;

[0113] An image transmission radio station, connected to the airborne equipment, is used to transmit the target image within the selected angle range in the panoramic image of the current position to the ground;

[0114] The digital radio is connected to the airborne equipment and is used to transmit the current position and the converted drone coordinate system to the ground.

[0115] In the specific implementation, the drone uses a compact multi-rotor drone suitable for the railway tunnel environment. It has high maneuverability and stability, and can hover in the air at a fixed point, move vertically upward, vertically downward, horizontally left, horizontally right, horizontally left yaw, and horizontally right yaw.

[0116] The camera system includes multiple high-definition cameras for capturing environmental images at different angles in the railway tunnel, providing operators with all-round visual perception. The field of view of the multiple high-definition cameras can cover the area in the railway tunnel that needs to be photographed, and the field of view of adjacent cameras can be set to overlap to facilitate stitching. The high-definition cameras are, for example, panoramic cameras, wide-angle cameras, ultra-wide-angle cameras, etc., which are not limited here.

[0117] As the drone moves forward, backward, left and right, multiple high-definition cameras take pictures simultaneously to obtain images of the railway tunnel. After stitching the captured images, they are transmitted to the ground in real time through the image transmission radio according to the selected target angle range.

[0118] The radar unit uses a laser radar, which is used to measure the distance between the drone body and obstacles such as tunnel walls and contact lines in real time, providing accurate obstacle avoidance information for the drone. The laser radar can emit lasers in real time forward, backward, left, right, up and down. When the laser encounters obstacles such as tunnel walls and contact lines, the laser will be reflected back to the laser radar. By calculating the time difference between the emission and reception of the reflected laser, the distance between the drone body and obstacles such as tunnel walls and contact lines can be determined. Among them, the lasers in the left, right, up and down directions are mainly used for positioning the drone body in the tunnel, and the front and rear lasers are mainly used for obstacle avoidance.

[0119] The Inertial Measurement Unit (IMU), which has built-in sensors such as accelerometers and gyroscopes, can be used to measure the attitude and acceleration of the drone body and provide the drone with the navigation information required for stable flight.

[0120] The airborne device can use an airborne computer to determine the current position of the drone body based on the distance between the drone body and obstacles such as tunnel walls and contact lines, as well as the attitude and acceleration.

[0121] The image transmission station consists of an aerial image transmitter and a ground image receiver. It encodes and modulates the image signals collected by the camera, converts them into radio signals and transmits them into the air. The receiver receives these signals, demodulates and decodes them, and restores them to image signals for real-time display, thereby realizing wireless transmission of video data.

[0122] The digital radio station includes a data air transmitter and a data ground receiver. It modulates the digital data such as the battery power of the drone, the distance between the drone and obstacles such as tunnel walls and contact lines collected by the laser radar, and the attitude and acceleration of the drone measured by the inertial measurement unit IMU, and converts them into radio frequency signals suitable for wireless transmission, which are then transmitted into the air through the antenna. After the antenna at the receiving end receives these radio frequency signals, it is restored to digital data through the demodulator.

[0123] In some cases, the UAV system may also include a lighting system, which includes a plurality of lamp heads distributed on the UAV body to provide light energy for the UAV system, illuminate the railway tunnel, and facilitate providing light energy for the camera system during image acquisition.

[0124] Embodiment 5

[0125] Another embodiment of the present application relates to a ground station system, comprising:

[0126] A ground data processing unit, used to receive the target image, current position and converted drone coordinate system transmitted by the drone system of the aforementioned embodiment, and also used to compare the feature points in the target image with the feature points of the pre-constructed three-dimensional map of the railway tunnel to determine the depth of the railway tunnel where the drone body is currently located;

[0127] The three-dimensional map display unit is connected to the ground data processing unit and is used to display the current depth of the UAV body in the railway tunnel and the current position of the UAV coordinate system in three dimensions using the railway tunnel section contour map and the plane map formed by the railway tunnel plumb line and the rail tangent.

[0128] In the specific implementation, the 3D map display unit displays the 3D map of the railway tunnel and the current position of the UAV in real time, so that the operator can intuitively understand the status of the UAV in the tunnel. It also displays the cross-section (section) contour of the railway tunnel and the plane formed by the plumb line of the railway tunnel and the tangent of the rail, providing the operator with an intuitive display of the height and mileage position of the UAV in the railway tunnel. The ground data processing unit is responsible for receiving and processing data from the UAV system, including images, lidar data and IMU data, and generating corresponding visualization information.

[0129] Embodiment 6

[0130] Another embodiment of the present application relates to a railway tunnel drone-assisted perception system, such as Figure 3 As shown, including:

[0131] The UAV system of the above embodiment;

[0132] The ground station system of the above embodiment;

[0133] Data is transmitted between the UAV system and the ground station system via a communication link.

[0134] In specific applications, the communication link ensures real-time communication between the UAV system and the ground station system, including data transmission and command issuance. Image data can also be compressed and optimized according to the bandwidth and stability of the communication link to ensure real-time transmission and clear display of image data.

[0135] The communication link is the data transmission channel between the air and the ground. The image transmission of the image transmission radio, the data transmission of the data transmission radio, and the control instructions sent by the ground station remote control to the UAV system all need to be transmitted in this transmission channel. The transmission of the three types of data is carried out through different communication frequency bands. In one example, the control link uses the 2.4GHz or 900MHz frequency band, the image transmission link uses the 5.8GHz frequency band to obtain high-definition images of the railway tunnel, and the data transmission link uses the 433MHz or 2.4GHz frequency band. During the transmission process, appropriate modulation methods (such as QAM, OFDM) are used to improve transmission efficiency and anti-interference capabilities; forward error correction coding (FEC) is used to improve the reliability of data transmission; automatic retransmission request (ARQ) is used to automatically retransmit when data transmission fails to ensure data integrity; if the railway tunnel inspection process is a confidential project, the control and data transmission can also be encrypted in the communication link to ensure the security of the communication link. Finally, the communication link ensures real-time communication between the UAV and the ground station, including image transmission, data transmission and command issuance.

[0136] Embodiment 7

[0137] Another embodiment of the present application relates to a computer-readable storage medium storing a computer program, which implements the above-mentioned railway tunnel drone-assisted perception method when executed by a processor.

[0138] Those skilled in the art can understand that all or part of the steps in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a program, and the program is stored in a storage medium, including a number of instructions to enable a device (which can be a single-chip microcomputer, chip, etc.) or a processor to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as: ROM), random access memory (Random Access Memory, referred to as: RAM), disk or optical disk and other media that can store program codes.

[0139] Those skilled in the art will appreciate that the above embodiments are specific embodiments for implementing the present application, and in actual applications, various changes may be made thereto in form and detail without departing from the spirit and scope of the present application.

Claims

1. A UAV-assisted perception method for railway tunnels, characterized in that: include: Acquire the current location and environmental images from different angles in the railway tunnel during the operation of the UAV system; Obtaining a panoramic image of the current location by stitching together environmental images at different angles; For the current position, converting the coordinate system of the drone into a railway tunnel coordinate system, wherein the railway tunnel coordinate system is a linear coordinate system that changes along the tangent direction of the track in the railway tunnel; The target image within the selected angle range in the panoramic image of the current position is transmitted to the ground, and the current position and the converted drone coordinate system are transmitted to the ground at the same time.

2. The method for UAV-assisted perception in railway tunnels according to claim 1 is characterized in that: Also includes: Establishing a drone coordinate system, wherein the x-axis of the drone coordinate system is the flight direction of the drone system, the y-axis is the left direction of the flight direction, and the z-axis is the direction vertical to the ground and upward; A railway tunnel coordinate system is established, wherein the x-axis of the railway tunnel coordinate system is the direction of the track tangent, the y-axis is the direction of the normal vector of the track tangent, and the z-axis is the direction vertical to the ground and upward.

3. The method for UAV-assisted perception in railway tunnels according to claim 1 is characterized in that: The method further comprises: Abstract each track in the railway tunnel into a three-dimensional arc, and establish a three-dimensional curve equation of the arc; The three-dimensional curve equation is sampled at equal intervals along the X-axis of the railway tunnel coordinate system to obtain a tunnel section equation at each sampling point, and the tunnel section equations of all sampling points are constructed as a dictionary data set; For the current position, the coordinate system of the UAV is converted to the coordinate system of the railway tunnel, including: Query the dictionary data set for a target tunnel section equation that matches the current position; The normal vector direction of the target tunnel section represented by the target tunnel section equation is determined as the new x-axis of the UAV coordinate system, and the direction parallel to the target tunnel section and perpendicular to the new x-axis is determined as the new y-axis of the UAV coordinate system.

4. The method for UAV-assisted perception in railway tunnels according to claim 1 is characterized in that: The panoramic image of the current location is obtained by stitching together environmental images from different angles, including: The feature extraction algorithm is used to extract feature points from environmental images at different angles; The feature point matching algorithm is used to determine the corresponding relationship between the feature points of environmental images at different angles; According to the correspondence relationship between the feature points, the environment images at different angles are aligned and spliced ​​using a homography matrix or a perspective transformation matrix.

5. A railway tunnel drone-assisted sensing device, characterized in that: include: An acquisition module is used to acquire the current position and environmental images at different angles in the railway tunnel during the operation of the UAV system; A stitching module, used for stitching environmental images at different angles to obtain a panoramic image of the current location; A conversion module, for converting the coordinate system of the drone to a railway tunnel coordinate system according to the current position, wherein the railway tunnel coordinate system is a linear coordinate system that changes along the tangent direction of the track in the railway tunnel; The transmission module is used to transmit the target image within the selected angle range in the panoramic image of the current position to the ground, and at the same time transmit the converted drone coordinate system to the ground.

6. An airborne device, characterized in that: include: at least one processor; as well as, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the railway tunnel drone-assisted perception method as described in any one of claims 1 to 4.

7. An unmanned aerial vehicle system, characterized in that: include: The drone itself; The camera system is mounted on the drone body and is used to capture environmental images from different angles in the railway tunnel; The radar unit is mounted on the drone body and is used to measure the distance between the drone body and obstacles; The inertial measurement unit is mounted on the drone body and is used to measure the attitude and acceleration of the drone body; The airborne device of claim 6, connected to the camera system, the radar unit and the inertial measurement unit; An image transmission radio station connected to the airborne device and used to transmit the target image within the selected angle range in the panoramic image of the current position to the ground; A digital radio station is connected to the airborne device and is used to transmit the current position and the converted drone coordinate system to the ground.

8. A ground station system, characterized in that: include: A ground data processing unit, used to receive the target image, current position and converted drone coordinate system transmitted by the drone system of claim 7, and also used to compare the feature points in the target image with the feature points of the pre-constructed three-dimensional map of the railway tunnel to determine the depth of the railway tunnel where the drone body is currently located; The three-dimensional map display unit is connected to the ground data processing unit and is used to display the current depth of the drone body in the railway tunnel and the current position in the drone coordinate system in three dimensions using the railway tunnel section contour map and the plane map formed by the railway tunnel plumb line and the rail tangent.

9. A railway tunnel drone-assisted perception system, characterized in that: include: The drone system according to claim 7; The ground station system as claimed in claim 8; The UAV system and the ground station system transmit data via a communication link.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the railway tunnel drone-assisted perception method according to any one of claims 1 to 4 is implemented.

Images