Unmanned flight device and method for bridge detection

Abstract

The invention discloses an unmanned flight device and method for bridge detection. The unmanned aerial vehicle identifies a calibration line in an image corresponding to a target bridge. The unmanned flight device controls the unmanned flight device to fly along the calibration line to execute a bridge detection task based on the image including a shielded portion and an unshielded portion, wherein a calibration start point and a calibration end point of the calibration line respectively correspond to longitude and latitude coordinates of a real world.

Description

Technical Field

[0001] The present invention relates to an unmanned aerial device and method for bridge detection. Specifically, the present invention relates to an unmanned aerial device and method capable of executing a bridge detection task. Background Art

[0002] In recent years, with the rapid development of Unmanned Aerial Vehicle (UAV) technology, a large number of applications related to UAVs have been proposed. In addition, due to the cost and speed advantages of UAVs, the task of performing object inspection (e.g., safety inspection of bridge structures) through UAVs is an important application.

[0003] In the prior art, when an unmanned aerial device is in operation, it needs to hover in the air based on its coordinates (e.g., Global Positioning System (GPS)), altitude (e.g., barometer) and heading (e.g., magnetometer). However, when the GPS signal is blocked by a building (e.g., under a bridge) or the weather is bad, the unmanned aerial device may have flight safety issues.

[0004] In the prior art, there are many auxiliary positioning methods for weak GPS environments, such as visual navigation (Visual Localization and Mapping) through vision, LiDAR, etc.

[0005] However, since the virtual map generated by visual navigation is based on the position of the UAV itself, it is unable to instantly and accurately mark the data position (ie, the world coordinate position) during inspection, and therefore cannot be used when inspecting bridges.

[0006] In view of this, how to provide a technology that can perform bridge inspection tasks is a goal that the industry urgently needs to work hard on. Summary of the invention

[0007] An object of the present invention is to provide an unmanned aerial device for bridge inspection. The unmanned aerial device includes an upward camera device and a processor, and the processor is electrically connected to the upward camera device. The upward camera device is used to obtain an image corresponding to a target bridge based on an upward shooting angle, wherein the image includes a shielded portion and an unshielded portion corresponding to a global positioning system. The processor identifies a calibration line corresponding to the target bridge in the image. Based on the image including the shielded portion and the unshielded portion, the processor controls the unmanned aerial device to fly along the calibration line to perform a bridge inspection task, wherein a calibration start point and a calibration end point of the calibration line each correspond to a latitude and longitude coordinate in the real world.

[0008] An object of the present invention is to provide an unmanned aerial device for bridge inspection. The unmanned aerial device includes an upward camera device and a processor, and the processor is electrically connected to the upward camera device. The upward camera device is used to obtain an image corresponding to a target bridge based on an upward shooting angle, wherein the image includes a bridge bottom portion and a sky portion corresponding to the target bridge. The processor identifies a calibration line corresponding to the target bridge in the image. Based on the image including the bridge bottom portion and the sky portion, the processor controls the unmanned aerial device to fly along the calibration line to perform a bridge inspection task, wherein a calibration start point and a calibration end point of the calibration line each correspond to a latitude and longitude coordinate in the real world.

[0009] Another object of the present invention is to provide a bridge detection method for an unmanned aerial device, the unmanned aerial device comprising an upward camera device and a processor, the upward camera device being used to acquire an image corresponding to a target bridge based on an upward shooting angle, the bridge detection method comprising the following steps: identifying a calibration line corresponding to the target bridge in the image; and based on the image, controlling the unmanned aerial device to fly along the calibration line to perform a bridge detection task, wherein a calibration start point and a calibration end point of the calibration line each correspond to a latitude and longitude coordinate in the real world.

[0010] The bridge detection technology (at least including the device and the method) provided by the present invention constructs a virtual map by identifying the calibration line of the target bridge with known location information and the actual flight path of the unmanned aerial device. Then, the bridge detection technology provided by the present invention calculates the coordinate transformation matrix corresponding to the virtual map and the real world based on the actual flight path and the calibration line. In addition, the bridge detection technology provided by the present invention can also control the unmanned aerial device to fly along the virtual detection paths based on multiple virtual detection paths and the virtual map to detect the target bridge, and generate multiple detection images corresponding to the target bridge and the real coordinate values ​​of the real world corresponding to each of the detection images. Due to the bridge detection technology provided by the present invention, the task of detecting bridges can be automatically and efficiently performed, solving the shortcomings of the prior art.

[0011] The detailed technology and implementation methods of the present invention are described below in conjunction with the accompanying drawings so that those skilled in the art in the technical field to which the present invention belongs can understand the technical features of the invention for which protection is sought. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 A schematic diagram of the structure and environment of the unmanned aerial device according to the first embodiment;

[0013] Figure 2 A schematic diagram for describing the inspection operation and images of the first embodiment;

[0014] Figure 3 A schematic diagram is provided to illustrate the output of certain embodiments;

[0015] Figure 4 A schematic diagram is provided to illustrate the relationship of certain embodiments;

[0016] Figure 5 A schematic diagram is provided to illustrate flight results of certain embodiments;

[0017] Figure 6 A partial flow chart for describing the bridge detection method according to the third embodiment; and

[0018] Figure 7 A flowchart is provided to illustrate the operation of certain embodiments.

[0019] Description of Reference Numerals

[0020] 1: Unmanned aerial vehicle

[0021] 3: Computer

[0022] 5: User operated device

[0023] NW: Network

[0024] 11: Camera device

[0025] 12: Distance measurement and obstacle avoidance device

[0026] 13: Processor

[0027] 14: Flight and control devices

[0028] 15: Positioning device

[0029] 16: Communication device

[0030] 100: Image diagram

[0031] TB: Target Bridge

[0032] CL: Calibration Line

[0033] FC: Floating Clouds

[0034] 200: Inspection operation diagram

[0035] SP: Calibration starting point

[0036] EP: calibration end point

[0037] 201: Image diagram

[0038] TA: Target Area

[0039] AG: Angle

[0040] RL: Reference Line

[0041] FD: Flight direction

[0042] CP: Center Point

[0043] VL: Vertical Line

[0044] 300: Feature map

[0045] 302: Coordinate sequence

[0046] 304: Starting point

[0047] 306: End

[0048] 400: World coordinate system

[0049] 402: Virtual Coordinate System

[0050] 404: Waypoint

[0051] 406: Virtual route

[0052] P: Waypoint

[0053] 500, 502, 504, 506: Waypoints

[0054] 508: Flight Path

[0055] 510: Virtual Map

[0056] 600: Bridge Inspection Methods

[0057] S601, S603: Steps

[0058] CS: Calibration Phase

[0059] S701, S702, S703, S704, S705: Operation

[0060] IS: Inspection Phase

[0061] S706, S707, S708, S709: Operation DETAILED DESCRIPTION

[0062] The following will explain an unmanned aerial device and method for bridge inspection provided by the present invention through implementation examples. However, these implementation examples are not intended to limit the present invention to any environment, application or method as described in these implementation examples. Therefore, the description of the implementation examples is only for the purpose of illustrating the present invention, and is not intended to limit the scope of the present invention. It should be understood that in the following implementation examples and drawings, elements that are not directly related to the present invention have been omitted and are not shown, and the size of each element and the size ratio between the elements are only for illustration, and are not intended to limit the scope of the present invention.

[0063] First, in the application environment of the present disclosure, the purpose is to detect a target bridge by an unmanned aerial device equipped with a camera (single or multiple). The present disclosure can perform a detection operation corresponding to the target bridge by using the unmanned aerial device 1 .

[0064] The first embodiment of the present invention is an unmanned aerial device 1, the structure of which is schematically shown in FIG. Figure 1 The unmanned aerial device 1 comprises a camera device 11 , a distance measuring and obstacle avoiding device 12 , a processing device 13 , a flight and control device 14 , a positioning device 15 and a communication device 16 .

[0065] In some embodiments, the camera device 11 includes an upward camera device. In some embodiments, the camera device 11 further includes a detection camera device and a rotation mechanism (e.g., a pan / tilt). The distance measurement and obstacle avoidance device 12 includes one or more distance measurement devices, such as an optical flow sensor, an optical radar (Light Detection And Ranging; LiDAR), an infrared distance measurement device, an ultrasonic distance measurement device, etc.

[0066] In some embodiments, the flight and control device 14 includes a power device (e.g., a rotor or fixed wing), an inertial measurement device (e.g., an accelerometer and a gyroscope), and an environmental sensing device (e.g., a barometer, a thermometer, a magnetometer, etc.). The positioning device 15 includes a GPS device. The communication device 16 includes an antenna device.

[0067] In some embodiments, the processing device 13 includes a processor, and the processing device 13 is electrically connected to the camera device 11 (for example, the processor is electrically connected to the upward camera device). The processor can be various processing units, a central processing unit (CPU), a microprocessor, or other computing devices known to those skilled in the art of the present invention.

[0068] It should be noted that the upward camera device is a camera device that can move the camera lens to shoot upward from the UAV 1, and therefore includes but is not limited to a fixed upward camera device. Therefore, any camera device that can shoot images from above should fall within the scope of the present disclosure, such as a camera device with a rotating mechanism (e.g., a gimbal) installed at the nose or tail of the aircraft.

[0069] In certain embodiments, in the operational application of the present disclosure, the unmanned aerial device 1 can transmit the relevant image content generated by the unmanned aerial device 1 to the computer 3 or related computing equipment on the ground side through the network NW for processing. In certain embodiments, the unmanned aerial device 1 can receive a bridge detection instruction from the user operating device 5, for example: performing a bridge detection task between the 13th pier and the 14th pier of the bridge numbered A3. In this embodiment, the upward camera device is used to obtain an image corresponding to a target bridge based on an upward shooting angle (for example: 70 degrees upward from the horizontal plane) (for example: Figure 7 In some other embodiments, the UAV 1 uses a rotating mechanism to rotate the camera device to obtain an image corresponding to a target bridge at an upward shooting angle.

[0070] In some embodiments, the image includes a bridge bottom portion and a sky portion corresponding to the target bridge.

[0071] For easier understanding, please refer to Figure 1 , which illustrates an image schematic diagram 100 obtained by the upward camera device. In the present disclosure, when the upward camera device of the unmanned aerial device 1 performs upward shooting, the following image can be obtained: Figure 1 The image shown includes a target object (ie, a target bridge TB) and a calibration line CL formed by the target object and a surrounding area. For example, the surrounding area may be a sky including floating clouds FC, or other areas with a large color difference from the target object.

[0072] It should be noted that the operation of the present disclosure can be divided into a calibration phase and an inspection phase (for example: Figure 7 First, in the calibration phase CS and the inspection phase IS, the UAV 1 flies a line segment (i.e., the calibration line) with known length and path information, and calculates the conversion matrix between the virtual coordinate system (i.e., the mapping coordinate system) and the real world coordinate system (i.e., the world coordinate system) by analyzing the features of the calibration line in the image, and outputs the virtual map as the basic map for the inspection phase IS. Then, the inspection phase IS uses the basic map for flight navigation and outputs the world coordinate values ​​of the captured image.

[0073] The following will first describe the specific operations of the UAV 1 executing the calibration phase CS in the present disclosure.

[0074] In this embodiment, the processor identifies a calibration line in the image corresponding to the target bridge (eg: Figure 7 Then, the processor controls the UAV 1 to fly along the calibration line based on the image including the bridge bottom portion and the sky portion (for example: Figure 7In some embodiments, a calibration start point and a calibration end point of the calibration line each correspond to a latitude and longitude coordinate of a real world.

[0075] For easier understanding, please refer to Figure 2 , which illustrates an inspection operation diagram 200. In the inspection operation diagram 200, an inspection target object (i.e., target bridge TB) has been pre-selected and an unmanned aerial device 1 is deployed near the target object. The unmanned aerial device 1 can be controlled by, for example, a remote controller, a computer, a mobile phone or a smart terminal.

[0076] In this example, a straight line area of ​​known length is pre-selected on the target bridge TB as the calibration line CL, and the processor identifies the calibration line CL corresponding to the target bridge in the image to control the unmanned aerial device 1 to fly along the calibration line (i.e., fly from the calibration starting point SP to the calibration end point EP). It should be noted that the world coordinates of the calibration starting point SP and the calibration end point EP of the calibration line CL are known (e.g., the latitude and longitude coordinates of the real world), and this world coordinate refers to a value that can be interpreted by a person or software, such as longitude and latitude, bridge mileage, or coordinate values ​​on map data.

[0077] In some embodiments, during the calibration phase CS, the UAV 1 can simultaneously perform inspection tasks, which can reduce the cost of repeated flights of the UAV 1 .

[0078] It should be noted that in the present disclosure, the unmanned aerial device 1 can perform visual positioning navigation through the acquired images. In some embodiments, the unmanned aerial device 1 further includes an optical flow module that looks down at the ground. The visual range of the ground photographed by the optical flow module can obtain corresponding sensing data, and the movement vector in a short period of time and the offset of the unmanned aerial device 1 relative to the ground can be calculated based on this optical flow positioning function. Based on this optical flow positioning function, the drone can hover in the air without GPS, and can perform slight flight movements to the surrounding area.

[0079] In some embodiments, to make the determination information more accurate, the calibration line CL is located at an image position corresponding to an area of ​​the target bridge TB of the image that accounts for approximately three quarters (ie, the target bridge TB occupies three quarters of the area, and the surrounding area occupies one quarter of the area).

[0080] In some embodiments, the processor may control the UAV 1 to fly close to the calibration line based on preset parameters to reduce the error generated by the measurement. Specifically, the processor calculates an angle between a flight direction of the UAV 1 and the calibration line and a distance between the UAV 1 and the calibration line. Then, the processor adjusts the flight direction of the UAV 1 based on the angle and the distance, wherein the angle between the flight direction and the calibration line is not greater than a preset angle value, and the distance between the UAV 1 and the calibration line is not greater than a preset distance value.

[0081] For example, when the processor determines that the distance between the UAV 1 and the calibration line is greater than the preset distance value, the processor controls the UAV 1 to perform a translation operation. In addition, when the processor determines that the angle between the flight direction of the UAV 1 and the calibration line is greater than the preset angle value, the processor controls the UAV 1 to perform a steering operation.

[0082] For easier understanding, please continue to refer to Figure 2 . Figure 2 The image diagram 201 captured by the UAV 1 is illustrated, which includes the target area TA and the calibration line CL. First, a virtual reference line RL is defined in the vertical direction of the image. The reference line RL is used to fit the calibration line CL on the image to ensure that the UAV 1 flies along the line (i.e., the closer the reference line RL and the calibration line CL are to the UAV 1, the better).

[0083] In this example, a perpendicular line VL is calculated from the center point CP of the reference line RL to the calibration line CL, and the distance (unit: pixel) between the reference line RL and the calibration line CL can be obtained from the length of the perpendicular line VL. The angle AG between the reference line RL and the calibration line CL can be obtained from the slopes of the reference line RL and the calibration line CL. Based on the distance and the angle AG, the flight direction FD and the relative position of the unmanned aerial device 1 can be adjusted to minimize the distance and the angle AG, so as to ensure that the unmanned aerial device 1 can fly along the calibration line CL.

[0084] It should be noted that since the calibration line CL is still located where the GPS signal can be received (i.e., the GPS signal is not completely blocked), the unmanned aerial device 1 can fly to the calibration starting point SP of the calibration line CL by GPS positioning, image recognition, etc., and can obtain the following information at any time point: Figure 2 An image schematic diagram 201 is shown.

[0085] Next, in this embodiment, the processor records an actual flight path of the UAV 1 to construct a virtual map (eg: Figure 7 Operation S704 in the above description).

[0086] In certain embodiments, the present disclosure generates the virtual map based on executing a visual simultaneous localization and mapping (SLAM) operation, and the virtual map is composed of a plurality of point clouds corresponding to a plurality of locations.

[0087] For example, the processor can extract a set of image features (e.g., edges or corners) from the input image, compare the features of the image taken by the upward camera at a similar time, and minimize the re-projection error of the features. Through SLAM calculation, the image taken by the upward camera can be calculated to correspond to the coordinate value of the virtual coordinate system, which represents the position of the upward camera (or the unmanned aerial device 1) in the virtual space.

[0088] For example, see Figure 3 , which shows a schematic diagram of the output of a visual SLAM, which includes a constructed feature map 300 and an upward camera device (or unmanned aerial device 1) coordinate sequence 302. The feature map 300 is composed of a point cloud, which can present the three-dimensional state of the photographed object, and any point in the coordinate sequence 302 has a corresponding timestamp and a coordinate value of a virtual coordinate system (i.e., a mapping coordinate system), which can be used to refer to the corresponding photographed image frame. In summary, through the starting point 304 and the end point 306 of the camera coordinate sequence 302, the coordinate values ​​of the starting point and the end point of the calibration line in the virtual coordinate system can be obtained.

[0089] In addition, after the unmanned aerial device 1 completes the calibration flight, the unmanned aerial device 1 can store the SLAM calculation model as a basic map (i.e., a virtual map), wherein the basic map at least includes: at least one of or a combination of camera parameters, feature maps, coordinate sequences, and corresponding timestamps for each of multiple locations. This basic map will be used for positioning of the IS during the inspection phase.

[0090] Finally, in this embodiment, the processor calculates a coordinate transformation matrix (eg: Figure 7 Operation S705).

[0091] It should be noted that the actual flight path corresponds to a virtual coordinate system and the calibration line CL corresponds to a world coordinate system. The coordinate transformation matrix is ​​used to transform the virtual coordinate system into the world coordinate system.

[0092] Specifically, the coordinate transformation matrix includes an alignment operation of a coordinate system orientation and a coordinate system scale.

[0093] For easier understanding, please refer to Figure 4 . Figure 4 4 is a schematic diagram of the relationship between the world coordinate system 400 and the virtual coordinate system 402, which includes the calibration starting point SP and the calibration end point EP of the calibration line CL. The length of the vertical line VL can be calculated between the position of the UAV 1 and the calibration line CL, which is regarded as the distance from the calibration line CL. The flight direction FD of the UAV 1 can be calculated with the calibration line CL to form an angle AG.

[0094] In this example, set is the coordinate value of the calibration starting point SP of the calibration line CL in the world coordinate system 400, and the coordinate values ​​of a point P 404 in the space in the world coordinate system 400 and the virtual coordinate system 402 are respectively (x w ,y w ) and (x s ,y s ), the two can be connected by a rotation and scaling matrix A 2x2 (i.e., coordinate transformation matrix A) for mutual conversion.

[0095] For example, the processor may calculate the coordinate transformation matrix A using the following equation:

[0096]

[0097] In some implementations, the processor may calculate an inverse matrix using the coordinate transformation matrix A to transform the coordinates / path represented by the world coordinate system into the coordinates / path represented by the virtual coordinate system.

[0098] For example, the processor may calculate the coordinate transformation inverse matrix A using the following equation: -1 :

[0099]

[0100] It should be noted that, since the coordinate values ​​of the calibration start point SP and the calibration end point EP of the calibration line CL in the world coordinate system 400 and the virtual coordinate system 402 are known, the coordinate transformation matrix A can be obtained by singular value decomposition.

[0101] Next, the specific operation of the unmanned aerial device 1 performing the inspection stage IS in the present disclosure will be described below.

[0102] After completing the calibration phase CS, the virtual map and the coordinate transformation matrix A are obtained. The UAV 1 first loads the virtual map during the inspection phase IS (for example: Figure 7Then, the same method as in the calibration phase CS is adopted, and the processor controls the UAV 1 to fly to the calibration starting point SP of the calibration line CL to continue the map construction (e.g., perform the visual SLAM operation). The UAV 1 can calculate that the calibration starting point SP of the calibration line CL is located at the origin of the virtual coordinate system (e.g., Figure 7 Operation S707).

[0103] In addition, during actual inspections, the unmanned aerial device 1 can execute a predetermined route mission. During the flight, it can continue to construct maps and obtain the coordinates of the upward camera device (or unmanned aerial device 1) in the virtual coordinate system. Through the aforementioned conversion operation, the photographic data can be converted from the virtual coordinate system to the world coordinate system to complete the data positioning procedure.

[0104] Specifically, the processor controls the UAV 1 to fly along the virtual detection paths based on the multiple virtual detection paths and the virtual map. Then, the processor records the actual flight path of the UAV 1 to continue to construct the virtual map. Finally, the processor positions the UAV 1 based on the coordinate transformation matrix to generate multiple detection images corresponding to the target bridge and a real coordinate value of the real world corresponding to each of the detection images.

[0105] It should be noted that the detection path includes multiple checkpoints, and the checkpoints in the present disclosure are the positions where the target object to be detected (i.e., the target bridge) needs to be detected. Taking the example of bridge detection as an example, the checkpoints are important positions for detecting the safety structure of the bridge (i.e., the unmanned aerial device needs to take images corresponding to the target object to be detected from the checkpoints), and these important positions need to be transferred to world coordinates for accurate interpretation.

[0106] In addition, in the inspection task mode, the UAV 1 can first convert a series of waypoints (eg, inspection points) in the world coordinates into a virtual coordinate system through the aforementioned conversion operation. Figure 4 As shown, the starting point of this task is the origin of the virtual coordinate system (i.e., the calibration starting point SP). Given a task waypoint P404, a virtual route 406 can be constructed, and the relative position and flight direction FD of the UAV 1 can be adjusted to minimize the relative distance and relative angle between the flight direction FD of the UAV 1 and the virtual route 406 (for example: Figure 7 The above-mentioned process can be repeated to complete the flight mission of each waypoint in sequence. During the flight, the map can be continuously constructed and the coordinates of the upward camera device (or unmanned aerial device 1) in the virtual coordinate system can be obtained. Through the above-mentioned conversion operation, the photographic data can be converted from the virtual coordinate system to the world coordinate system to complete the data positioning process (for example: Figure 7 Operation S709).

[0107] In some embodiments, the processor generates an inverse coordinate transformation matrix corresponding to the coordinate transformation matrix A based on the coordinate transformation matrix A. Then, the processor transforms the plurality of detection paths corresponding to the real world based on the inverse coordinate transformation matrix to generate the virtual detection paths.

[0108] in addition, Figure 5 Example for using Figure 3 A schematic diagram of the inspection mission flight results of the basic model, wherein four sets of waypoints 500, 502, 504, 506 are pre-set. A series of flight paths 508 of the UAV can be generated by flying in the above-mentioned route mission mode, and SLAM is simultaneously executed to construct a virtual map 510 to assist in continuous positioning and navigation.

[0109] In some embodiments, the checkpoints (or detection paths) corresponding to the target object to be detected may be generated by a processor based on a plurality of detection rules (eg, a checkpoint needs to be set every 5 meters), or may be pre-set by a professional.

[0110] It should be noted that the checkpoints in the present disclosure may also be referred to as waypoints, which may include information such as longitude, latitude, altitude, drone heading or camera angle (i.e., the angle at which the drone takes photos at the waypoint), etc.

[0111] In this embodiment, the processor can perform inspection operations through visual navigation of the UAV 1 based on the coordinate transformation matrix and virtual map generated in the calibration phase CS when there is no GPS signal (ie, under the target bridge).

[0112] It can be seen from the above description that the unmanned aerial device 1 provided by the present invention constructs a virtual map by identifying the calibration line of the target bridge with known position information and the actual flight path of the unmanned aerial device 1. Then, the unmanned aerial device 1 provided by the present invention calculates the coordinate transformation matrix corresponding to the virtual map and the real world based on the actual flight path and the calibration line. In addition, the unmanned aerial device 1 provided by the present invention can be further controlled to fly along the virtual detection paths based on multiple virtual detection paths and the virtual map to detect the target bridge, and generate multiple detection images corresponding to the target bridge and the real coordinate values ​​of the real world corresponding to each of the detection images. Due to the unmanned aerial device 1 provided by the present invention, the task of detecting bridges can be automatically and efficiently performed, solving the shortcomings of the prior art.

[0113] Next, the specific operation of the second embodiment will be described in detail below. In short, in addition to the operation performed in the first embodiment, in the second embodiment, the image corresponding to the target bridge obtained by the upward camera device includes a shielded portion (i.e., no GPS signal) and an unshielded portion (i.e., with GPS signal) corresponding to a global positioning system (Global Positioning System; GPS) (for example: Figure 7 Operation S701).

[0114] For example, Figure 1 As shown, the left half of the image schematic diagram 100 is the area where the GPS signal cannot be received (i.e., the area below the target bridge is the part where the GPS signal is blocked), and the right half of the image schematic diagram 100 is the area where the GPS signal can be received (i.e., the part where the GPS signal is not blocked).

[0115] Since the UAV 1 is located at the boundary of the area where GPS signals can be received, the processor can control the UAV 1 to perform the calibration phase CS along the calibration line (i.e., from the calibration start point to the calibration end point) based on the image including the bridge bottom part and the sky part through the operation of the first embodiment. In addition, the processor performs the inspection phase IS based on the coordinate transformation matrix and the virtual map generated in the calibration phase CS.

[0116] The third embodiment of the present invention is a bridge detection method, the flow chart of which is described in Figure 6 The bridge detection method 600 is applicable to an unmanned aerial device, such as the unmanned aerial device 1 described in the first embodiment. The unmanned aerial device includes an upward camera device and a processor, such as the upward camera device and the processor described in the first embodiment. The upward camera device is used to obtain an image corresponding to a target bridge based on an upward shooting angle (for example: Figure 7 The bridge detection method 600 calculates a coordinate transformation matrix corresponding to the virtual map and the real world through steps S601 to S603.

[0117] In step S601, the UAV identifies a calibration line (eg, Figure 7 Operation S702).

[0118] Next, in step S603, the UAV is controlled to fly along the calibration line based on the image to perform a bridge inspection task (e.g.: Figure 7 Operation S703 in the figure), wherein a calibration start point and a calibration end point of the calibration line each correspond to a latitude and longitude coordinate of a real world.

[0119] In some embodiments, the bridge detection method 600 further includes the following steps: recording an actual flight path of the UAV to construct a virtual map (e.g.: Figure 7 and calculating a coordinate transformation matrix corresponding to the virtual map and the real world based on the actual flight path and the calibration line (for example: Figure 7 Operation S705).

[0120] In some embodiments, the bridge detection method 600 further includes the following steps: based on multiple virtual detection paths and the virtual map, controlling the unmanned aerial device to fly along the virtual detection paths; recording the actual flight path of the unmanned aerial device to continue to construct the virtual map; and positioning the unmanned aerial device based on the coordinate transformation matrix to generate multiple detection images corresponding to the target bridge and a real coordinate value of the real world corresponding to each of the detection images.

[0121] In some embodiments, the virtual detection paths are generated based on the following steps: based on the coordinate transformation matrix, generating an inverse coordinate transformation matrix corresponding to the coordinate transformation matrix; and based on the inverse coordinate transformation matrix, converting multiple detection paths corresponding to the real world to generate the virtual detection paths.

[0122] In some embodiments, the step of controlling the unmanned aerial device to fly along the virtual detection paths also includes the following steps: calculating an angle between a flight direction of the unmanned aerial device and the calibration line and a distance between the unmanned aerial device and the calibration line; and adjusting the flight direction of the unmanned aerial device based on the angle and the distance, wherein the angle between the flight direction and the calibration line is not greater than a preset angle value, and the distance between the unmanned aerial device and the calibration line is not greater than a preset distance value.

[0123] In addition to the above steps, the third embodiment can also perform all operations and steps of the unmanned aerial device 1 described in the first embodiment and the second embodiment, has the same functions, and achieves the same technical effects. A person skilled in the art of the present invention can directly understand how the third embodiment performs these operations and steps based on the first embodiment and the second embodiment, has the same functions, and achieves the same technical effects, so it will not be described in detail.

[0124] In summary, the bridge detection technology (at least including the device and the method) provided by the present invention constructs a virtual map by identifying the calibration line of the target bridge with known location information and the actual flight path of the unmanned aerial device. Then, the bridge detection technology provided by the present invention calculates the coordinate transformation matrix corresponding to the virtual map and the real world based on the actual flight path and the calibration line. In addition, the bridge detection technology provided by the present invention can be based on multiple virtual detection paths and the virtual map, control the unmanned aerial device to fly along the virtual detection paths to detect the target bridge, and generate multiple detection images corresponding to the target bridge and the real coordinate values ​​of the real world corresponding to each of the detection images. Due to the bridge detection technology provided by the present invention, the task of detecting bridges can be automatically and efficiently performed, solving the shortcomings of the prior art.

[0125] The above embodiments are only used to illustrate some embodiments of the present invention and to explain the technical features of the present invention, and are not used to limit the protection scope and range of the present invention. Any changes or equivalent arrangements that can be easily completed by technicians in the technical field to which the present invention belongs to the scope claimed by the present invention, and the scope of protection of the present invention shall be subject to the claims.

Claims

1. An unmanned aerial device for bridge inspection, comprising: An upward camera device, used for acquiring an image corresponding to a target bridge based on an upward shooting angle, wherein the image includes a shielded portion and an unshielded portion corresponding to a global positioning system; and A processor is electrically connected to the upward camera device and is used to perform the following operations: identifying a calibration line in the image corresponding to the target bridge; and Based on the image including the shielded portion and the unshielded portion, the unmanned aerial device is controlled to fly along the calibration line to perform a bridge inspection task, wherein a calibration start point and a calibration end point of the calibration line respectively correspond to latitude and longitude coordinates in the real world.

2. The unmanned aerial vehicle for bridge inspection according to claim 1, wherein controlling the unmanned aerial vehicle to fly along the calibration line comprises the following operations: Calculating the angle between the flight direction of the UAV and the calibration line and the distance between the UAV and the calibration line; and Based on the angle and the distance, the flight direction of the unmanned aerial device is adjusted, wherein the angle between the flight direction and the calibration line is not greater than a preset angle value, and the distance between the unmanned aerial device and the calibration line is not greater than a preset distance value.

3. The unmanned aerial device for bridge inspection according to claim 1, wherein the processor further performs the following operations: Recording the actual flight path of the UAV to construct a virtual map; and Based on the actual flight path and the calibration line, a coordinate transformation matrix corresponding to the virtual map and the real world is calculated. 4 . The unmanned aerial device for bridge inspection according to claim 3 , wherein the actual flight path corresponds to a virtual coordinate system and the calibration line corresponds to a world coordinate system, and the coordinate transformation matrix is ​​used to transform the virtual coordinate system into the world coordinate system. 5 . The unmanned aerial device for bridge inspection according to claim 3 , wherein the coordinate transformation matrix comprises an alignment operation of a coordinate system direction and a coordinate system scale.

6. The unmanned aerial device for bridge inspection according to claim 3, wherein the processor further performs the following operations: Based on a plurality of virtual detection paths and the virtual map, controlling the unmanned aerial device to fly along the plurality of virtual detection paths; Recording the actual flight path of the UAV to continue constructing the virtual map; and Based on the coordinate transformation matrix, the unmanned aerial device is positioned to generate a plurality of detection images corresponding to the target bridge and a real coordinate value of the real world corresponding to each of the plurality of detection images.

7. The unmanned aerial device for bridge inspection according to claim 6, wherein the plurality of virtual inspection paths are generated based on the following operations: Based on the coordinate transformation matrix, generating an inverse coordinate transformation matrix corresponding to the coordinate transformation matrix; and Based on the coordinate transformation inverse matrix, a plurality of detection paths corresponding to the real world are transformed to generate the plurality of virtual detection paths. 8 . The unmanned aerial device for bridge inspection according to claim 3 , wherein the virtual map is generated based on performing visual simultaneous positioning and map building operations, and the virtual map is composed of a plurality of point clouds corresponding to a plurality of locations. 9 . The unmanned aerial device for bridge inspection according to claim 3 , wherein the virtual map comprises at least one of or a combination of camera parameters, feature maps, coordinate sequences, and time stamps of each of a plurality of locations.

10. An unmanned aerial device for bridge inspection, comprising: An upward camera device, used for acquiring an image corresponding to a target bridge based on an upward shooting angle, wherein the image includes a bottom portion and a sky portion corresponding to the target bridge; and A processor is electrically connected to the upward camera device and is used to perform the following operations: identifying a calibration line in the image corresponding to the target bridge; and Based on the image including the bridge bottom part and the sky part, the unmanned aerial device is controlled to fly along the calibration line to perform a bridge inspection task, wherein a calibration start point and a calibration end point of the calibration line respectively correspond to latitude and longitude coordinates in the real world.

11. The unmanned aerial vehicle for bridge inspection according to claim 10, wherein controlling the unmanned aerial vehicle to fly along the calibration line comprises the following operations: Calculating the angle between the flight direction of the UAV and the calibration line and the distance between the UAV and the calibration line; and Based on the angle and the distance, the flight direction of the unmanned aerial device is adjusted, wherein the angle between the flight direction and the calibration line is not greater than a preset angle value, and the distance between the unmanned aerial device and the calibration line is not greater than a preset distance value.

12. The unmanned aerial device for bridge inspection according to claim 10, wherein the processor further performs the following operations: Recording the actual flight path of the UAV to construct a virtual map; and Based on the actual flight path and the calibration line, a coordinate transformation matrix corresponding to the virtual map and the real world is calculated. 13 . The unmanned aerial device for bridge inspection according to claim 12 , wherein the actual flight path corresponds to a virtual coordinate system and the calibration line corresponds to a world coordinate system, and the coordinate transformation matrix is ​​used to transform the virtual coordinate system into the world coordinate system. 14 . The unmanned aerial device for bridge inspection according to claim 12 , wherein the coordinate transformation matrix comprises an alignment operation of a coordinate system direction and a coordinate system scale.

15. The unmanned aerial device for bridge inspection according to claim 12, wherein the processor further performs the following operations: Based on a plurality of virtual detection paths and the virtual map, controlling the unmanned aerial device to fly along the plurality of virtual detection paths; Recording the actual flight path of the UAV to continue constructing the virtual map; and Based on the coordinate transformation matrix, the unmanned aerial device is positioned to generate a plurality of detection images corresponding to the target bridge and a real coordinate value of the real world corresponding to each of the plurality of detection images.

16. The unmanned aerial device for bridge inspection according to claim 15, wherein the plurality of virtual inspection paths are generated based on the following operations: Based on the coordinate transformation matrix, generating an inverse coordinate transformation matrix corresponding to the coordinate transformation matrix; and Based on the coordinate transformation inverse matrix, a plurality of detection paths corresponding to the real world are transformed to generate the plurality of virtual detection paths. 17 . The unmanned aerial device for bridge inspection according to claim 12 , wherein the virtual map is generated based on performing visual simultaneous localization and map building operations, and the virtual map is composed of a plurality of point clouds corresponding to a plurality of locations. 18 . The unmanned aerial device for bridge inspection according to claim 12 , wherein the virtual map comprises at least one of or a combination of camera parameters, feature maps, coordinate sequences, and time stamps of each of a plurality of locations.

19. A bridge detection method for an unmanned aerial device, wherein the unmanned aerial device comprises an upward camera device and a processor, wherein the upward camera device is used to obtain an image corresponding to a target bridge based on an upward shooting angle, wherein the bridge detection method comprises the following steps: identifying a calibration line in the image corresponding to the target bridge; and Based on the image, the unmanned aerial device is controlled to fly along the calibration line to perform a bridge inspection task, wherein a calibration start point and a calibration end point of the calibration line respectively correspond to latitude and longitude coordinates in the real world.

20. The bridge inspection method according to claim 19, wherein the step of controlling the unmanned aerial device to fly along the calibration line further comprises the following steps: Calculating the angle between the flight direction of the UAV and the calibration line and the distance between the UAV and the calibration line; and Based on the angle and the distance, the flight direction of the unmanned aerial device is adjusted, wherein the angle between the flight direction and the calibration line is not greater than a preset angle value, and the distance between the unmanned aerial device and the calibration line is not greater than a preset distance value.

Images