Infrastructure safety inspection system using artificial intelligence-based unmanned aerial vehicle for autonomous flight in GPS shaded area

Abstract

The present invention relates to an infrastructure safety inspection system using an artificial intelligence-based unmanned aerial vehicle for autonomous flight in a GPS shaded area. The infrastructure safety inspection system using an artificial intelligence-based unmanned aerial vehicle for autonomous flight in a GPS shaded area, according to the present invention, comprises: a replaceable variable rotor; an optical camera; a thermal imaging camera; a movable lidar sensor; a gyro sensor; an acceleration sensor; a temperature sensor; a three-axis laser range finder; an infrared sensor; an ultrasonic sensor; an RF sensor; an assembly frame; a remote controller capable of controlling a UAV; a controller for performing wired / wireless remote control of equipment and a photographing device; a control system; and a control unit in which an algorithm for controlling the artificial intelligence-based unmanned aerial vehicle for autonomous flight in a GPS shaded area is installed.

Description

Infrastructure safety inspection system using AI-based autonomous unmanned aerial vehicles for GPS blind spots

[0001] The present invention relates to an infrastructure facility safety inspection system using an artificial intelligence-based autonomous flight unmanned aerial vehicle in GPS blind spots.

[0002] This invention was derived from research conducted with the support of the 2023 Daejeon-type Convergence New Industry Creation Special Zone Technology Demonstration Leading Project "Demonstration of a Subscription Service for Safety Inspection of XAI-based Infrastructure Facilities" (Research period: December 5, 2023 – November 4, 2024), funded by Daejeon Metropolitan City and the Daejeon Science & Industry Promotion Agency.

[0003] According to conventional technology, an unmanned aerial vehicle (UAV) uses four or six rotors to approach the surface of the superstructure and substructure of a bridge facility and inspect the exterior of the bridge facility. However, when flying in a GPS blind spot under the bridge superstructure, the autonomous flight function does not operate and the performance of the obstacle detection function is degraded, leading to problems such as the unmanned aerial vehicle colliding with the surface of the bridge superstructure and substructure or degrading image quality.

[0004] The present invention is proposed to solve the aforementioned problems and aims to provide an infrastructure facility safety inspection system using an AI-based autonomous unmanned aerial vehicle capable of identifying surrounding conditions in real time based on multiple LiDAR sensors, even when flying in GPS blind spots.

[0005] The infrastructure facility safety inspection system using an artificial intelligence-based GPS blind spot autonomous flight unmanned aerial vehicle according to the present invention includes: a replaceable variable rotor; an optical camera; a thermal imaging camera; a movable lidar sensor; a gyroscope sensor; an accelerometer sensor; a temperature sensor; a 3-axis laser rangefinder; an infrared sensor; an ultrasonic sensor; an RF sensor; an assembled frame; a remote controller capable of controlling the UAV; a controller that performs wired or wireless remote control of equipment and a shooting device; a control system; and a control unit equipped with an algorithm for controlling the artificial intelligence-based GPS blind spot autonomous flight unmanned aerial vehicle.

[0006] The control unit analyzes signals received from the mobile lidar sensor, optical camera, and various sensors, and is equipped with a Fused Flow function that intelligently analyzes the position and path of the autonomous unmanned aerial vehicle, performs AI-based 3D point cloud, sensing data, and image analysis, and performs control for obstacle avoidance, collision prevention with obstacles, and direction change based on a Map-Based System.

[0007] The above control unit controls commands to display 3D point cloud and positioning data of infrastructure facilities and surrounding conditions on the monitoring equipment screen of the control system, and provides a function to display the predicted movement path of the autonomous unmanned aerial vehicle.

[0008] The above variable rotor is provided in a preset number of replaceable types within three types so as to be able to respond to surrounding environments, such as the height, width, and shape of infrastructure facilities including wind speed.

[0009] It performs AI-based 3D point cloud, sensing data, and video analysis, and forms a Safety Zone at a certain distance from infrastructure facilities and obstacles to account for the positioning error of the UAV in order to perform obstacle avoidance, collision prevention with obstacles, and direction change, and provides an infographic related to the formed Safety Zone to the user, while also providing a function for the autonomous unmanned aerial vehicle to return to the starting point without colliding with obstacles in the event of an emergency.

[0010] The control unit explains the reasons for the estimated cracks and defects in text based on a Feature Ablation algorithm, and highlights the key features serving as the basis for the crack and defect estimation on the image as a heatmap (a contour-shaped heat map).

[0011] The above-mentioned AI-based GPS blind spot autonomous flight unmanned aerial vehicle includes a bridge facility-specific positioning system composed of a gyroscope sensor, accelerometer, temperature sensor, 3-axis laser rangefinder, 3D LiDAR, a vision camera for obstacle recognition, and a multi-modal map-based algorithm attached for positioning under the bridge.

[0012] It includes a multi-modal map-based 3D simulator for performing sub-functions including collision prevention with bridges, location identification and mapping, obstacle avoidance, and path planning for autonomous flight unmanned aerial vehicles in GPS blind spots based on the above artificial intelligence.

[0013] The above 3D simulator includes a function to set the location of the approach point, obstacles, start, and destination of the AI-based GPS blind spot autonomous flight unmanned aerial vehicle, and a function to upload the entire 2D / 3D map, thereby implementing Sim-to-Real generalization.

[0014] The above control unit performs control using a deep learning algorithm that outputs the turning angle and collision status of the UAV using a single 2D color image as input in a GPS blind spot.

[0015] According to the present invention, the artificial intelligence-based 3D point cloud and sensing data analysis algorithm analyzes data obtained through a LiDAR sensor to detect obstacles present around a bridge facility, automatically sets an optimal path to avoid obstacles while flying, and prevents collision with obstacles. By equipping this algorithm to an intelligent unmanned aerial vehicle, it has the effect of quickly identifying the condition around the bridge facility and providing a safe and highly reliable bridge facility safety inspection function.

[0016] The effects of the present invention are not limited to those mentioned above, and other unmentioned effects will be clearly understood by those skilled in the art from the description below.

[0017] FIG. 1 illustrates a positioning system dedicated to the lower part of a bridge structure, comprising an IMU (inertial navigation unit), a gyroscope sensor, a temperature sensor, a 3-axis laser rangefinder, a 3D LiDAR, a vision camera for obstacle recognition, and a multi-modal map-based artificial intelligence algorithm attached to the UAV body for positioning of the UAV according to an embodiment of the present invention.

[0018] FIG. 2 is a system configuration diagram of an intelligent unmanned aerial vehicle device for a bridge facility according to an embodiment of the present invention.

[0019] FIG. 3 illustrates an intelligent robot camera dedicated to bridge facilities according to an embodiment of the present invention.

[0020] FIG. 4 is a configuration diagram of a subscription-based artificial intelligence-based infrastructure facility safety inspection platform, such as bridge facilities, according to an embodiment of the present invention.

[0021] FIG. 5 is a UI of a subscription-based artificial intelligence-based infrastructure facility safety inspection platform, such as bridge facilities, according to an embodiment of the present invention.

[0022] FIG. 6 is a configuration diagram of an explainable artificial intelligence model according to an embodiment of the present invention.

[0023] Figure 7 is an image output result of a Feature Ablation explainable artificial intelligence model according to an embodiment of the present invention (Center: correct answer, Right: output result).

[0024] FIG. 8 is a configuration diagram of a Feature Ablation explainable artificial intelligence model and a data processing / visualization process according to an embodiment of the present invention.

[0025] FIG. 9 is an image output result of a Feature Ablation explainable artificial intelligence model according to an embodiment of the present invention (Center: correct answer, Right: output result).

[0026] Figure 10 is a feature ablation visualization process according to an embodiment of the present invention.

[0027] FIG. 11 is a block diagram showing a computer system for implementing a method according to an embodiment of the present invention.

[0028] FIG. 12 illustrates a process for outputting the positioning, turning angle, and collision status of a UAV in a GPS denied zone according to an embodiment of the present invention, using various sensing data as input.

[0029] FIG. 13 illustrates a conceptual diagram of a deep learning algorithm that outputs the turning angle and collision status of a UAV using a single image as input in a GPS shadow area according to an embodiment of the present invention.

[0030] The aforementioned objectives of the present invention, as well as other objectives, advantages, and features, and the methods for achieving them, will become clear from the embodiments described in detail below together with the accompanying drawings.

[0031] However, the present invention is not limited to the embodiments disclosed below but can be implemented in various different forms, and the following embodiments are provided merely to easily inform those skilled in the art of the purpose, structure, and effects of the invention, and the scope of the rights of the present invention is defined by the description in the claims.

[0032] Meanwhile, the terms used in this specification are for describing the embodiments and are not intended to limit the invention. In this specification, the singular form includes the plural form unless specifically stated otherwise in the text. As used in this specification, "comprises" and / or "comprising" do not exclude the presence or addition of one or more other components, steps, actions, and / or elements to the mentioned components, steps, actions, and / or elements.

[0033] FIG. 1 illustrates a positioning system dedicated to the underside of a bridge facility, composed of an IMU (Inertial Navigation Unit), a gyroscope, a temperature sensor, a 3-axis laser rangefinder, a 3D LiDAR, a vision camera for obstacle recognition, and a multi-modal map-based artificial intelligence algorithm attached to the UAV body for positioning of the UAV according to an embodiment of the present invention. FIG. 2 is a system configuration diagram of an intelligent unmanned aerial vehicle device for a bridge facility according to an embodiment of the present invention. FIG. 3 illustrates an intelligent robot camera equipment dedicated to a bridge facility according to an embodiment of the present invention. FIG. 4 is a configuration diagram of a subscription-based artificial intelligence-based infrastructure facility safety inspection platform, such as a bridge facility, according to an embodiment of the present invention. FIG. 5 is a UI of a subscription-based artificial intelligence-based infrastructure facility safety inspection platform, such as a bridge facility, according to an embodiment of the present invention. FIG. 6 is a configuration diagram of an explainable artificial intelligence model according to an embodiment of the present invention. FIG. 7 is an image output result of a Feature Ablation explainable artificial intelligence model according to an embodiment of the present invention (Center: Correct Answer, Right: Output Result). FIG. 8 is a configuration diagram and data processing / visualization process of a Feature Ablation explainable artificial intelligence model according to an embodiment of the present invention. FIG. 9 is an image output result of a Feature Ablation explainable artificial intelligence model according to an embodiment of the present invention (Center: Correct answer, Right: Output result). FIG. 10 is a Feature Ablation visualization process according to an embodiment of the present invention. FIG. 11 is a block diagram showing a computer system for implementing a method according to an embodiment of the present invention. FIG. 12 illustrates a process for outputting the positioning, turning angle, and collision status of a UAV using various sensing data as input in a GPS denied zone according to an embodiment of the present invention. FIG. 13 illustrates a conceptual diagram of a deep learning algorithm for outputting the turning angle and collision status of a UAV using a single image as input in a GPS denied zone according to an embodiment of the present invention.

[0034] The infrastructure facility safety inspection system using an artificial intelligence-based GPS blind spot autonomous flight unmanned aerial vehicle according to the present invention includes: a replaceable variable rotor; an optical camera; a thermal imaging camera; a movable lidar sensor; a gyroscope sensor; an accelerometer sensor; a temperature sensor; a 3-axis laser rangefinder; an infrared sensor; an ultrasonic sensor; an RF sensor; an assembled frame; a remote controller capable of controlling the UAV; a controller that performs wired or wireless remote control of equipment and a shooting device; a control system; and a control unit equipped with an algorithm for controlling the artificial intelligence-based GPS blind spot autonomous flight unmanned aerial vehicle.

[0035] The control unit analyzes signals received from the mobile lidar sensor, optical camera, and various sensors, and is equipped with a Fused Flow function that intelligently analyzes the position and path of the autonomous unmanned aerial vehicle, performs AI-based 3D point cloud, sensing data, and image analysis, and performs control for obstacle avoidance, collision prevention with obstacles, and direction change based on a Map-Based System.

[0036] The above control unit controls commands to display 3D point cloud and positioning data of infrastructure facilities and surrounding conditions on the monitoring equipment screen of the control system, and provides a function to display the predicted movement path of the autonomous unmanned aerial vehicle.

[0037] The above variable rotor is provided in a preset number of replaceable types within three types so as to be able to respond to surrounding environments, such as the height, width, and shape of infrastructure facilities including wind speed.

[0038] It performs AI-based 3D point cloud, sensing data, and video analysis, and forms a Safety Zone at a certain distance from infrastructure facilities and obstacles to account for the positioning error of the UAV in order to perform obstacle avoidance, collision prevention with obstacles, and direction change, and provides an infographic related to the formed Safety Zone to the user, while also providing a function for the autonomous unmanned aerial vehicle to return to the starting point without colliding with obstacles in the event of an emergency.

[0039] The control unit explains the reasons for the estimated cracks and defects in text based on a Feature Ablation algorithm, and highlights the key features serving as the basis for the crack and defect estimation on the image as a heatmap (a contour-shaped heat map).

[0040] The above-mentioned AI-based GPS blind spot autonomous flight unmanned aerial vehicle includes a bridge facility-specific positioning system composed of a gyroscope sensor, accelerometer, temperature sensor, 3-axis laser rangefinder, 3D LiDAR, a vision camera for obstacle recognition, and a multi-modal map-based algorithm attached for positioning under the bridge.

[0041] It includes a multi-modal map-based 3D simulator for performing sub-functions including collision prevention with bridges, location identification and mapping, obstacle avoidance, and path planning for autonomous flight unmanned aerial vehicles in GPS blind spots based on the above artificial intelligence.

[0042] The above 3D simulator includes a function to set the location of the approach point, obstacles, start, and destination of the AI-based GPS blind spot autonomous flight unmanned aerial vehicle, and a function to upload the entire 2D / 3D map, thereby implementing Sim-to-Real generalization.

[0043] The above control unit performs control using a deep learning algorithm that outputs the turning angle and collision status of the UAV using a single 2D color image as input in a GPS blind spot.

[0044]

[0045] Feature Ablation Explainable Artificial Intelligence Algorithms

[0046] To analyze and understand semantic segmentation AI models, we interpret the semantic segmentation model using the XAI Feature Ablation algorithm. The existing semantic segmentation model DeepLabv3 was trained on dam facilities and inference was performed. The architecture of DeepLabv3 consists of a spatial pyramid pooling module and an encoder-decoder structure with atrous convolution.

[0047] The trained Deeplabv3 model was saved and tested on the dam validation data. Performance evaluation was also performed.

[0048] Among multiple semantic segmentation metrics, Mean IOU (Intersection over Union) is used. IOU is the intersection area of ​​the union of the predicted segmentation and the Ground Truth, and Mean IOU takes the average value for each class when there are multiple defects in an image.

[0049] Recently, we have been interpreting a semantic segmentation model called deepLabv3 using the Feature Ablation XAI algorithm within the interpretability library of the platform 'Captum'. The purpose of interpreting this model is to understand which pixels and regions of the image input to the semantic segmentation task contribute to a specific class (target class).

[0050] For example, as shown in the figure below, feature ablation can be used to visualize how specific areas of an image affect white film defect prediction. The results suggest that the image background contributes positively to delivery prediction.

[0051] Leakage and peeling do not appear to affect the prediction of white film.

[0052]

[0053] RPN-based artificial intelligence algorithm

[0054] RPN is an algorithm that uses a Region Proposal Network to extract features of objects contained in an image, recommends multiple candidate bounding boxes based on those features, and then infers the final detection result by progressively reducing the number of recommended bounding boxes based on probability.

[0055] To explain the process in more detail, first, the calculation of feature extraction for each Region of Interest (RoI) is shared, and it can be implemented by introducing a deep learning-based RPN.

[0056] After extracting feature maps using many CNN layers, a large number of windows likely to contain objects are generated via RPN. The algorithm then searches the feature maps within each window, scales them to a fixed size (RoI pooling), and predicts class probabilities and more accurate bounding boxes for the corresponding objects.

[0057] One point to consider here is how RPN generates windows, as it uses anchor boxes. However, the difference from other similar algorithms is that these anchor boxes are generated with a fixed size and shape rather than from the data. These anchor boxes can cover the image more densely, and RPN performs only binary classification based on whether an object is contained within the window, instead of classifying objects into multiple categories.

[0058]

[0059] Feature Pyramid Network

[0060] A Feature Pyramid Network (FPN) is a network that inputs a single-scale image of arbitrary size into a convolutional network and outputs feature maps of various scales. Generally, proven networks such as AlexNet, GooglNet, and ResNet are used. An FPN can be described as a network that extracts and modifies feature maps for each layer of various sizes specified in the existing convolutional network. The process by which an FPN extracts feature maps and constructs a pyramid includes bottom-up pathways, top-down pathways, and lateral connections. While using an FPN offers the advantage of detecting damage of various sizes contained in an image and extracting various features such as boundaries and internal patterns within the damage, computation time increases as the number of layers in the FPN increases. Therefore, in this invention, a 4-layer FPN structure was applied to satisfy both computation time and object detection accuracy.

[0061]

[0062] Explainable AI module

[0063] XAI (eXplainable AI) refers to explainable artificial intelligence. Its purpose is to generate 'visual explanations' and 'captions' regarding the reliability of a model and its decisions.

[0064] Grad-CAM is a technology that generates 'visual descriptions' from CNN-based models to explain the structure of a system in a way that humans can understand, meaning it possesses 'interpretability'.

[0065] CAM (Class Activation Mapping) provides a visual explanation when determining many classes in CNN-based networks.

[0066] The Grad-CAM (Gradient-weighted CAM) mentioned here uses the gradient for the "target class (caption, mask also possible)" which "flows into the final convolution to produce an approximate local map that highlights important parts within the predicted image" when calculating the CAM.

[0067] Image Captioning is research that connects the two major currents of the artificial intelligence field: 'Computer Vision' and 'Natural Language Processing'.

[0068] Image Captioning adopts an Encoder-Decoder structure in which the original image is encoded using a CNN as the Encoder and the caption part is decoded using an LSTM as the Decoder.

[0069] When training the Encoder, the pre-trained classification model is fine-tuned using a transfer learning algorithm, the received model is imported, and the final fully connected layer is not used.

[0070] When training the Decoder, the embedding vector output from the Encoder is passed through the LSTM to output each word of the sentence in order.

[0071]

[0072] Intensity Optimizer

[0073] Just as human cognitive abilities are affected by changes in lighting, the performance of AI models is also influenced by changes in image brightness. For example, an AI model primarily trained using high-brightness images taken in a well-lit environment experiences a significant decline in object detection performance when low-brightness images taken in a dark environment are input. Therefore, when testing a new, untrained image after building an AI model trained on images within a specific target brightness range (e.g., 150–200), high performance can be expected by analyzing the brightness spectrum of the image and converting it to the specific target brightness range.

[0074]

[0075] Spot Noise Filter

[0076] Images captured by optical cameras contain point-shaped noise (generated during the stepwise electrical signal processing) for various reasons. Since this noise degrades the object detection performance of deep learning models, it is removed in advance using a noise filter dedicated to point shapes.

[0077]

[0078] A platform that can be executed via a web browser

[0079] The platform is not affected by the execution location and can be run in a web browser without the need for separate software installation; it possesses a list of functions necessary for the entire process, including UAV control, video monitoring (control), and video processing.

[0080]

[0081] Deep learning algorithms and SVM based on attention mechanism

[0082] The basic idea of ​​Attention (in the field of natural language processing) is that at every time step where the decoder predicts an output word, the entire input sentence from the encoder is referenced once again.

[0083] However, instead of referring to the entire input sentence at the same rate, it focuses more attention on the parts of the input words that are related to the word to be predicted at that point.

[0084] The process of the attention mechanism is as follows.

[0085] 1) Calculate the Attention Score.

[0086] 2) Obtain the attention distribution using the softmax function.

[0087] 3) Obtain the attention value by weighting the attention weights and hidden states of each encoder.

[0088] 4) Concatenate the attention value and the decoder's hidden state at time t.

[0089] The fundamental idea behind attention in the field of image processing is that the receptive field of a module can be expanded by effectively utilizing spatial attention. In other words, by appropriately using spatial attention to increase the receptive field, it is possible to develop a network that outperforms existing networks with very little additional computation.

[0090] Attention plays a crucial role in improving the performance of recent networks. In fact, the Residual Attention Network (RAN) achieved better performance than the existing Residual Network by stacking attention modules multiple times.

[0091] By creating a multi-stage receptive field using an attention module, properties that are invariant to translation and scale can be obtained.

[0092] SVM is a binary classifier that classifies two categories.

[0093] The main objective is to find a hyperplane, which is the optimal decision boundary that maximizes the margin between the two classes.

[0094] Observations that support the hyperplane are called support vectors, and the margin refers to the distance between the decision boundary and the support vectors.

[0095] SVM must maximize the margin while correctly separating data points, so ultimately, it is important to handle outliers well.

[0096] If you set decision boundaries based on criteria that do not allow outliers in an attempt to avoid missing individual training data, an overfitting problem may occur.

[0097] In SVM, since support vectors ultimately define the decision boundary, numerous unnecessary data points can be ignored by effectively selecting only the support vectors from the data points; this characteristic is the reason why SVM runs very fast.

[0098]

[0099] Transfer learning algorithm

[0100] It is a concept of transferring a deep learning model trained on a specific task (Classification, Detection, Segmentation, etc.) to another task to train the model.

[0101] Transfer learning refers to utilizing parts of a neural network trained in a specific field to train neural networks used in similar or completely different fields. It is a method that accelerates training speed and improves accuracy when building a new model using an existing one. It is particularly helpful when working on projects with limited training datasets.

[0102] The types of transfer learning include the following.

[0103] 1) Fine-tune: Training is performed by updating only the weights of the last FCL (Fully Connected Layer) of the pre-trained ConvNet (Convolutional Network).

[0104] 2) Pre-trained model: Apply the weights of a pre-trained model to the new model.

[0105] 3) Domain adaptation: When training based on rich data, the domain distinction ability is learned weakly to build a model capable of classifying target data.

[0106] 4) Layer reuse: Reuse some layers of an existing model to build models with insufficient data.

[0107]

[0108] FIG. 12 is a block diagram showing a computer system for implementing a method according to an embodiment of the present invention.

[0109] Referring to FIG. 12, a computer system (1300) may include at least one of a processor (1310), memory (1330), an input interface device (1350), an output interface device (1360), and a storage device (1340) that communicate via a bus (1370). The computer system (1300) may also include a communication device (1320) coupled to a network. The processor (1310) may be a central processing unit (CPU) or a semiconductor device that executes instructions stored in memory (1330) or storage device (1340). Memory (1330) and storage device (1340) may include various forms of volatile or non-volatile storage media. For example, memory may include read-only memory (ROM) and random access memory (RAM). In the embodiments of this description, memory may be located inside or outside the processor, and memory may be connected to the processor through various known means. Memory is a volatile or non-volatile storage medium of various forms, and for example, memory may include read-only memory (ROM) or random access memory (RAM).

[0110] Accordingly, embodiments of the present invention may be implemented as a method implemented on a computer or as a non-transient computer-readable medium storing computer-executable instructions. In one embodiment, when executed by a processor, the computer-readable instructions may perform a method according to at least one aspect of the present description.

[0111] The communication device (1320) can transmit or receive wired or wireless signals.

[0112] In addition, the method according to an embodiment of the present invention may be implemented in the form of program instructions that can be executed through various computer means and may be recorded on a computer-readable medium.

[0113] The above computer-readable medium may include program instructions, data files, data structures, etc., either individually or in combination. The program instructions recorded on the computer-readable medium may be specially designed and configured for embodiments of the present invention, or they may be known and available to a person skilled in the art of computer software. The computer-readable recording medium may include a hardware device configured to store and execute program instructions. For example, the computer-readable recording medium may be magnetic media such as hard disks, floppy disks, and magnetic tapes; optical recording media such as CD-ROMs and DVDs; magneto-optical media such as floptical disks; ROM; RAM; flash memory, etc. The program instructions may include not only machine code, such as that generated by a compiler, but also high-level language code that can be executed by a computer through an interpreter, etc.

[0114] Although embodiments of the present invention have been described in detail above, the scope of the present invention is not limited thereto, and various modifications and improvements by those skilled in the art using the basic concept of the present invention as defined in the following claims also fall within the scope of the present invention.

Claims

1. Replaceable variable rotor; Optical camera; Thermal imaging camera; Mobile LiDAR sensor; Gyroscope sensor; Accelerometer; Temperature sensor; 3-axis laser rangefinder; Infrared sensor; Ultrasonic sensor; RF sensor; Prefabricated frame; A remote controller capable of controlling a UAV; A controller that performs wired and wireless remote control of equipment and imaging devices; Control system; and A control unit equipped with an algorithm for controlling an AI-based autonomous unmanned aerial vehicle in GPS blind spots. Infrastructure facility safety inspection system using an AI-based autonomous unmanned aerial vehicle for GPS blind spots, including 2. In Paragraph 1, The above control unit analyzes signals received from the mobile LiDAR sensor, optical camera, and various sensors, and is equipped with a Fused Flow function that intelligently analyzes the position and path of the autonomous unmanned aerial vehicle, performs AI-based 3D point cloud, sensing data, and image analysis, and performs control for obstacle avoidance, collision prevention with obstacles, and direction change based on a Map-Based System. Infrastructure facility safety inspection system using AI-based autonomous unmanned aerial vehicles for GPS blind spots.

3. In Paragraph 1, The above control unit controls commands to display 3D point cloud and positioning data of infrastructure facilities and surrounding conditions on the monitoring equipment screen of the control system, and provides a function to display the predicted movement path of the autonomous unmanned aerial vehicle. Infrastructure facility safety inspection system using AI-based autonomous unmanned aerial vehicles for GPS blind spots.

4. In Paragraph 1, The above variable rotor is provided in a preset number of replaceable types, up to three types, to respond to surrounding environments such as the height, width, and shape of infrastructure facilities including wind speed. Infrastructure facility safety inspection system using AI-based autonomous unmanned aerial vehicles for GPS blind spots.

5. In Paragraph 2, Performing AI-based 3D point cloud, sensing data, and image analysis, and forming a Safety Zone at a certain distance from infrastructure facilities and obstacles to account for the positioning error of the UAV in order to perform obstacle avoidance, collision prevention with obstacles, and control for changing direction, and providing an infographic related to the formed Safety Zone to the user, while simultaneously providing a function for the autonomous unmanned aerial vehicle to return to the starting point without colliding with obstacles in the event of an emergency. Infrastructure facility safety inspection system using AI-based autonomous unmanned aerial vehicles for GPS blind spots.

6. In Paragraph 1, The above control unit explains the reasons for estimating cracks and defects in text based on a Feature Ablation algorithm, and highlights key features serving as the basis for the crack and defect estimation on the image as a heatmap (a contour-shaped heat map). Infrastructure facility safety inspection system using AI-based autonomous unmanned aerial vehicles for GPS blind spots.

7. In Paragraph 1, The above-mentioned AI-based GPS blind spot autonomous unmanned aerial vehicle includes a positioning system dedicated to bridge facilities, comprising a gyroscope, accelerometer, temperature sensor, 3-axis laser rangefinder, 3D LiDAR, a vision camera for obstacle recognition, and a multi-modal map-based algorithm attached for positioning under the bridge. Infrastructure facility safety inspection system using AI-based autonomous unmanned aerial vehicles for GPS blind spots.

8. In Paragraph 1, Includes a multi-modal map-based 3D simulator for performing sub-functions including collision prevention with bridges, position determination and mapping, obstacle avoidance, and path planning for the above-mentioned AI-based autonomous flight unmanned aerial vehicle in GPS blind spots. Infrastructure facility safety inspection system using AI-based autonomous unmanned aerial vehicles for GPS blind spots.

9. In Paragraph 8, The above 3D simulator includes the function of setting the locations of the approach point, obstacles, start, and destination of the AI-based GPS blind spot autonomous flight unmanned aerial vehicle, and the function of uploading the entire 2D / 3D map to implement Sim-to-Real generalization. Infrastructure facility safety inspection system using AI-based autonomous unmanned aerial vehicles for GPS blind spots.

10. In Paragraph 1, The above control unit performs control using a deep learning algorithm that outputs the turning angle and collision status of the UAV using a single 2D color image as input in a GPS blind spot. Infrastructure facility safety inspection system using AI-based autonomous unmanned aerial vehicles for GPS blind spots.

Images