Subscription-based software and generative artificial intelligence-powered safety inspection system for internal tunnels and reservoir intake facilities of dam infrastructure
The subscription-based AI system with deformable wheels and advanced sensors addresses obstacle recognition issues in CCTV robots, ensuring comprehensive and safe inspections of dam facilities by using AI for real-time obstacle avoidance and defect detection.
Patent Information
- Application Number
- KR1020250212736
- Authority / Receiving Office
- KR · KR
- Patent Type
- Patents
- Current Assignee / Owner
- Priority Date
- 2024-12-30
- Filing Date
- 2025-12-29
- Publication Date
- 2026-11-06
- Estimated Expiration
- 2045-12-29
AI Technical Summary
Conventional CCTV camera robots used for inspecting reservoir water intake facilities in dam facilities struggle with obstacle recognition, often getting stuck or unable to move due to debris, leading to incomplete inspections.
A subscription-based software and generative artificial intelligence system equipped with an optical camera, thermal imaging camera, 3D LiDAR, and a deformable 3-axis wheel system, utilizing AI for obstacle avoidance and defect detection, including a 3D point cloud analysis and BIM-SLAM for precise mapping and inspection.
Enables intelligent, safe, and reliable inspection of dam facilities by identifying obstacles and defects in real-time, providing accurate defect detection and mapping, even in challenging environments.
Smart Images

Figure 112025148282310-PAT00001_ABST
Abstract
Description
Technology Field
[0001] The present invention relates to a subscription-based software and generative artificial intelligence-based safety inspection system for internal tunnels and reservoir water intake facilities within dam facilities. Background Technology
[0003] According to conventional technology, a CCTV camera robot used to inspect and examine the interior of a reservoir water intake facility (culvert) is positioned on the inner floor of the culvert using two or three axes of wheels and moves forward to inspect the interior of the culvert. However, there is a problem in that when it encounters obstacles (sludge, concrete debris, plant roots, gravel, sand, etc.) accumulated on the inner floor of the culvert, it lacks a function to recognize and avoid them, causing the wheels to get caught on concrete debris or fall into sludge, making further movement and filming impossible. The problem to be solved
[0005] The present invention is proposed to solve the aforementioned problems and aims to provide a safety inspection system for internal tunnels of dam facilities and reservoir water intake facilities based on subscription software and generative artificial intelligence. means of solving the problem
[0007] The subscription-based software and generative artificial intelligence-based safety inspection system for internal tunnels and reservoir water intake facilities of dam facilities according to the present invention includes: an information acquisition unit that acquires and transmits information regarding internal tunnels and reservoir water intake facilities of dam facilities, including an optical camera, a thermal imaging camera, and 3D Lidar; a shooting robot that includes a driving control unit that performs autonomous driving while avoiding obstacles; and an embedded board that stores a program for detecting defects in agricultural reservoir water intake facilities by performing artificial intelligence-based analysis on the information acquired from the shooting robot, and performs calculations for the analysis of image data and sensing data.
[0008] The driving control unit controls a variable 3-axis wheel to perform the autonomous driving, and the variable 3-axis wheel is deformable according to the internal condition of the agricultural reservoir water intake facility and is deformed into a wheel for a dry floor slab, a wheel for a flooded floor slab, or a wheel for a sediment-deposited floor slab. The camera robot includes a sensor unit that detects at least one of ground condition and obstacle information in real time, and a deformation unit that varies the shape or stiffness of the wheel according to the signal of the sensor unit. The wheel system for a moving body includes the variable 3-axis wheel, which maintains a circular stiffness structure when driving on flat ground and operates so that the outer periphery of the wheel is elastically deformed to conform to the shape of the obstacle when driving on obstacles or rough ground. The deformation unit includes at least one of a wire spoke, a chain block, or an origami structure, and includes a variable wheel system that selectively changes the radius, width, or stiffness of the wheel through the control of tension or deployment angle.
[0009] The above-described shooting robot acquires driving information by including at least one of a 3D LiDAR sensor, an encoder and gyroscope sensor equipped with odometry technology, an accelerometer, a 3-axis laser rangefinder, an infrared sensor, an ultrasonic sensor, an RF sensor, and a temperature sensor, and the driving control unit performs indoor positioning by recording the position of the robot in real time using the driving information.
[0010] The above-mentioned embedded board includes an artificial intelligence-based 3D point cloud analysis module that analyzes the driving information, and the driving control unit receives control commands from the above-mentioned embedded board and performs obstacle avoidance, forward, backward, and temporary stop according to the obstacle detection result.
[0011] The above-mentioned embedded board analyzes information acquired from the above-mentioned imaging robot to detect the above-mentioned defects classified as cracks, spalling, deformation, rebar exposure, intake hole leakage, culvert leakage, culvert scouring, and sediment deposition.
[0012] The embedded board analyzes information obtained from the camera robot to remove blurred images and low-brightness images, restores an image captured at an angle into an orthophoto using angle and position data, assigns position information and an identifier to the orthophoto, and removes noise included in the orthophoto.
[0013] The above-described embedded board utilizes a generative artificial intelligence model capable of cross-modal learning that artificially generates and outputs major defects based on video, thermal, acoustic sensing, and LiDAR data. After the AI model understands the characteristics and generation conditions of the major defects included in the user's text input, it automatically outputs the artificially generated defects.
[0014] The above-described embedded board provides a function in which a generative artificial intelligence model automatically generates a 3D mesh using a 3D mesh generation model for building a digital twin model and multiple overlapping 2D images as input.
[0015] The above-described embedded board utilizes a generative artificial intelligence model capable of cross-modal learning that outputs a 3D mesh for building a digital twin model to understand the characteristics of the 3D mesh and the conditions for generating the 3D mesh included in the image and text input provided from the user terminal, and provides the function of generating a low-resolution, coarse mesh in the first step and automatically outputting a high-quality 3D mesh with detailed textures in the second step.
[0016] The above-described shooting robot includes: a 3D LiDAR sensor; a map generation unit that generates or loads at least one of a point cloud map and a BIM map of an indoor environment using sensing values obtained from the 3D LiDAR sensor; a position recognition unit that estimates a current position based on at least one of the point cloud map and the BIM map generated by the map generation unit; and a path planning unit that generates a driving path using the position estimation result of the position recognition unit and destination information on the map, and the driving control unit performs autonomous driving according to the driving path generated by the path planning unit.
[0017] The embedded board comprises: an image input unit for collecting image data; a feature extraction module for extracting feature information from the image data; a deep learning-based detection model for detecting crack and defect candidates based on the feature information; an inference engine that receives the detection result of the deep learning-based detection model as input and determines at least one of the type, location, size, and severity of the crack; a decision module that dynamically adjusts at least one of detection parameters and detection strategies according to the determination result of the inference engine; an AI Agent that autonomously performs defect detection using the feature extraction module, the deep learning-based detection model, the inference engine, and the decision module; and a result output unit for storing or visualizing the output result of the AI Agent.
[0018] A subscription-based software and generative AI-based safety inspection system for internal tunnels and reservoir intake facilities of dam facilities according to an embodiment of the present invention performs BIM (Building Information Modeling)-linked simultaneous position estimation and mapping (BIM-SLAM) to perform AI-based safety inspections inside internal tunnels and reservoir intake facilities of dam facilities, and further includes a BIM-SLAM-based indoor AI safety inspection unit that improves the position estimation accuracy of SLAM and map reliability by utilizing a BIM model containing geometric structure and attribute information of the facility as static reference information. The BIM-SLAM-based indoor AI safety inspection unit comprises: a BIM data input unit that receives BIM model data containing geometric information and attribute information of the facility; a sensor data acquisition unit that acquires sensor data including at least one of LiDAR data, image data, and inertial data from a sensor for acquiring indoor environment data; and a data matching and position estimation unit that estimates the current position and attitude of the system and generates an initial environment map by matching structural features including planar elements extracted from the sensor data with structural information corresponding to the walls, floors, and ceilings of the BIM model. It includes a BIM-based map correction unit that generates high-precision map data corresponding to the actual environment by utilizing the above BIM model as a static standard to compare and correct SLAM drift errors accumulated in the initial environment map, and by excluding or separately managing at least one of the dynamic objects and temporary environmental changes detected from real-time sensor data from the map correction target; and an AI safety inspection execution unit that detects an area of discrepancy between the actual environment data and the BIM model on the high-precision map data, and inputs at least one of the image and point cloud data of the area into an artificial intelligence model to identify and analyze cracks, deformations, obstacles, or safety hazards of the facility. Effects of the invention
[0020] According to the present invention, unlike conventional mechanical CCTV shooting robots equipped with an optical camera capable only of forward and backward movement, a thermal imaging camera is additionally equipped in addition to the optical camera and a LiDAR sensor is provided, thereby enabling the intelligent robot shooting equipment to identify the front condition of an agricultural reservoir water intake facility (abundance) in real time while driving.
[0021] According to the present invention, an artificial intelligence-based 3D point cloud analysis algorithm analyzes objects included in a point cloud acquired through a LiDAR sensor to detect sludge, concrete debris, plant roots, gravel, sand, etc. present in an agricultural reservoir intake facility (culvert) and automatically sets an optimal path, and an intelligent robot camera equipment is equipped with a steering device capable of moving forward while avoiding obstacles, thereby enabling the rapid identification of the condition inside the agricultural reservoir intake facility (culvert) and providing a safe and reliable inspection function for the agricultural reservoir intake facility (culvert).
[0022] The effects of the present invention are not limited to those mentioned above, and other unmentioned effects will be clearly understood by those skilled in the art from the description below. Brief explanation of the drawing
[0024] FIG. 1 illustrates an intelligent robot shooting control system for managing agricultural reservoir water intake facilities capable of autonomous driving and obstacle recognition based on artificial intelligence according to an embodiment of the present invention. FIG. 2 illustrates an intelligent robot shooting control method dedicated to an agricultural reservoir water intake facility (vent) according to an embodiment of the present invention. Figure 3 illustrates the process of change in the safety inspection paradigm of an agricultural reservoir water intake facility according to an embodiment of the present invention. FIGS. 4 and 5 illustrate the configuration of an intelligent robot camera equipment for an agricultural reservoir water intake facility according to an embodiment of the present invention. FIG. 6 illustrates major damage (Type A) occurring in an agricultural reservoir water intake facility according to an embodiment of the present invention. FIG. 7 illustrates major damage (Type B) occurring in an agricultural reservoir water intake facility according to an embodiment of the present invention. FIG. 8 is a configuration diagram of a subscription-based artificial intelligence-based infrastructure facility safety inspection platform, such as an agricultural reservoir water intake facility, according to an embodiment of the present invention. FIG. 9 illustrates a UI for a subscription-based artificial intelligence-based infrastructure facility safety inspection platform, such as an agricultural reservoir water intake facility (abundance), according to an embodiment of the present invention. FIG. 10 illustrates a configuration diagram of an artificial intelligence model dedicated to safety inspection of an agricultural reservoir water intake facility (abundance) according to an embodiment of the present invention. FIG. 11 illustrates an infographic dedicated to safety inspection of an agricultural reservoir water intake facility (abundance) according to an embodiment of the present invention. FIG. 12 illustrates a cross-modal generative artificial intelligence model structure that understands the characteristics of a 3D mesh included in a user's text input and the conditions for generating the mesh, and provides a function to automatically output the mesh. FIG. 13 illustrates a cross-modal generative artificial intelligence model structure that understands the characteristics of cracks included in a user's text input and crack generation conditions, and provides a function to automatically output cracks. Figure 14 illustrates a schematic diagram of a model that generates a 3D mesh for building a generative AI-based digital twin model. FIG. 15 illustrates the generation of indoor positioning values and the matching results with damage data using a multi-sensor robo-inspector for quality inspection of temporary construction materials according to an embodiment of the present invention. FIG. 16 illustrates a generative artificial intelligence model-based crack generation-detection integrated pipeline according to an embodiment of the present invention. FIG. 17 illustrates a generative artificial intelligence model capable of cross-modal learning that outputs a 3D mesh for building a digital twin model according to an embodiment of the present invention. FIG. 18 illustrates an AI agent dedicated to detecting infrastructure facility defects according to an embodiment of the present invention. FIG. 19 is a block diagram showing a computer system for implementing a method according to an embodiment of the present invention. FIG. 20 illustrates a pipeline composed of three stages: BIM-based session data generation, multi-session anchoring, and map updating, according to an embodiment of the present invention. Specific details for implementing the invention
[0025] The aforementioned objectives of the present invention, as well as other objectives, advantages, and features, and the methods for achieving them, will become clear from the embodiments described in detail below together with the accompanying drawings.
[0026] However, the present invention is not limited to the embodiments disclosed below but can be implemented in various different forms, and the following embodiments are provided merely to easily inform those skilled in the art of the purpose, structure, and effects of the invention, and the scope of the rights of the present invention is defined by the description in the claims.
[0027] Meanwhile, the terms used in this specification are for describing the embodiments and are not intended to limit the invention. In this specification, the singular form includes the plural form unless specifically stated otherwise in the text. As used in this specification, "comprises" and / or "comprising" do not exclude the presence or addition of one or more other components, steps, actions, and / or elements to the mentioned components, steps, actions, and / or elements.
[0029] RPN-based artificial intelligence algorithm
[0030] RPN is an algorithm that uses a Region Proposal Network to extract features of objects contained in an image, recommends multiple candidate bounding boxes based on those features, and then infers the final detection result by progressively reducing the number of recommended bounding boxes based on probability.
[0031] To explain the process in more detail, first, the calculation of feature extraction for each Region of Interest (RoI) is shared, and it can be implemented by introducing a deep learning-based RPN.
[0032] After extracting feature maps using many CNN layers, a large number of windows likely to contain objects are generated via RPN. The algorithm then searches the feature maps within each window, scales them to a fixed size (RoI pooling), and predicts class probabilities and more accurate bounding boxes for the corresponding objects.
[0033] One point to consider here is how RPN creates windows, and RPN uses anchor boxes.
[0034] However, the anchor box is generated with a fixed size and shape rather than from data, and this is the difference from other similar algorithms.
[0035] This anchor box can cover the image more densely, and RPN performs only binary classification based on whether an object is included in the window, instead of classification for multiple object categories.
[0036] Feature Pyramid Network
[0037] A Feature Pyramid Network (FPN) is a network that inputs a single-scale image of arbitrary size into a convolutional network and outputs feature maps of various scales.
[0038] Generally, validated AlexNet, GooglNet, ResNet, etc. are used.
[0039] FPN can be described as a network that extracts and modifies feature maps for each layer of various sizes specified in existing convolutional networks.
[0040] The process by which FPN extracts feature maps and constructs pyramids includes bottom-up pathway, top-down pathway, and lateral connections methods.
[0041] Using FPN has the advantage of detecting damage of various sizes contained in images and extracting various features such as boundaries and internal patterns contained in the damage, but the more layers in FPN there are, the longer the computation time becomes.
[0042] According to an embodiment of the present invention, a 4-layer FPN structure capable of satisfying both computation time and object detection accuracy is applied.
[0043] Explainable AI module
[0044] XAI (eXplainable AI) refers to explainable artificial intelligence. Its purpose is to generate 'visual explanations' and 'captions' regarding the reliability of a model and its decisions.
[0045] Grad-CAM is a technology that generates 'visual descriptions' from CNN-based models to explain the structure of a system in a way that humans can understand, meaning it possesses 'interpretability'.
[0046] CAM (Class Activation Mapping) provides a visual explanation when determining many classes in CNN-based networks.
[0047] The Grad-CAM (Gradient-weighted CAM) mentioned here uses the gradient for the "target class (caption, mask also possible)" which "flows into the final convolution to produce an approximate local map that highlights important parts within the predicted image" when calculating the CAM.
[0048] Image Captioning is research that connects the two major currents of the artificial intelligence field: 'Computer Vision' and 'Natural Language Processing'.
[0049] Image Captioning adopts an Encoder-Decoder structure in which the original image is encoded using a CNN as the Encoder and the caption part is decoded using an LSTM as the Decoder.
[0050] When training the Encoder, the pre-trained classification model is fine-tuned using a transfer learning algorithm, the received model is imported, and the final fully connected layer is not used.
[0051] When training the Decoder, the embedding vector output from the Encoder is passed through the LSTM to output each word of the sentence in order.
[0052] Intensity Optimizer
[0053] Just as human cognitive abilities are affected by changes in lighting, the performance of AI models is also influenced by changes in image brightness. For example, an AI model primarily trained using high-brightness images taken in a well-lit environment experiences a significant decline in object detection performance when low-brightness images taken in a dark environment are input. Therefore, when testing a new, untrained image after building an AI model trained on images within a specific target brightness range (e.g., 150–200), high performance can be expected by analyzing the brightness spectrum of the image and converting it to the specific target brightness range.
[0054] Spot Noise Filter
[0055] Images captured by optical cameras contain point-shaped noise (generated during the stepwise electrical signal processing) for various reasons.
[0056] Since such noise degrades the object detection performance of deep learning models, it is removed in advance using a point-type dedicated noise filter.
[0057] A platform that can be executed via a web browser
[0058] The platform is not affected by the execution location and can be run in a web browser without the need for separate software installation; it possesses a list of functions necessary for the entire process, including UAV control, video monitoring (control), and video processing.
[0060] According to an embodiment of the present invention, in addition to an optical camera, a thermal imaging camera and a LiDAR sensor are provided so that an intelligent robot imaging device can identify the condition of the front of an agricultural reservoir water intake facility (abundance) in real time while driving.
[0061] According to an embodiment of the present invention, an artificial intelligence-based 3D point cloud analysis algorithm analyzes objects included in a point cloud acquired through a LiDAR sensor to detect sludge, concrete debris, plant roots, gravel, sand, etc. present in an agricultural reservoir water intake facility (abundance), and then automatically sets an optimal path.
[0062] According to an embodiment of the present invention, a steering device capable of advancing while avoiding obstacles is mounted on an intelligent robot camera, thereby quickly identifying the internal condition of an agricultural reservoir water intake facility (aby) and providing a safe and highly reliable inspection function for the agricultural reservoir water intake facility (aby).
[0063] FIG. 1 illustrates an intelligent robot shooting control system for managing agricultural reservoir water intake facilities capable of autonomous driving and obstacle recognition based on artificial intelligence according to an embodiment of the present invention.
[0064] A subscription-based software and generative artificial intelligence-based safety inspection system for internal tunnels and reservoir water intake facilities of dam facilities according to an embodiment of the present invention includes an information acquisition unit (110) that acquires and transmits information about an agricultural reservoir water intake facility, a driving control unit (120) that performs autonomous driving while avoiding obstacles, a shooting robot (100) that includes an artificial intelligence-based analysis of the information acquired from the shooting robot to detect defects in the agricultural reservoir water intake facility, and an embedded board (200) that stores a program for analyzing image data and sensing data and performs calculations for the analysis of the information and sensing data.
[0065] The information acquisition unit (110) includes an optical camera and a thermal imaging camera.
[0066] The above driving control unit (120) controls the variable 3-axis wheels to perform the above autonomous driving.
[0067] The driving control unit (120) controls the variable 3-axis wheel to perform the autonomous driving, and the variable 3-axis wheel is deformable according to the internal condition of the agricultural reservoir water intake facility and is deformed into a wheel for a dry floor slab, a wheel for a flooded floor slab, and a wheel for a sediment-deposited floor slab, and the camera robot includes a sensor unit that detects at least one of ground condition and obstacle information in real time, and a deformation unit that varies the shape or stiffness of the wheel according to the signal of the sensor unit, and the wheel system for a moving body that maintains a circular stiffness structure when driving on flat ground, and operates such that the outer periphery of the wheel is elastically deformed to conform to the shape of the obstacle when driving on obstacles or rough ground, wherein the deformation unit includes at least one of a wire spoke, a chain block, or an origami structure, and includes a variable wheel system that selectively changes the radius, width, or stiffness of the wheel through the control of tension or deployment angle.
[0068] The above variable 3-axle wheel can be replaced depending on the internal condition of the agricultural reservoir water intake facility, and the type varies depending on the dry bottom slab, flooded bottom slab, and sediment-deposited bottom slab.
[0069] The above-mentioned shooting robot (100) acquires driving information including a lidar sensor, a gyroscope sensor, an accelerometer sensor, a temperature sensor, a 3-axis laser rangefinder, an infrared sensor, an ultrasonic sensor, an RF sensor, and an encoder.
[0070] According to an embodiment of the present invention, the variable wheel system includes a variable wheel that changes the shape and stiffness of the wheel in real time depending on the condition of the ground or the presence of obstacles in the tunnel inside the dam facility and the reservoir intake facility.
[0071] A variable wheel system includes a sensor unit for detecting unevenness, slope, obstacle height, and impact on the ground, a control unit for controlling the shape or stiffness of the wheel based on information received from the sensor unit, and a deformation unit for performing structural deformation of the wheel.
[0072] The sensor unit may include at least one of an acceleration sensor, a torque sensor, a distance sensor (LiDAR or ultrasonic sensor), and a vision sensor, and detects the condition of the ground on which the moving body is traveling or the size and shape of an obstacle ahead in real time.
[0073] The control unit analyzes data collected from the sensor unit to determine whether it is a flat driving mode or an obstacle overcoming mode.
[0074] The deformation part includes a plurality of wire spokes, chain blocks, or origami structures constituting the outer circumference of the wheel, and the tension, connection angle, or unfolded state is adjusted according to a control signal from the control unit. For example, when driving on flat ground, maximum tension is applied to the wire spokes to maintain the wheel in a circular, high-rigidity structure, thereby ensuring driving stability and high-speed driving performance. On the other hand, when an obstacle such as stairs, rocks, or unpaved terrain is detected, the control unit gradually relieves the tension or unfolds the origami structure so that the outer circumference of the wheel is elastically deformed.
[0075] The wheels adapt to the shape of obstacles to increase the contact area, enabling them to effectively traverse obstacles taller than the wheel radius. Furthermore, when a deployable structure is applied, physically expanding the wheel diameter or width allows for stable driving even on soft ground or near-vertical slopes.
[0076] The variable wheel system according to an embodiment of the present invention can be applied to delivery robots, industrial or military unmanned robots, space exploration rovers, etc., and provides the effect of simultaneously improving driving performance and stability in various environments that are difficult to overcome with conventional fixed wheels.
[0077] According to an embodiment of the present invention, a multi-sensor is provided to perform mapping within a water intake facility, autonomous driving, indoor positioning in GPS blind spots, obstacle detection, stopping, and avoidance.
[0078] The embedded board (200) includes an artificial intelligence-based 3D point cloud analysis module (210) that analyzes the driving information, and the driving control unit (120) receives a control command from the embedded board (200) and performs obstacle avoidance, forward, backward, and temporary stop according to the obstacle detection result.
[0079] The above-mentioned embedded board (200) processes 3D point cloud data regarding the internal state of the agricultural reservoir water intake facility and provides visual information to the manager.
[0080] The above-mentioned embedded board (200) provides an infographic of information received from the above-mentioned shooting robot (100).
[0081] The embedded board (200) analyzes information obtained from the imaging robot (100) to detect defects classified as cracks, spalling, deformation, rebar exposure, water intake hole leakage, pipe leakage, pipe scouring, and sediment deposition.
[0082] The embedded board (200) analyzes information obtained from the shooting robot (100) to remove blurry images and low-brightness images in the preprocessing stage, restores an image captured at an angle to an orthophoto using angle and position data, assigns position information and an identifier to the orthophoto, and removes noise included in the orthophoto.
[0083] The above embedded board (200) analyzes the increase in the number of concurrent users and the task increase rate to provide recommendation information regarding CPU, GPU, and memory resources to the administrator, and performs defect detection of the agricultural reservoir water intake facility through a web browser capable of scaling up.
[0084] The above-mentioned embedded board (200) uses a generative artificial intelligence model capable of cross-modal learning that artificially generates and outputs major defects based on video, thermal imaging, acoustic sensing, and LiDAR data, and automatically outputs the artificially generated defects after the artificial intelligence model understands the characteristics and generation conditions of the major defects included in the user's text input.
[0085] According to an embodiment of the present invention, major defects in tunnels inside dam facilities and reservoir water intake facilities are artificially generated using generative artificial intelligence and utilized for safety inspections.
[0086] A subscription-based software and generative artificial intelligence-based safety inspection system for internal tunnels and reservoir water intake facilities of dam facilities according to an embodiment of the present invention includes a plurality of sensor units that collect video, thermal imaging, acoustic sensing, and LiDAR data, an embedded board that processes data collected from the sensor units in real time, and a generative artificial intelligence model that operates in conjunction with the embedded board.
[0087] The embedded board can be mounted on a robot or a stationary inspection device moving inside a dam tunnel, water intake facility, or structure under inspection, and synchronizes and collects visible light image data, thermal image data, acoustic signals generated inside the structure, and LiDAR-based 3D spatial data. The collected data undergoes a preprocessing stage in which noise removal, alignment, and feature extraction are performed, after which it is provided as input data for a generative artificial intelligence model.
[0088] The generative AI model features a cross-modal learning structure capable of simultaneously learning image, thermal, acoustic, and LiDAR data, and is configured to learn the morphological, thermal, acoustic, and spatial characteristics of major defects such as cracks, spalling, leakage, cavities, internal voids, and rebar exposure. Furthermore, the generative AI model can be trained based on training data that reflects various environmental conditions and defect progression stages, in addition to normal and actual defect data.
[0089] Users can input generation conditions, such as the type, location, size, extent of progression, and environmental conditions of defects, via text input through subscription software. A generative AI model understands the defect features and generation conditions included in the text input based on natural language processing, and then automatically generates artificial defect data that meets those conditions through a cross-modal generation process. The generated artificial defects are output in the form of images, thermal images, acoustic patterns, or 3D LiDAR data, and are provided in a format similar to the actual inspection environment.
[0090] The artificial defect data generated in this way can be utilized to augment training data for safety inspection AI models, verify inspection scenarios, train inspectors, and conduct pre-risk analysis. In particular, by artificially generating defect cases that occur infrequently in reality or are difficult to obtain, it provides the effect of improving the reliability and accuracy of the safety inspection system.
[0091] The subscription software according to an embodiment of the present invention operates in a remote server or cloud environment and can efficiently manage the safety of internal structures of dam facilities by providing continuous updates, model improvements, and defect generation services for multiple dam facilities.
[0092] The above-described embedded board (200) provides a function in which a generative artificial intelligence model automatically generates a 3D mesh using a 2D image as input, by using a 3D mesh generation model for building a digital twin model.
[0093] The safety inspection system for internal tunnels and reservoir water intake facilities of dam facilities based on subscription software and generative artificial intelligence according to an embodiment of the present invention performs safety inspections of internal tunnels and reservoir water intake facilities of dam facilities using generative artificial intelligence and subscription software, and in particular provides a function to automatically generate a 3D mesh for building a digital twin model.
[0094] A subscription-based software and generative artificial intelligence-based safety inspection system for internal tunnels and reservoir water intake facilities of dam facilities according to an embodiment of the present invention includes an embedded board mounted on a filming robot or a fixed inspection device, and the embedded board provides a function to automatically generate a 3D mesh by receiving a plurality of superimposed 2D images as input using a 3D mesh generation model for building a digital twin model.
[0095] The embedded board collects multiple 2D images of the walls, ceilings, and structural surfaces of the dam's internal tunnel or water intake facility under inspection, captured from various viewpoints and angles. The 2D images are stored along with shooting time information, camera parameters, or position information, and are configured to have mutually overlapping regions.
[0096] The 3D mesh generation model takes multiple overlapping 2D images as input, performs feature point matching, depth estimation, and spatial alignment between images, and then automatically generates a 3D mesh for a structure under inspection through a reconstruction process based on generative artificial intelligence. The 3D mesh generation model may include a neural network-based 3D reconstruction model or an image-to-mesh generative model.
[0097] The generated 3D mesh includes the shape, irregularities, and geometric deformations of the structure's surface, and defect information such as cracks, spalling, or traces of leakage can be mapped onto the mesh. Through this, a digital twin model of the structure under inspection is automatically constructed, providing a 3D representation similar to the actual structure.
[0098] The digital twin model can be updated by accumulating inspection data over time and can be utilized to analyze the progression of defects or trends in structural deformation through comparison with past inspection points. Additionally, the digital twin model can be visualized and managed in a remote server or cloud environment via subscription software.
[0099] According to an embodiment of the present invention, the 3D mesh automatic generation function reduces the burden of manual modeling for field inspectors and provides the effect of enabling intuitive and precise safety inspection of the internal structure of dam facilities.
[0100] The above-mentioned embedded board (200) uses a generative artificial intelligence model capable of cross-modal learning to output a 3D mesh for building a digital twin model, understands the characteristics of the 3D mesh included in the user's text input and the conditions for generating the 3D mesh, and provides a function to automatically output the 3D mesh.
[0101] Although it was described as an embedded board in the aforementioned embodiment, it is also possible to provide it as a server, which is a hardware configuration separate from the shooting robot.
[0102] According to an embodiment of the present invention, a system for performing safety inspections of internal tunnels of dam facilities and reservoir water intake facilities using generative artificial intelligence and subscription software is proposed, and in particular includes a 3D LiDAR-based autonomous shooting robot.
[0103] A subscription-based software and generative artificial intelligence-based safety inspection system for tunnels inside dam facilities and reservoir water intake facilities according to an embodiment of the present invention includes a camera robot that travels inside a tunnel or water intake facility inside a dam facility and collects inspection data, an embedded board linked with the camera robot, and generative artificial intelligence-based subscription-based software.
[0104] The camera robot includes a 3D LiDAR sensor, and the 3D LiDAR sensor senses distance, shape, and spatial information of surrounding structures in real time while driving. The map generation unit can generate a point cloud map of the inspection target space using the sensing values acquired from the 3D LiDAR sensor, or load a pre-built BIM (Building Information Modeling) map. At this time, the point cloud map and the BIM map can be used independently or combined by aligning them.
[0105] The position recognition unit estimates the current position and attitude of the camera robot in real time by matching with LiDAR sensing data, based on at least one of the point cloud map and BIM map generated or loaded by the map generation unit. Through this, stable position recognition is possible even in environments such as tunnels inside a dam or water intake facilities where GPS signals do not reach.
[0106] The path planning unit generates a driving path using the position estimation result of the position recognition unit and the inspection destination information or inspection scenario information set on the map. The driving path is set to efficiently photograph tunnel walls, ceilings, floors, and structures around the water intake, and can be dynamically updated considering obstacle avoidance and passage through narrow sections.
[0107] The driving control unit performs autonomous driving by controlling the drive unit of the camera robot according to the driving path generated by the path planning unit. During autonomous driving, the camera robot simultaneously collects video, thermal, acoustic sensing, and LiDAR data, and the collected data is preprocessed via an embedded board and then provided as input data for a generative artificial intelligence model.
[0108] Generative AI-based subscription software enables efficient inspection and management of the safety of internal structures of dam facilities by providing defect analysis, artificial defect generation, inspection scenario verification, and safety inspection support functions based on the collected multi-sensor data.
[0110] FIG. 2 illustrates an intelligent robot shooting control method dedicated to an agricultural reservoir water intake facility (bottle) according to an embodiment of the present invention. A bottle refers to a reservoir water intake facility.
[0111] The abdominal pain CCV imaging robot includes a LiDAR sensor and a steering device.
[0112] The abdominal pain CCTV filming robot uses a camera to film internal defects in the abdominal pain and acquires video.
[0113] The obstacle avoidance module is equipped with an AI-based automatic obstacle avoidance function, and the autonomous driving module is equipped with an AI-based autonomous driving function.
[0114] The internal status of the abdominal pain and the predicted path of movement are displayed through the monitoring screen of the remote controller (control box).
[0115] Using the joystick on the remote controller (control box), the administrator can control the movement of the abdominal CCTV recording robot while checking the recorded video.
[0116] A subscription-based software and generative artificial intelligence-based safety inspection system for internal tunnels and reservoir water intake facilities of dam facilities according to an embodiment of the present invention comprises: a shooting robot (100) including an information acquisition unit (110) that acquires and transmits information about an agricultural reservoir water intake facility and a driving control unit (120) that performs autonomous driving while avoiding obstacles; and an embedded board (200) that stores a program for detecting defects in the agricultural reservoir water intake facility by performing an artificial intelligence-based analysis on the information acquired from the shooting robot and performs calculations for the analysis of image data and sensing data.
[0117] The information acquisition unit (110) includes an optical camera, a thermal imaging camera, and a 3D Lidar.
[0118] The above driving control unit (120) controls the variable 3-axis wheels to perform the above autonomous driving.
[0119] The above variable 3-axle wheel can be replaced according to the internal condition of the agricultural reservoir water intake facility and is replaced with a wheel for a dry bottom slab, a wheel for a flooded bottom slab, and a wheel for a sediment-deposited bottom slab.
[0120] The above-mentioned shooting robot (100) acquires driving information by including at least one of a 3D LiDAR sensor, an encoder and gyroscope sensor equipped with odometry technology, an accelerometer, a 3-axis laser distance measuring device, an infrared sensor, an ultrasonic sensor, an RF sensor, and a temperature sensor, and the driving control unit (120) performs indoor positioning by recording the position of the robot in real time using the driving information.
[0121] The embedded board (200) includes an artificial intelligence-based 3D point cloud analysis module that analyzes the driving information, and the driving control unit (120) receives a control command from the embedded board (200) and performs obstacle avoidance, forward, backward, and temporary stop according to the obstacle detection result.
[0122] The embedded board (200) analyzes information obtained from the imaging robot (100) to detect defects classified as cracks, spalling, deformation, rebar exposure, water intake hole leakage, pipe leakage, pipe scouring, and sediment deposition.
[0123] The embedded board (200) analyzes information obtained from the shooting robot (100) to remove blurry images and low-brightness images, restores an image captured at an angle to an orthophoto using angle and position data, assigns position information and an identifier to the orthophoto, and removes noise included in the orthophoto.
[0124] The above-mentioned embedded board (200) uses a generative artificial intelligence model capable of cross-modal learning that artificially generates and outputs major defects based on video, thermal imaging, acoustic sensing, and LiDAR data, and automatically outputs the artificially generated defects after the artificial intelligence model understands the characteristics and generation conditions of the major defects included in the user's text input.
[0125] The above-described embedded board (200) provides a function in which a generative artificial intelligence model automatically generates a 3D mesh using a 3D mesh generation model for building a digital twin model and inputs a plurality of superimposed 2D images.
[0126] The above-described embedded board (200) uses a generative artificial intelligence model capable of cross-modal learning to output a 3D mesh for building a digital twin model, and understands the characteristics of the 3D mesh and the conditions for generating the 3D mesh included in the image and text input provided from the user terminal, and in the first step, generates a low-resolution, coarse mesh, and in the second step, automatically outputs a high-quality 3D mesh with detailed textures.
[0127] The above-described shooting robot (100) includes: a 3D LiDAR sensor; a map generation unit that generates or loads at least one of a point cloud map and a BIM map of an indoor environment using a sensing value obtained from the 3D LiDAR sensor; a location recognition unit that estimates the current location based on at least one of the point cloud map and the BIM map generated by the map generation unit; and a path planning unit that generates a driving path using the location estimation result of the location recognition unit and destination information on the map, and the driving control unit performs autonomous driving according to the driving path generated by the path planning unit.
[0128] The embedded board (200) comprises: an image input unit for collecting image data; a feature extraction module for extracting feature information from the image data; a deep learning-based detection model for detecting crack and defect candidates based on the feature information; an inference engine that receives the detection result of the deep learning-based detection model as input and determines at least one of the type, location, size, and severity of the crack; a decision module that dynamically adjusts at least one of the detection parameters and detection strategy according to the judgment result of the inference engine; an AI Agent that autonomously performs defect detection using the feature extraction module, the deep learning-based detection model, the inference engine, and the decision module; and a result output unit for storing or visualizing the output result of the AI Agent.
[0129] The present invention relates to a system for performing safety inspections of internal tunnels and reservoir water intake facilities of dam facilities using generative artificial intelligence and subscription software, and in particular, to a safety inspection system including an AI agent that autonomously detects defects on an embedded board.
[0130] A subscription-based software and generative artificial intelligence-based safety inspection system for internal tunnels and reservoir water intake facilities of dam facilities according to an embodiment of the present invention includes an embedded board mounted on a shooting robot or a fixed inspection device, and the embedded board includes an image input unit, a feature extraction module, a deep learning-based detection model, an inference engine, a decision-making module, an AI agent, and a result output unit.
[0131] The image input unit collects image data of the structural surfaces of tunnels or water intake facilities inside the dam facility through a visible light camera or a multispectral camera. The collected image data can be processed in real-time or near-real-time within the embedded board.
[0132] The feature extraction module extracts visual feature information related to defects, such as cracks, spalling, traces of leakage, and changes in surface roughness, from the image data. The feature extraction can be performed using a convolutional neural network-based feature extractor or a pre-trained neural network.
[0133] A deep learning-based detection model receives the above feature information as input and detects candidate regions for cracks and defects. The detection model has an object detection or segmentation structure and outputs the location and shape of the defect candidates.
[0134] The inference engine receives the detection result of the deep learning-based detection model as input and determines at least one of the type, location, size, or severity of the crack. At this time, the inference engine may consider multi-frame analysis, spatial continuity, or past inspection history data together.
[0135] The decision module dynamically adjusts at least one of the detection parameters or detection strategies of the deep learning-based detection model according to the judgment result of the inference engine. For example, if it is determined that the severity of the crack is high, it can control the detection sensitivity for the corresponding area to be increased or the shooting resolution to be increased.
[0136] The AI Agent performs defect detection autonomously without human intervention by integrally utilizing the feature extraction module, deep learning-based detection model, inference engine, and decision module. The AI Agent can be configured to iteratively improve its detection strategy according to changes in the inspection environment.
[0137] The result output unit stores or visualizes the output results of the AI Agent and provides them to the user. For example, it can superimpose crack locations onto an image or transmit inspection result data containing defect information to a remote server via subscription software.
[0138] The shooting robot (100) according to an embodiment of the present invention is designed to enable stable driving and shooting even in a sealed environment where it is difficult for personnel to directly access and GPS signal reception is impossible, such as the inside of a hull of an agricultural reservoir water intake facility. In particular, the optical camera, thermal camera, and 3D Lidar included in the information acquisition unit (110) acquire data of different physical characteristics at the same time, thereby enabling the comprehensive identification of leaks, moisture infiltration, temperature abnormalities, or internal surface defects that are difficult to observe in the visible light region.
[0139] The driving control unit (120) controls the driving state, rotational speed, and steering angle of the variable 3-axis wheels in real time, thereby enabling the filming robot (100) to maintain a stable posture and move forward or backward even if the floor condition inside the hull changes to a dry state, a flooded state, or a sediment-accumulated state. At this time, the variable 3-axis wheels are not limited to a simple replaceable structure, but can be operated in a manner where an optimal combination of wheels is selected and mounted according to internal state information or the manager's settings.
[0140] The 3D LiDAR sensor, encoder, gyroscope sensor, accelerometer sensor, and other sensors equipped in the shooting robot (100) are fused together and used to simultaneously estimate the robot's travel distance, rotation angle, attitude change, and environmental shape, thereby improving indoor positioning accuracy even in GPS blind spots such as inside a stomach. These indoor positioning results are used for recording the driving path, repeatedly inspecting the same section, reproducing the location of defects, and accumulating management history.
[0141] An artificial intelligence-based 3D point cloud analysis module included in the embedded board (200) receives point cloud data acquired from a LiDAR sensor as input, reconstructs the internal spatial structure of the abdomen in three dimensions, and distinguishes floor surfaces, walls, ceilings, and obstacle objects. In this process, obstacles that may hinder driving, such as sludge, concrete debris, gravel, sand, and plant roots, are automatically recognized, and based on the result, commands to avoid, stop, or reset the path are transmitted to the driving control unit (120).
[0142] The embedded board (200) uses image data and sensing data received from the shooting robot (100) to automatically classify defects such as cracks, spalling, deformation, rebar exposure, water intake hole leakage, pipe leakage, pipe scouring, and sediment deposition inside the pipe, and structurally records the location and occurrence pattern of each defect. Through this, beyond simple image storage, it becomes possible to manage defects by type and analyze risks.
[0143] The embedded board (200) improves the recognition accuracy of the artificial intelligence model by removing blurred images and low-brightness images during the preprocessing stage, and restores images captured at an angle into orthophotos using angle information and position information. Position information and an identifier are assigned to the generated orthophotos, making it possible to track changes over time of the same defect and align them with a digital twin model.
[0144] Furthermore, generative AI models based on video, thermal, acoustic, and LiDAR data improve the learning efficiency and generalization performance of the AI models by artificially generating defect data even for defect types where it is difficult to secure sufficient training data during actual inspection processes. In this process, defect characteristics, occurrence locations, or generation conditions included in the user's text input are reflected in the model, automatically outputting defect data that meets management objectives.
[0145] The 3D mesh generation function for building digital twin models provides a virtual model with the same shape as the actual facility by reproducing the internal structure of the facility in three dimensions based on multiple overlapping 2D images. In particular, by first generating a low-resolution, coarse mesh and then outputting a high-quality 3D mesh with detailed textures in stages, it is possible to reduce the computational burden while simultaneously ensuring visual completeness and structural accuracy.
[0146] The map generation unit, location recognition unit, and path planning unit included in the shooting robot (100) systematically manage spatial information inside the tube based on a point cloud map or BIM map, and automatically generate an optimal driving path according to a destination or inspection section set by the manager. Accordingly, the same driving conditions can be reproduced even during repeated inspections, thereby improving the reliability of the inspection results.
[0147] The AI Agent implemented in the embedded board (200) operates as an intelligent agent in which feature extraction, defect detection, inference, and decision-making functions are organically combined, rather than as a single artificial intelligence model. The AI Agent has the characteristic of gradually improving defect detection performance during the iterative inspection process by independently evaluating defect detection results and dynamically adjusting detection parameters and detection strategies according to environmental conditions or defect types.
[0148] Thus, the system according to the embodiment of the present invention automates the safety inspection of an agricultural reservoir water intake facility centered on a shooting robot (100), an embedded board (200), and an AI Agent, and achieves a technical effect of overcoming the limitations of the conventional human-centered inspection method by providing objective and reproducible inspection results.
[0150] Figure 3 illustrates the process of change in the safety inspection paradigm of an agricultural reservoir water intake facility according to an embodiment of the present invention. The term "intake facility" refers to a reservoir water intake facility. According to the prior art, the method relied on human labor, was analog, had high risk, and relied on subjective evaluation. In contrast, according to an embodiment of the present invention, it is possible to detect cracks and defects using explainable artificial intelligence through robots, image processing, XAI detection, and objective evaluation.
[0151] In accordance with this change in the safety inspection paradigm, the embodiments of the present invention go beyond mere visual inspection of captured images and enable the quantitative derivation of the presence, location, shape, and severity of defects by analyzing image data and sensing data acquired through a shooting robot based on artificial intelligence. In particular, by applying Explainable Artificial Intelligence (XAI), heatmaps, bounding boxes, segmentation results, and natural language descriptions are provided for areas where cracks and defects are detected, allowing managers to intuitively understand the basis of the AI's judgment.
[0152] In addition, in conventional technology, variations in inspection results occur depending on the inspector's proficiency, experience, and on-site conditions, and there were problems with the difficulty of consistent evaluation even during repeated inspections of the same facility. However, according to an embodiment of the present invention, the reproducibility and objectivity of inspection results are significantly improved by standardizing the robot driving path, shooting conditions, and analysis algorithm. Accordingly, it becomes possible to track changes over time regarding cracks and defects inside the abdomen, and scientific decision-making for preventive maintenance and disaster response becomes possible.
[0153] As illustrated in FIG. 3, an embodiment of the present invention reduces the time required for inspection and minimizes the risk of safety accidents caused by manpower input by converting the safety inspection process into a real-time or near-real-time processing structure centered on robots, image processing, and artificial intelligence analysis. As a result, the maintenance management system of agricultural reservoir water intake facilities achieves the technical effect of transitioning from a post-inspection-centered inspection method to an intelligent safety management system capable of proactive prevention and continuous monitoring.
[0155] FIGS. 4 and 5 illustrate the configuration of an intelligent robot camera equipment for an agricultural reservoir water intake facility according to an embodiment of the present invention.
[0156] An intelligent robot shooting control device for managing agricultural reservoir water intake facilities capable of autonomous driving and obstacle recognition based on artificial intelligence according to an embodiment of the present invention performs autonomous driving by including a variable 3-axis wheel, an optical camera, a thermal imaging camera, an assembly frame, a movable LiDAR sensor, a remote controller controllable in a closed space, a controller capable of wired and wireless remote control of equipment and shooting device, and a control system.
[0157] The variable 3-axis wheel according to the present invention can be replaced with a wheel for a dry floor slab, a wheel for a flooded floor slab, and a wheel for a soil-deposited floor slab. The wheel for a dry floor slab is a wheel with a smooth shape that enables fast movement of the robot without vibration, the wheel for a flooded floor slab is a wheel with a smooth shape that enables stable movement of the robot without slipping, and the wheel for a soil-deposited floor slab is a wheel with an L-shaped smooth shape that enables forward movement of the robot through a soil-deposited floor. According to an embodiment of the present invention, the length, shape, distribution density per unit area, and the shape and spacing of the smoothness arranged from the center to the edge of the wheel for the floor slab can be replaced differently depending on the degree of flooding. According to an embodiment of the present invention, the wheels for a sediment-deposited floor slab can be replaced differently by considering information on the sediment deposition state (sediment deposition depth, physical characteristics, etc.), such that the length, shape, and distribution density per unit area of the irregularities of the wheels, the shape and spacing of the irregularities arranged from the center to the edge, and the angle of the L-shaped form are close to a right angle or diverge from a right angle at a preset angle.
[0158] An intelligent robot shooting control device for managing an agricultural reservoir water intake facility capable of autonomous driving and obstacle recognition based on artificial intelligence according to an embodiment of the present invention includes an artificial intelligence-based 3D point cloud analysis module that analyzes signals received from a mobile lidar sensor, an autonomous driving module that performs obstacle avoidance, forward and backward movement, and a steering device.
[0159] The 3D point cloud analysis module analyzes signals received from a mobile LiDAR sensor to determine the possibility of an object's existence within the point cloud and displays the object's location. After classifying whether the object is an obstacle such as sludge, concrete debris, plant roots, gravel, or sand, it presents the reasoning as an infographic.
[0160] An intelligent robot shooting control device for managing agricultural reservoir water intake facilities capable of autonomous driving and obstacle recognition based on artificial intelligence according to an embodiment of the present invention includes a monitoring device that displays 3D Point Cloud data of the internal state of the agricultural reservoir water intake facility obtained using a mobile LiDAR sensor, and the monitoring device displays a predicted movement path of the shooting robot.
[0161] The monitoring equipment displays 3D point cloud data of the internal state of the agricultural reservoir water intake facility (abundance) received by the mobile lidar sensor on the screen, and displays the predicted movement path of the intelligent robot camera equipment in the form of a guideline.
[0162] The variable 3-axle wheels are provided in three interchangeable types to accommodate the internal conditions of the agricultural reservoir water intake facility.
[0163] The variable 3-axle wheels are provided in three interchangeable types to respond to the internal conditions (no obstacles, presence of sludge) of the agricultural reservoir intake facility (hull).
[0164] The control system transmits equipment signals and video acquired during filming in real time and provides infographics to the user.
[0165] The control system transmits signals and images from the camera in real time, determines the possibility of an object's existence within the point cloud and displays the object's location, classifies whether the object is an obstacle such as sludge, concrete debris, plant roots, gravel, or sand, and provides an infographic including the reasoning.
[0166] According to this configuration, the intelligent robot shooting control device according to the embodiment of the present invention is designed with integrated hardware and software to actively respond to changes in the internal environment of the culvert of an agricultural reservoir water intake facility. In particular, the combined structure of a variable 3-axis wheel and a steering device functions to prevent the robot from tipping over, slipping, or becoming unable to drive even when the floor condition inside the culvert varies by section or changes rapidly during driving.
[0167] In addition, the uneven shape and arrangement structure of the variable 3-axis wheels are designed not merely to improve traction, but to comprehensively consider the contact area with the inner bottom surface of the culvert, load distribution characteristics, and soil penetration performance. Accordingly, in the case of wheels for soil-deposited floor slabs, the wheel unevenness can push away or dig into the soil depending on the depth and viscosity of the soil to secure forward power, and wheels for submerged floor slabs minimize slipping caused by water film, thereby enabling stable driving.
[0168] Meanwhile, 3D point cloud data acquired from mobile LiDAR sensors is not only used to determine the presence or absence of obstacles, but is also utilized to comprehensively assess the cross-sectional shape inside the hull, sediment distribution patterns, and structural deformation status. In this case, an AI-based 3D point cloud analysis module comprehensively analyzes the shape, size, density, and location information of objects to classify the types of obstacles, and improves the manager's understanding by providing the reasoning basis for the classification results in the form of visual infographics.
[0169] In addition, the autonomous driving module and the obstacle avoidance module operate in conjunction with each other, and when an obstacle is detected, an avoidance path is automatically calculated by considering a pre-set safety distance, the robot's turning radius, and the drivable area. This avoidance path is updated in real time, enabling the robot to perform continuous filming without colliding with walls or obstacles, even in the narrow space inside the abdomen.
[0170] The 3D point cloud data and predicted movement paths displayed on the monitoring equipment can be expressed in the form of guidelines, color information, or icons, going beyond the simple provision of visual information to allow for an intuitive understanding of the robot's current status and future movement path. Through this, the manager can monitor the robot's autonomous driving status in real time from a remote location and perform manual intervention via a remote controller if necessary.
[0171] The control system integrates and manages video, sensor signals, and analysis results collected in real-time during filming. By providing obstacle classification results and the AI reasoning basis in the form of infographics, it functions as a decision support tool that goes beyond simple monitoring. Consequently, managers can quickly identify risk factors within the system and efficiently set maintenance priorities and take follow-up actions based on inspection results.
[0172] As such, the intelligent robot camera equipment configuration shown in FIGS. 4 and 5 provides technical effects that enable stable autonomous driving, precise obstacle recognition, and highly reliable safety inspection even in the complex and variable internal environment of an agricultural reservoir water intake facility.
[0174] FIG. 6 illustrates major damage (Type A) occurring in an agricultural reservoir water intake facility according to an embodiment of the present invention, and FIG. 7 illustrates major damage (Type B) occurring in an agricultural reservoir water intake facility according to an embodiment of the present invention.
[0175] The major damages illustrated in FIGS. 6 and 7 exemplify typical types of defects that may occur within the culvert of an agricultural reservoir water intake facility due to long-term use, changes in water pressure, environmental factors, and structural aging. In this case, Type A and Type B refer to damage patterns that are distinguished from one another based on the form of defect occurrence, progression, or impact on the structure.
[0176] Specifically, the major damage (Type A) illustrated in FIG. 6 is a type of defect that has a relatively distinct shape and is easy to visually identify in images, and may include surface cracks, localized spalling, discoloration of the concrete surface, or areas of partial damage. Since structural stability can be maintained through repair and reinforcement if such Type A damage is discovered in the early stages, early detection is particularly important.
[0177] On the other hand, the major damage (Type B) illustrated in Fig. 7 is a type of defect that includes minute surface changes, irregular shapes, or complex damage patterns, which are often difficult to identify by visual observation alone. For example, this may include potential leakage areas due to internal moisture penetration, temperature anomalies visible only in thermal images, or defects partially obscured by deposits.
[0178] According to an embodiment of the present invention, multimodal data obtained through an optical camera, a thermal imaging camera, and a lidar sensor is combined to enable the mutual detection of Type A and Type B damages illustrated in FIGS. 6 and 7. In particular, by comprehensively considering the morphological characteristics, contrast, temperature distribution, and spatial location information of the defect through artificial intelligence-based analysis, misidentification between defect types can be minimized.
[0179] In addition, the damage types illustrated in FIGS. 6 and 7 are used as training data for an artificial intelligence model to detect defects with higher accuracy when similar damage patterns occur in the future. In this way, by systematically classifying major damage types and reflecting them in model training, technical effects are achieved in which the accuracy and reliability of safety inspections of agricultural reservoir water intake facilities are continuously improved.
[0181] FIG. 8 is a configuration diagram of a subscription-based artificial intelligence-based infrastructure facility safety inspection platform, such as an agricultural reservoir water intake facility, according to an embodiment of the present invention, and FIG. 9 illustrates a UI of a subscription-based artificial intelligence-based infrastructure facility safety inspection platform, such as an agricultural reservoir water intake facility (abundance), according to an embodiment of the present invention.
[0182] The subscription-based AI-based safety inspection platform illustrated in Fig. 8 represents an integrated operational structure that transmits video data and sensing data collected from a filming robot to a cloud or server environment for storage, analysis, and management. This platform is provided as a Software as a Service (SaaS) model, allowing users to utilize safety inspection services for various infrastructure facilities, including agricultural reservoir water intake facilities, through a web browser or a dedicated interface without the need to install separate high-performance equipment on-site.
[0183] In addition, the platform configuration illustrated in Fig. 8 can be logically divided into a data collection layer, an artificial intelligence analysis layer, and a result provision layer, and data input from a camera robot is provided to the user after undergoing preprocessing, defect detection, XAI-based interpretation, and generative AI-based data expansion processes. Through this structure, stable service provision is possible even when multiple facilities, multiple users, and repetitive inspection tasks are performed simultaneously.
[0184] The platform UI illustrated in Fig. 9 is configured to allow users to intuitively manage infrastructure facilities subject to inspection and visually check inspection results. For example, various types of infrastructure facilities, such as bridges, tunnels, and pipelines as well as agricultural reservoir water intake facilities (culverts), can be managed on a project basis, and for each project, the shooting history, defect detection results, and analysis reports can be viewed in an integrated manner.
[0185] Furthermore, the platform UI supports managers in intuitively understanding complex analysis results by providing defect locations, types, and severity in the form of maps, images, heatmaps, or infographics. As a result, managers can quickly identify the status of defects and efficiently set maintenance priorities, establish repair plans, and formulate long-term facility management strategies.
[0186] As such, the subscription-based AI-based safety inspection platform illustrated in FIGS. 8 and 9 provides technical effects that improve the efficiency, scalability, and management convenience of infrastructure facility safety inspection work by providing on-site inspections based on filming robots and analysis results based on AI as a single integrated service.
[0188] FIG. 10 illustrates a configuration diagram of an artificial intelligence model dedicated to safety inspection of an agricultural reservoir water intake facility (abundance) according to an embodiment of the present invention.
[0189] Image data is used as input for detecting defects; each image is captured in high resolution and includes various information such as the shape, size, and location of the defects.
[0190] ResNet101 is a deep learning model based on a Residual Network that extracts image features using 101 layers.
[0191] As a model trained on large-scale datasets such as ImageNet, it is possible to extract basic image features without specialized training on defect images, and extracts low-level (edges, color) and high-level (patterns, shape) features of defect images.
[0192] Average Pooling (Avg Pool2D) is used to reduce the dimensionality of feature maps extracted from ResNet101 by averaging pixel values from each feature map to generate a feature vector. By reducing the model's computational load and retaining key features, the efficiency of defect classification is increased.
[0193] The Fully Connected Layer (FC) receives feature vectors as input, predicts the defect category, and outputs the defect type. It is possible to construct the final defect classification model through the FC layer.
[0194] The learned model parameters can be saved in a file format for use, and the saved model is loaded to perform predictions on new data.
[0195] The encoder includes a classification model and takes an input image to perform conversion into a feature vector.
[0196] LSTM (Long Short-Term Memory) is a type of RNN (Recurrent Neural Network) that generates text data and is capable of learning both long-term and short-term dependencies. It takes an embedding vector as input and generates a natural language description based on the features of a defect image.
[0197] The Fully Connected Layer (Decoder: FC) takes the output (hidden state) of the LSTM as input and predicts the next word at each step (defect, shape, triangle, etc.).
[0198] It provides natural language descriptions of defective images and visualizes the areas focused by the model during the image classification process through Grad-CAM (Gradient-weighted Class Activation Mapping). It calculates the output of the model's final convolutional layer and its gradient, performs a weighted sum to generate an activation map, and generates a heatmap by overlaying the activation map onto the original image.
[0199] A saliency map visualizes the sensitivity of an input image to a specific class. It calculates the model's output change rate (gradient) for each pixel in the input image and highlights areas with high change rates to indicate the parts the model considers important information.
[0200] Heatmaps visualize data generated by Grad-CAM or Saliency Maps, represent important parts of the image using color (e.g., red: high importance, blue: low importance), provide transparency in the defect detection and classification process, and help managers intuitively understand why the model classified specific defects.
[0201] Based on this configuration of the artificial intelligence model, the AI model dedicated to safety inspection of an agricultural reservoir intake facility (culvert) according to an embodiment of the present invention is designed to not merely classify defects but to comprehensively interpret the location and morphological characteristics of the defects. In particular, since the high-resolution input image data includes detailed defect features such as micro-cracks, spalling boundaries, and changes in surface texture inside the culvert, it is possible for the AI model to precisely analyze even the initial state of damage to the structure.
[0202] In the feature extraction process based on ResNet101, feature learning is performed using a residual connection structure that minimizes information loss even in deep layers. This enables the stable extraction of complex visual features, such as irregular patterns, repetitive structures, and wear marks from long-term use on abdominal surfaces. The extracted results are processed through an Average Pooling process to reduce dimensionality while preserving key information crucial for defect identification.
[0203] Furthermore, the defect categories output through the Fully Connected Layer are not limited to single-class predictions but can be provided in the form of probability distributions that consider the similarity or potential overlap between defect types. Accordingly, more realistic analysis results can be provided even when multiple defects exist simultaneously within a single image.
[0204] Meanwhile, a natural language description generation structure composed of an encoder and an LSTM-based decoder converts defect detection results into human-readable language, enabling managers without specialized image analysis knowledge to intuitively grasp the characteristics of defects. In this case, information regarding the location, size, shape, and surrounding environment of the defects is reflected in the sentence structure, so that descriptive results are output rather than a simple list of defect names.
[0205] The visualization process using Grad-CAM and Saliency Maps enhances the reliability of defect detection results by visually providing the basis for the AI model's judgments. For example, if the model identifies a specific crack as a defect, the area contributing to that judgment is superimposed on the original image in the form of a heatmap, allowing managers to intuitively verify the AI's decision-making process.
[0206] The heatmap generated in this way can express the importance or severity of defects through color intensity, allowing the manager to grasp the distribution of defects and risk areas within the abdomen at a glance. Consequently, the artificial intelligence model configuration illustrated in Fig. 10 provides technical effects that not only improve the accuracy of defect detection but also simultaneously ensure the interpretability of analysis results and practical applicability.
[0208] FIG. 11 illustrates an infographic dedicated to safety inspection of an agricultural reservoir water intake facility (abundance) according to an embodiment of the present invention.
[0209] For input images, the deep learning model uses a CNN to process the image and perform the tasks necessary to recognize specific objects or regions. The CNN analyzes input data through multiple convolutional layers that perform image feature extraction, while the RPN proposes object recognition and bounding boxes. The RPN generates candidate regions based on the input image and defines the location of objects through bounding boxes.
[0210] Modulized XAI utilizes explainable artificial intelligence techniques to support easy interpretation of prediction results from CNNs or RPNs. During the process of generating bounding boxes, a heatmap is created, visually indicating which parts of the image the network recognized as important features. Additionally, captioning is performed to explain the generated bounding boxes, providing text descriptions regarding the presence or location of objects.
[0211] SegNet is a neural network that performs image segmentation by analyzing images pixel by pixel to classify the category to which each pixel belongs. A segmented mask is a mask generated to distinguish specific objects or regions within an image, and SegNet accurately distinguishes the boundaries of objects within each bounding box, clearly visualizing the location and shape of the objects.
[0212] The segmented mask is input into a classification neural network and passes through multiple noise filters to remove noise. A denoised mask is generated through the noise filters, preserving important information while removing noise or errors.
[0213] M-Net, a model capable of simultaneously predicting various types of information, takes a denoised mask as input, extracts various features, and generates a quantized output based on them. Quantization refers to the process of converting the continuous characteristics of output values into a limited range, and it quantifies and outputs the area, width, and length information predicted by the model.
[0214] Intensity refers to the strength of pixel brightness within an image, and defects are observed as areas darker than the surrounding background; this is interpreted as a likely result of reduced light reflection caused by defects in the water intake facility. By utilizing brightness contrast, it is possible to recognize boundaries related to defects in the water intake facility and improve the accuracy of defect recognition.
[0215] In addition, based on the results of the shape analysis, if the defect shape is close to a triangle, the defect contour is extracted, and the most similar shape is extracted by comparing the contour with the geometric shape. According to an embodiment of the present invention, the defect of the corresponding geometric shape is analyzed.
[0216] According to an embodiment of the present invention, the characteristics of a specific region of interest within an image are expressed in natural language, and the characteristics (brightness, shape, size, location) of surface defects of a water intake facility are analyzed and output as text in a form understandable to humans. For example, text is output such as "Dark triangular defect area found on the surface," "The defect has lower intensity than the background and has a shape close to a triangle," or "A triangular defect was identified in the lower left corner."
[0217] According to an embodiment of the present invention, a manager can receive important information about a defect in a summarized form, which supports the manager in quickly understanding the characteristics of the defect and determining appropriate follow-up measures, and has the effect of reducing analysis time.
[0218] According to an embodiment of the present invention, a defect detection and analysis process is performed by utilizing explainable artificial intelligence during the captioning process, and by providing not only simple defect visualization (heatmap) but also specific descriptions of defect characteristics, it supports managers in understanding the characteristics of defects more accurately and deeply.
[0219] A CNN (Convolutional Neural Network)-based network analyzes the contours and brightness of defects within an image and converts the extracted image feature data into a continuous text format using LSTM (Long Short-Term Memory).
[0220] According to an embodiment of the present invention, by visualizing defects in water intake facilities as a heatmap, it is possible to support more intuitive verification of defect analysis results.
[0221] A heatmap is defined as a graphic representation that supports managers in achieving a visual and intuitive understanding by expressing specific characteristics of data using color. According to an embodiment of the present invention, the severity, probability of occurrence, or importance of a specific defect is expressed using color. In this case, red and orange colors indicate high importance or severity, while blue and green colors indicate low importance or a less severe state.
[0222] According to an embodiment of the present invention, the importance of defect characteristics is determined by assigning weights to defect size (physical size of the defect area), depth (degree of damage), type (crack, corrosion, etc.), and frequency of occurrence (frequency of occurrence of a specific defect). For example, relatively high weights are assigned to defects that are large in size or deep in depth, and it is possible to consider specific patterns (continuity of cracks, etc.) depending on the defect type.
[0223] For image or scan data, a defect area is extracted through image processing, a deep learning-based algorithm (XAI, SegNet, etc.) learns the features of the defect area, and the importance (weight) of each feature is calculated.
[0224] Based on the analyzed weight data, a color is assigned to each defect area, a heatmap is displayed as an overlay on the input image, and importance information is superimposed on the surface image along with the defect location.
[0225] Finally, the heatmap provides visualization results that enable intuitive understanding of the severity and distribution of defects.
[0226] Areas where defects are concentrated display a more intense color, while areas where defects are relatively evenly distributed across the entire surface display a continuous, wide color. According to an embodiment of the present invention, not only the size of the defect but also the importance varying according to severity and type is supported through a weight-based representation, enabling intuitive recognition and subsequent action.
[0227] In accordance with such infographic configuration, the embodiment of the present invention provides defect detection results not as simple image output or numerical data, but as visual information integrating the location, shape, importance, and analysis basis of the defect. Through this, the manager can grasp the defect status inside the abdomen at a glance and intuitively recognize the area where the defect occurred and its severity.
[0228] In particular, the structure in which CNN, RPN, SegNet, M-Net, and XAI modules are interconnected stepwise is designed to simultaneously ensure both the accuracy and interpretability of defect detection. The bounding boxes and pixel-level segmentation results proposed in the object recognition stage are utilized complementarily, enabling stable defect region extraction even when defect boundaries are unclear or have complex shapes.
[0229] Furthermore, the quantification process performed based on the noise-removed segmented mask quantifies physical characteristics such as defect area, length, and width, enabling objective comparison and historical management beyond qualitative judgment. This quantitative information can be utilized to compare inspection results from multiple points in time or to evaluate the progression of defects within the same facility.
[0230] Intensity-based analysis and geometric shape analysis are combined to prevent misidentification caused by simple changes in brightness and contrast, and to more precisely identify areas corresponding to actual structural defects. As a result, the possibility of image distortion due to surface contamination, lighting changes, or shooting angles being misidentified as defects is reduced.
[0231] Furthermore, natural language captioning results are provided along with heatmaps and quantified numerical information, enabling managers to understand the status of defects without requiring specialized expertise. This reduces analysis time during the reporting, recording, and decision-making processes of inspection results, thereby improving management efficiency.
[0232] As such, the safety inspection infographic illustrated in Fig. 11 provides an integrated visualization, quantification, and verbalization of AI-based defect detection results, thereby providing a technical effect that enables the maintenance of agricultural reservoir water intake facilities to be performed more systematically and reliably.
[0233] FIG. 12 illustrates a cross-modal generative artificial intelligence model structure that understands the characteristics of a 3D mesh included in a user's text input and the conditions for generating the mesh, and provides a function to automatically output the mesh. Referring to FIG. 12, a description of the conditioning mesh generation for additional data is illustrated.
[0234] According to an embodiment of the present invention, a cross-modal generative artificial intelligence model structure is proposed that performs the generation and manipulation of a 3D mesh and can automatically generate a 3D mesh that satisfies specific conditions and features based on text input.
[0235] Existing 3D mesh generation technologies have primarily relied on specialized software and complex user interfaces, posing a problem of being difficult for non-experts to access. Furthermore, technologies that generate desired meshes using natural language input, such as text, have faced significant limitations in the effective mapping and generation processes between text and 3D data. To address these issues, a crossmodal AI model is required that learns the correlation between text and 3D meshes and can automatically generate meshes based on text input by the user.
[0236] According to an embodiment of the present invention, a crossmodal generative artificial intelligence model is provided that supports a user in efficiently generating a 3D mesh with specific features and conditions through text input. Through this, even non-experts can easily create 3D content.
[0237] The text encoder processes text input by the user and converts it into high-dimensional feature vectors that describe the characteristics of the desired 3D mesh. To achieve this, it utilizes natural language processing (NLP) technology and deep neural networks (DNN).
[0238] A mesh encoder processes 3D mesh data to extract structural features of the mesh. In this process, feature vectors are generated that reflect the geometric characteristics and topological information of the mesh.
[0239] The joint feature representation module fuses feature vectors generated from the text encoder and mesh encoder into a joint feature representation, learns the association between the two data modalities during the fusion process, and provides information necessary to generate a suitable 3D mesh from the text.
[0240] The mesh decoder is a decoder that generates an actual 3D mesh based on information obtained from joint feature representations, outputs the geometric shape and structure of the 3D mesh, and is designed to satisfy the user's text input conditions.
[0241] When a user provides the generation conditions and characteristics of a 3D mesh via text input, a text encoder analyzes this and converts it into unique feature vectors. At the same time, existing meshes stored in the database are processed through a mesh encoder to extract structural features. The feature vectors are fused during the joint feature representation stage, and a mesh decoder generates a mesh that satisfies the user's conditions based on this information.
[0242] When a user inputs text such as "a mesh-shaped structure with a spherical geometric pattern," the model according to an embodiment of the present invention automatically generates a 3D mesh that satisfies the corresponding conditions based on this.
[0243] According to such a cross-modal generative artificial intelligence model structure, embodiments of the present invention are designed to effectively resolve the semantic gap between different data modalities, such as text and 3D mesh. In particular, feature vectors extracted from the text encoder and the mesh encoder, respectively, are not combined in simple parallel but are fused into a joint feature representation that considers semantic consistency, thereby ensuring that the user's natural language input is faithfully reflected in the actual 3D geometric structure.
[0244] In addition, the process of generating a conditioning mesh for additional data can be extended to flexibly respond to cases where multiple condition parameters, such as the size range of the structure, surface roughness, density, connection pattern, or usage information, are input together with basic text input. Accordingly, it becomes possible to generate different types of 3D meshes even under the same text conditions.
[0245] Structural features extracted through the mesh encoder are not limited to simple vertex coordinate information but include face configuration, adjacency relationships, topological structure, and curvature information, so the generated 3D mesh can maintain not only visual similarity but also structural consistency. These structural features contribute to minimizing geometric distortion at the mesh decoder stage and outputting a mesh with a shape suitable for practical use.
[0246] Furthermore, the cross-modal generative artificial intelligence model according to the embodiment of the present invention can be utilized in the process of building a digital twin model, and it is possible to rapidly reproduce the internal structure of a pipe or the shape of a water intake facility using only text-based commands. Through this, even when on-site shooting data is incomplete or data for some sections is missing, a reasonable 3D structure can be supplemented and generated using text conditions.
[0247] In addition, the 3D mesh generated based on text entered by the user can be utilized for subsequent simulations, structural analysis, or maintenance planning, and it is also possible to iteratively modify and supplement the generated mesh as needed. As such, the cross-modal generative artificial intelligence model structure illustrated in FIG. 12 improves the accessibility of 3D mesh generation and provides technical effects that significantly enhance the efficiency of digital twin construction in the field of infrastructure facility management.
[0248] FIG. 13 illustrates a cross-modal generative artificial intelligence model structure that understands the characteristics of cracks included in a user's text input and crack generation conditions, and provides a function to automatically output cracks.
[0249] According to an embodiment of the present invention, by utilizing a Generative AI model that analyzes crack characteristics and crack generation conditions included in a user's text input and provides a function to automatically output cracks based thereon, the time and cost required to effectively collect and generate crack data of infrastructure facilities are reduced, and the learning accuracy of the AI model is improved.
[0250] The process of collecting and analyzing crack data for the maintenance of infrastructure facilities is time-consuming and costly. Traditionally, collecting crack data requires capturing images directly on-site or generating data within specific environments, which consumes significant physical and human resources. To address this, technologies for artificially generating crack data have been proposed, but existing techniques have the following limitations. Artificially adding new cracks to images that do not contain them is unrealistic, and the generated data is highly likely to fail to reflect the actual environment. Furthermore, using unrealistic crack data can actually degrade performance during the AI model training process.
[0251] According to an embodiment of the present invention, a generative AI model is provided that utilizes an image containing fine cracks to understand a user's text command and generates modified crack data based thereon.
[0252] According to an embodiment of the present invention, a generative AI model includes a user input analysis module that analyzes a text command entered by a user to extract key features such as the shape, size, direction, and density of a crack, as well as crack generation conditions. For example, if a user enters "long and thin crack," the corresponding command is parsed to set appropriate crack generation conditions.
[0253] According to an embodiment of the present invention, a generative AI model includes a data-driven learning module that trains the AI model using real images containing fine cracks. During the training process, generative AI technologies such as a Generative Adversarial Network (GAN) or a Diffusion model are utilized to expand and generate crack data.
[0254] According to an embodiment of the present invention, a generative AI model includes a crack generation module that generates a crack image based on user commands and a learned model. To generate realistic cracks, this includes a texture matching algorithm that generates cracks that harmonize with the surface texture of an existing image, and a physical crack modeling algorithm that generates a natural crack shape by simulating the growth path and physical characteristics of the cracks.
[0255] According to an embodiment of the present invention, a generative AI model includes a result output module that saves the generated crack image in various formats (e.g., PNG, JPEG) or provides it to a user through a visualization tool. The generated crack image can be directly utilized for AI model training or infrastructure analysis.
[0256] According to an embodiment of the present invention, there is a technical feature of dynamically generating crack images that meet the user's requirements through user command-based crack generation, rather than simply replicating data.
[0257] According to an embodiment of the present invention, it is possible to provide realistic data by learning the characteristics of cracks occurring in a real environment.
[0258] According to an embodiment of the present invention, it is possible to generate a large amount of crack data without existing data through generative AI technology, thereby significantly reducing training costs.
[0259] According to an embodiment of the present invention, it can be utilized in a crack detection and prediction system for infrastructure facilities such as reservoir water intake facilities, efficiently generate a dataset for training an AI model in an environment where crack data is scarce, and perform simulations of crack patterns of structures using crack data.
[0260] Based on such a crossmodal generative artificial intelligence model structure, embodiments of the present invention are designed to reflect, to the fullest extent, the occurrence mechanisms and visual characteristics of cracks observed in actual infrastructure facilities. In particular, since learning is performed based on actual images containing microcracks, the generated crack data naturally includes the continuity, branching structure, and irregularity of cracks observable in the actual environment, rather than simple synthetic patterns.
[0261] In addition, crack generation conditions extracted from the user input analysis module are converted into detailed parameters such as the length direction, propagation angle, branching status, and density of the crack, and are passed to the crack generation module. Accordingly, even with the same text command, different types of crack images can be generated depending on the combination of input conditions, which contributes to effectively expanding the diversity of training data.
[0262] The texture matching algorithm included in the crack generation module analyzes the surface texture, brightness distribution, and noise characteristics of existing images to ensure that the generated cracks blend naturally with the surrounding background without any sense of incongruity. In addition, the physical crack modeling algorithm simulates physical properties such as stress distribution, crack propagation direction, and growth rate to generate shapes similar to cracks occurring in actual structures.
[0263] Furthermore, the generative AI model according to an embodiment of the present invention may include a process of iteratively evaluating generated crack images and excluding or correcting unrealistic results that do not contribute to the improvement of the learning model's performance. Through this, the quality of the generated data can be continuously maintained, and a crack dataset optimized for AI model training can be constructed.
[0264] As such, the crossmodal generative artificial intelligence model illustrated in FIG. 13 effectively solves the problem of crack data scarcity and provides technical effects that enable the generation of highly reliable data that can be used for crack detection, progression prediction, and maintenance strategy formulation for infrastructure facilities such as reservoir water intake facilities.
[0265] Figure 14 illustrates a schematic diagram of a model that generates a 3D mesh for building a generative AI-based digital twin model.
[0266] FIG. 14 illustrates a schematic diagram of a model that generates a 3D mesh for building a generative AI-based digital twin model.
[0267] According to an embodiment of the present invention, in relation to 3D computer graphics and deep learning, a 3D SDF (indicated distance field) and a texture field are generated through end-to-end learning, a 3D surface mesh is extracted based thereon, and high-quality 2D and 3D rendered images are generated using adversarial learning techniques.
[0268] According to an embodiment of the present invention, a 3D SDF and a texture field are generated using DMTet, a deep learning-based expanded tetrahedron generation model, thereby providing a high-quality 3D model and a 2D rendered image.
[0269] According to an embodiment of the present invention, a Sign Distance Field (SDF) is generated in 3D space through a deep learning-based learning model. The SDF represents the distance and direction from each point in space to a surface and defines the shape of a 3D object. The generated SDF is converted into a 3D surface mesh using a DMTet model.
[0270] DMTet is a deep learning model that represents 3D objects as a tetrahedral-based unfolded structure and efficiently extracts accurate and detailed 3D surface meshes from SDF. At this time, the vertices and faces of the mesh are automatically generated, and the geometric features of the surface can be preserved.
[0271] After the surface mesh is extracted, the color of each point is obtained using the texture field of the points existing on the surface. The texture field contains color information for the visual representation of a 3D object.
[0272] The texture field is generated through a surface point extraction process that calculates texture coordinates (UV Mapping) at each vertex of the mesh surface, a texture field request process that predicts the color value of each surface point through a deep learning model and expresses the predicted color value in RGB channels to complete the appearance of the 3D object.
[0273] According to an embodiment of the present invention, a 3D model is converted into a 2D image to perform adversarial learning, and a rasterization-based differential renderer is used to generate a high-quality 2D image utilizing a 3D mesh and a texture field. At this time, an RGB image and a silhouette of an object are generated, and this process is designed to be differentiable so that it can be used for training a deep learning model.
[0274] In addition, training is performed using an adversarial loss value defined for 2D images, and this adversarial loss induces training so that the generated images are indistinguishable from real images.
[0275] According to an embodiment of the present invention, the quality of the generated data is evaluated by utilizing a 2D discriminator / discriminator for each RGB image and silhouette. The discriminator determines whether the input image is real (actual data) or fake (generated data), thereby increasing the realism of the generated 2D image and improving the learning performance of the deep learning model.
[0276] According to an embodiment of the present invention, it is designed as an end-to-end learning structure, so that all processes from data input to output are automated.
[0277] Through this, it is possible to perform 3D SDF generation, mesh extraction, texture field generation, 2D rendering, and adversarial learning integrally within a single model, and to maximize the precision of 3D and 2D data.
[0278] According to the 3D mesh generation structure for constructing a generative AI-based digital twin model, the embodiment of the present invention aims to reproduce the structural shape and visual characteristics of an actual agricultural reservoir water intake facility with high precision. In particular, through an end-to-end structure that simultaneously learns and generates 3D SDF and texture fields, a digital twin model that satisfies both geometric accuracy and visual realism can be efficiently constructed.
[0279] SDF-based representations enable higher resolution and smoother surface representation compared to existing voxel-based or simple mesh-based representation methods by defining complex internal structures, curved surfaces, edges, and irregular shapes as continuous distance functions. Accordingly, the generated 3D surface mesh can more accurately reflect minute shape changes of the structure or geometric features of damaged areas.
[0280] Furthermore, the mesh extraction process using DMTet utilizes a tetrahedral-based unfolding structure, enabling stable mesh generation even for 3D objects with complex shapes while minimizing mesh distortion or the occurrence of holes. The vertex and face information automatically generated during this process is configured to be immediately usable for subsequent simulation, visualization, and structural analysis of the digital twin model.
[0281] The texture field generation process goes beyond simply applying color information; by reflecting changes in surface brightness and contrast, material properties, and visual patterns based on the shooting environment, a 3D model with an appearance similar to the actual facility is created. Accordingly, the generated digital twin model contributes not only to the visual review of inspection results but also to providing a virtual environment that managers or decision-makers can intuitively understand.
[0282] In addition, the 2D image generation and adversarial learning process using a differentiable renderer plays a role in continuously correcting the generated 3D model so that it visually matches the actual captured image. Through this, the digital twin model evolves beyond simple shape reproduction into a virtual model with high consistency with the actual inspection environment.
[0283] As such, the 3D mesh generation structure for constructing a generative AI-based digital twin model illustrated in FIG. 14 enables the implementation of a high-precision digital twin that comprehensively reflects the spatial structure, surface condition, and damage information of an agricultural reservoir water intake facility, and provides technical effects that can be utilized for the analysis of safety inspection results, simulation, and the establishment of long-term maintenance strategies.
[0284] AI Agent Dedicated to Crack and Defect Detection
[0285] The term "AI Agent" in the present invention does not refer to a single artificial intelligence model or a simple image recognition algorithm, but rather to an intelligent agent in which a plurality of functional modules are organically combined to set and update judgment criteria based on feature information extracted from an input image and crack and defect detection results, and to autonomously adjust defect detection strategies and parameters.
[0286] The above AI Agent is distinguished from rule-based systems that operate according to pre-fixed rules or artificial intelligence models that perform only simple inference, and is characterized by iteratively and adaptively improving crack and defect detection performance by including decision logic that performs an internal evaluation of detection results and reflects the evaluation results in subsequent detection processes.
[0287] The AI Agent-based crack and defect detection automation system according to the present invention is intended to automatically detect and analyze defects such as cracks, spalling, delamination, and corrosion by using images of infrastructure facilities such as reservoirs, dams, and tunnels as input.
[0288] The system includes an image input unit, an AI Agent, and a result output unit, and the AI Agent is implemented as an autonomous defect detection entity including a feature extraction module, a detection model, an inference engine, and a decision module.
[0289] The video input unit collects images of infrastructure facilities acquired by drones, mobile robots, fixed cameras, or portable recording devices. The collected images undergo preprocessing steps, such as resolution normalization, distortion correction, and noise removal, before being transmitted to the AI Agent.
[0290] In this process, environmental factors such as shooting angle, lighting conditions, and shooting distance can be stored together as metadata, and said metadata can be utilized in the decision-making process of the AI Agent later.
[0291] The feature extraction module extracts defect-related feature information from the input image, such as crack boundaries, orientation, texture changes, and contrast. The feature extraction may be performed using a Convolutional Neural Network (CNN), a Vision Transformer (ViT), or a combination thereof, but is not limited thereto.
[0292] The detection model receives the above feature information as input and detects candidate regions for cracks and defects. In this case, defect detection can be performed using an object detection method, a segmentation method, or a hybrid method, and can simultaneously detect microcracks and large-scale defects by utilizing multi-scale features.
[0293] The inference engine analyzes detected defect candidates to determine the type, location, length, width, distribution pattern, or severity of the cracks. Additionally, when multiple images are input, it can infer whether the defect has progressed through the analysis of time-series changes.
[0294] The decision module evaluates the output of the inference engine and dynamically adjusts at least one of the detection threshold, region of interest, resolution selection, and the order of application of the detection model based on the evaluation result. Through this, the AI Agent autonomously selects the optimal defect detection strategy according to the shooting environment or facility type.
[0295] The crack and defect analysis results derived by the AI Agent are visualized on images or stored in the form of defect location coordinates, severity indices, risk scores, etc. These results can be utilized by linking with facility maintenance systems, digital twin platforms, or inspection history management systems.
[0296] According to the configuration of such an AI Agent, in an embodiment of the present invention, the AI Agent dedicated to crack and defect detection operates to autonomously select an optimal detection strategy based on the inspection environment, shooting conditions, and defect distribution characteristics. For example, the internal operation logic can be adjusted such that the weights of contrast and texture-based features are increased in environments with low illumination or severe surface contamination, and parameters specialized for microcrack detection are preferentially applied in sections where cracks are widely distributed.
[0297] Furthermore, by cumulatively analyzing multiple detection results, the AI Agent can perform defect determination considering time-series change patterns without relying on the analysis of a single image or single frame. Accordingly, it becomes possible to infer whether a crack has expanded, the rate of defect progression, or the occurrence of a new defect by comparing the differences between images repeatedly captured at the same location.
[0298] Furthermore, the decision logic of the AI Agent can be continuously updated by reflecting external inputs such as detection reliability, false positive frequency, and administrator feedback. For example, if recurring false positives occur for a specific type of defect, the detection threshold for that defect type can be automatically adjusted, or additional verification procedures can be performed during subsequent analysis steps.
[0299] Thus, the AI Agent dedicated to crack and defect detection according to the embodiment of the present invention goes beyond a simple automation tool and functions as an intelligent entity that adapts to the inspection environment and improves its performance autonomously, thereby achieving the technical effect of providing high reliability and scalability in the long-term safety management of infrastructure facilities, including agricultural reservoir water intake facilities.
[0300] FIG. 15 illustrates the indoor positioning value generation and matching results with damage data using a multi-sensor robo-inspector according to an embodiment of the present invention.
[0301] An accelerometer, gyroscope, odometry, and lidar are attached to a device (Robo Inspector) that acquires damage data for temporary construction equipment, equipped with an optical imaging sensor, a laser profiling sensor, and a magnetic flux sensor; indoor positioning values are generated from the acquired data, and the result of matching the indoor positioning values with the damage data is provided.
[0302] According to an embodiment of the present invention, a multi-sensor robo-inspector is configured to acquire damage data while moving autonomously or semi-autonomously, and the position information and damage information generated during the movement process are processed in synchronization on the same time axis. Accordingly, the position of the robo-inspector at the time when specific damage is detected can be accurately calculated in an indoor coordinate system.
[0303] According to an embodiment of the present invention, an acceleration sensor and a gyroscope sensor provide information on attitude changes, rotational angular velocity, and acceleration of the Robo Inspector, and are utilized to correct errors caused by vibrations or tilt changes occurring during movement. Odometry estimates a relative movement path based on the amount of wheel rotation or distance traveled, and LiDAR measures distance information with surrounding structures to generate a spatial map of the indoor environment and correct the position estimation results.
[0304] According to an embodiment of the present invention, data acquired from sensors is integrated and processed by a processor through a sensor fusion algorithm, thereby enabling the generation of highly reliable indoor positioning values even in environments where indoor GPS is not available. The generated indoor positioning values are converted into coordinate information based on a base point, providing a spatial reference for the entire inspection area.
[0305] According to an embodiment of the present invention, damage data acquired through an optical image sensor, a laser profiling sensor, and a magnetic flux sensor is automatically matched with the indoor positioning value as information reflecting surface defects, shape deformation, internal or material abnormalities, respectively. At this time, information regarding the type, size, severity, and detection time of the damage is stored together with location coordinates, so that the damage data is managed in a form linked to spatial information.
[0306] According to an embodiment of the present invention, the matching result between indoor positioning values and damage data can be visually displayed on a two-dimensional floor plan or a three-dimensional workspace model, and the movement path of the robo-inspector and the location of each damage are shown together. Through this, the manager can intuitively identify the exact location where damage occurred and can quickly access the location during subsequent inspection or maintenance work.
[0307] According to an embodiment of the present invention, when repeated inspections are performed on the same space, it is possible to align past inspection results with current inspection results based on location to analyze whether new damage has occurred, whether it has expanded, or the rate of progression. Accordingly, it becomes possible to manage damage history and analyze changes in state over time, going beyond inspection results at a single point in time.
[0308] Accordingly, by using a multi-sensor robo-inspector to integrally manage damage data and location information, it provides technical effects that improve the accuracy, reproducibility, and management efficiency of quality inspection.
[0309] FIG. 16 illustrates a generative artificial intelligence model-based crack generation-detection integrated pipeline according to an embodiment of the present invention.
[0310] According to an embodiment of the present invention, a generative artificial intelligence model-based crack generation-detection integrated pipeline is configured such that crack data generation, quality verification, detection model training, and iterative improvement are performed in a single continuous structure using an original image as input. In this case, the original image is an image captured of the surface of an actual infrastructure facility and may include an image in a normal state where cracks exist or where cracks do not exist.
[0311] According to an embodiment of the present invention, a generative artificial intelligence model restrictively designates the area where cracks are to be generated by using mask information along with the input original image. By defining candidate regions for crack generation, the mask prevents cracks from being generated randomly across the entire image and controls the formation of cracks only in areas where actual cracks are likely to occur. Through this, the generated cracks have a spatial distribution similar to that of the actual environment.
[0312] In addition, according to an embodiment of the present invention, a user-defined prompt consists of text information describing the type, size, shape, and texture of the crack, such as “wall with a damaged surface,” “fine and continuous crack,” or “crack on the surface of aged concrete,” and a generative artificial intelligence model controls the direction, thickness, branching shape, and density of the crack generation based on the prompt. At this time, a caption automatically generated from the original image using an image captioning model such as BLIP2 can be used as the prompt, thereby enabling the generation of cracks while maintaining the style and consistency of the original image.
[0313] According to an embodiment of the present invention, a generative artificial intelligence model utilizes Stable Diffusion or a similar diffusion-based generative model to output a generated image containing cracks based on an input image, a mask, and a prompt. In this process, the cracks are generated to naturally fuse with the existing surface texture, thereby minimizing traces of artificial synthesis.
[0314] According to an embodiment of the present invention, the generated crack image undergoes quality verification through a filter step. In the filter step, the generated result is evaluated based on the continuity of the crack, contrast, thickness variation, and harmony with the background, and unrealistic patterns or images unsuitable for detection training are removed. For images that pass the filtering, a Ground Truth (GT) mask is automatically generated based on the generated crack region, and the GT mask is utilized as ground truth data for training a crack detection model.
[0315] According to an embodiment of the present invention, the filtered generated image and the GT mask are input into the iterative training of a crack detection model. At this time, the crack detection model starts training using a pretrained model as an initial value and repeatedly performs training and testing on the generated crack data. Through this, stable detection performance can be secured not only for actual data but also for generated data.
[0316] According to an embodiment of the present invention, images in which cracks are not properly detected or false detections occur during the detection model training process are classified as detection failure images. These detection failure images are fed back to a generative artificial intelligence model, and new crack images are regenerated by modifying the seed, guidance, mask, or prompt conditions. This process is repeated in a direction that strengthens the crack patterns of types that the crack detection model is vulnerable to.
[0317] According to an embodiment of the present invention, the performance of the generative artificial intelligence model and the crack detection model are mutually enhanced through an iterative structure; in the generation stage, crack types that are difficult to detect are progressively added, and in the detection stage, generalization performance for various crack patterns is strengthened. Accordingly, the entire pipeline is configured in a closed-loop form, enabling continuous performance improvement.
[0318] In addition, according to an embodiment of the present invention, the integrated pipeline can be utilized not only in the model training stage but also in the inference stage in an actual inspection environment, and provides an adaptive learning structure that immediately generates new crack data and retrains the detection model when detection performance deteriorates for new images collected in the field.
[0319] Accordingly, the embodiment of the present invention provides technical effects that solve the problem of crack data scarcity, continuously improve the performance of the detection model, and significantly enhance the accuracy and reliability of crack diagnosis in infrastructure facilities by implementing generative AI-based crack data generation and crack detection model training into a single integrated pipeline.
[0320] FIG. 17 illustrates a generative artificial intelligence model capable of cross-modal learning that outputs a 3D mesh for building a digital twin model according to an embodiment of the present invention.
[0321] A generative AI model capable of cross-modal learning that outputs a 3D mesh for building a digital twin model understands the characteristics of the 3D mesh included in the image and text inputs provided by the user and the conditions for generating the 3D mesh, and provides the ability to automatically generate a low-resolution, coarse mesh in the first step and a high-quality 3D mesh with detailed textures in the second step.
[0322] According to an embodiment of the present invention, a generative artificial intelligence model capable of cross-modal learning has a multimodal structure that uses image input and text input simultaneously as conditions, and is configured to generate a 3D mesh suitable for constructing a digital twin model by progressively refining the shape and semantic characteristics of a 3D object. At this time, the image input provides visual information including the appearance of the actual infrastructure, and the text input provides semantic constraints such as the number of floors, structural features, morphological attributes, texture, and detailed shape conditions.
[0323] According to an embodiment of the present invention, in Step 1, a low-resolution, coarse 3D shape satisfying input image and text conditions is generated by utilizing a representation based on Instant NGP (Neural Graphics Primitives). In this step, the density, color, and geometric structure within the neural field are generally optimized using a low-resolution image diffusion prior, and the overall contour, proportion, and spatial arrangement of the object are formed first. Through this, a stable 3D structure corresponding to the basic framework of the digital twin model is rapidly converged.
[0324] According to an embodiment of the present invention, the representation generated in step 1 undergoes a 3D mesh extraction process based on DMTet (Deep Marching Tetrahedra) and is converted into a crude 3D mesh having a tetrahedral unfolded structure. In this process, Signed Distance Field (SDF) information is utilized to maintain surface continuity and geometric consistency, and a basic mesh that can be used in a digital twin environment is generated.
[0325] According to an embodiment of the present invention, in the second step, high-resolution optimization is performed using a high-resolution latent diffusion prior with a coarse 3D mesh as input. In this step, the detailed shape, surface curvature, structural details, and texture information of the object are enhanced, and semantic conditions included in the text input are reflected more precisely. In particular, the floor structure of the building, exterior wall patterns, and visual characteristics of the material are progressively enhanced in the high-resolution latent space.
[0326] According to an embodiment of the present invention, in the high-resolution optimization process, a differentiable rendering structure is utilized to generate multi-angle 2D rendering images from the generated 3D mesh, and an additional loss function is calculated based on this. Through this, learning is induced to minimize the visual difference between the actual image and the generated image, and as a result, a high-quality 3D mesh with high realism is output.
[0327] According to an embodiment of the present invention, through a two-stage generation structure, the overall shape is first stably secured and then detailed information is selectively enhanced, thereby simultaneously satisfying computational efficiency and generation quality. Accordingly, it is possible to automatically generate a high-resolution 3D mesh suitable for building digital twin models, even for buildings or large-scale facilities with complex structures.
[0328] Accordingly, the generative artificial intelligence model capable of cross-modal learning according to the embodiment of the present invention supports the automatic output of a high-quality 3D mesh for building a digital twin model using only image and text inputs provided by the user, and provides a technical effect that significantly reduces time, cost, and dependence on professional personnel compared to the existing manual-based 3D modeling method.
[0329] FIG. 18 illustrates an AI agent dedicated to detecting infrastructure facility defects according to an embodiment of the present invention.
[0330] The AI Agent dedicated to infrastructure defect detection illustrated in Fig. 18 represents an intelligent decision-making structure in which external data sources, vector DBs, language models, and multiple tools (APIs) are organically linked around a user query. In this case, the AI Agent does not merely perform the role of responding to user queries, but functions as an entity that establishes a work plan optimized for the specific purpose of defect detection and executes it step by step.
[0331] The query entered by the user may include the type, location, severity, and scope of the analysis request for the defect, and the language model interprets the query to plan the necessary work steps. During this process, the AI Agent searches past inspection history, defect cases, and learned feature information stored in the vector database to extract information semantically similar to the query, and utilizes this for defect analysis.
[0332] The defect data API is an interface that provides structured data related to infrastructure defects, such as cracks, spalling, and leakage, and is used by AI agents to verify or supplement defect detection results. For example, by comparing the consistency between image-based detection results and defect patterns accumulated in a database, it is possible to determine whether a detection is false positive or to correct the reliability of the defects.
[0333] External data sources periodically update the latest information, such as inspection standards, maintenance guidelines, or environmental data, and reflect it in the Vector DB. Based on this information, the AI Agent re-evaluates the risk of defects or makes decisions that meet management standards. Consequently, the AI Agent is capable of performing defect analysis that incorporates the latest information, rather than relying on fixed knowledge.
[0334] The AI Agent synthesizes search results and tool call results to generate a final response. This response may include not only the defect detection results but also an explanation of the cause of the defect, a severity assessment, whether priority action is required, and recommendations for future inspections. These responses are provided to managers in an easy-to-understand format, supporting rapid and rational decision-making.
[0335] The AI Agent dedicated to infrastructure defect detection illustrated in Fig. 18 integrates language models, data search, defect-dedicated APIs, and external information sources to function as an intelligent defect management entity capable of judgment, planning, and execution beyond simple analysis, thereby providing technical effects that dramatically improve the level of automation and reliability of infrastructure safety inspections.
[0337] FIG. 19 is a block diagram showing a computer system for implementing a method according to an embodiment of the present invention.
[0338] Referring to FIG. 19, a computer system (1300) may include at least one of a processor (1313), memory (1330), an input interface device (1350), an output interface device (1360), and a storage device (1340) that communicate via a bus (1370). The computer system (1300) may also include a communication device (1320) coupled to a network. The processor (1310) may be a central processing unit (CPU) or a semiconductor device that executes instructions stored in memory (1330) or storage device (1340). Memory (1330) and storage device (1340) may include various forms of volatile or non-volatile storage media. For example, memory may include read-only memory (ROM) and random access memory (RAM). In the embodiments of this description, memory may be located inside or outside the processor, and memory may be connected to the processor through various known means. Memory is a volatile or non-volatile storage medium of various forms, and for example, memory may include read-only memory (ROM) or random access memory (RAM).
[0339] Accordingly, embodiments of the present invention may be implemented as a method implemented on a computer or as a non-transient computer-readable medium storing computer-executable instructions. In one embodiment, when executed by a processor, the computer-readable instructions may perform a method according to at least one aspect of the present description.
[0340] The communication device (1320) can transmit or receive wired or wireless signals.
[0341] In addition, the method according to an embodiment of the present invention may be implemented in the form of program instructions that can be executed through various computer means and may be recorded on a computer-readable medium.
[0342] The above computer-readable medium may include program instructions, data files, data structures, etc., either individually or in combination. The program instructions recorded on the computer-readable medium may be specially designed and configured for embodiments of the present invention, or they may be known and available to a person skilled in the art of computer software. The computer-readable recording medium may include a hardware device configured to store and execute program instructions. For example, the computer-readable recording medium may be magnetic media such as hard disks, floppy disks, and magnetic tapes; optical recording media such as CD-ROMs and DVDs; magneto-optical media such as floptical disks; ROM; RAM; flash memory, etc. The program instructions may include not only machine code, such as that generated by a compiler, but also high-level language code that can be executed by a computer through an interpreter, etc.
[0344] FIG. 20 illustrates a pipeline composed of three stages: BIM-based session data generation, multi-session anchoring, and map updating, according to an embodiment of the present invention.
[0345] A subscription-based software and generative AI-based safety inspection system for internal tunnels and reservoir intake facilities of dam facilities according to an embodiment of the present invention performs BIM (Building Information Modeling)-linked simultaneous position estimation and mapping (BIM-SLAM) to perform AI-based safety inspections inside internal tunnels and reservoir intake facilities of dam facilities, and further includes a BIM-SLAM-based indoor AI safety inspection unit that improves the position estimation accuracy of SLAM and map reliability by utilizing a BIM model containing geometric structure and attribute information of the facility as static reference information. The BIM-SLAM-based indoor AI safety inspection unit comprises: a BIM data input unit that receives BIM model data containing geometric information and attribute information of the facility; a sensor data acquisition unit that acquires sensor data including at least one of LiDAR data, image data, and inertial data from a sensor for acquiring indoor environment data; and a data matching and position estimation unit that estimates the current position and attitude of the system and generates an initial environment map by matching structural features including planar elements extracted from the sensor data with structural information corresponding to the walls, floors, and ceilings of the BIM model. It includes a BIM-based map correction unit that generates high-precision map data corresponding to the actual environment by utilizing the above BIM model as a static standard to compare and correct SLAM drift errors accumulated in the initial environment map, and by excluding or separately managing at least one of the dynamic objects and temporary environmental changes detected from real-time sensor data from the map correction target; and an AI safety inspection execution unit that detects an area of discrepancy between the actual environment data and the BIM model on the high-precision map data, and inputs at least one of the image and point cloud data of the area into an artificial intelligence model to identify and analyze cracks, deformations, obstacles, or safety hazards of the facility.
[0346] According to an embodiment of the present invention, a BIM (Building Information Modeling) linked simultaneous location estimation and mapping (BIM SLAM) system for indoor AI safety inspection of facilities is proposed, which is characterized by improving the accuracy and reliability of SLAM by utilizing a BIM model.
[0347] The BIM data input unit receives BIM model data containing geometric and attribute information of the building.
[0348] The sensor data acquisition unit includes one or more sensors (e.g., LiDAR sensor, RGB-D camera, IMU, etc.) for acquiring indoor environment data.
[0349] The data registration and position estimation unit registers real-time environmental data acquired through the sensor data acquisition unit with BIM model data input through the BIM data input unit to estimate the current position and attitude of the system and generate an environmental map. The data registration and position estimation unit improves the accuracy of position estimation by matching structural features, such as planar elements extracted from the sensor data, with structural information of the BIM model, such as walls, floors, and ceilings.
[0350] The BIM-based map correction unit compares and corrects errors in the initial environment map generated by the data matching and position estimation unit (especially cumulative SLAM drift errors) with BIM model data to generate high-precision map data that corresponds to the actual environment. The BIM-based map correction unit utilizes the BIM model as static information to distinguish dynamically occurring object or environmental changes from real-time sensor data, and compensates for the weaknesses of SLAM in dynamic environments by not reflecting them in map correction or managing them separately.
[0351] The AI safety inspection unit identifies and analyzes safety hazards of a building based on high-precision map data and acquired environmental data. The AI safety inspection unit detects areas of discrepancy between actual environmental data and the BIM model on the high-precision map data, and inputs image or point cloud data of the area into an artificial intelligence model to determine safety risk factors.
[0352] The BIM-SLAM-based indoor AI safety inspection unit further includes a result output unit, and the result output unit provides the analysis results of the AI safety inspection execution unit to the user.
[0353] Although embodiments of the present invention have been described in detail above, the scope of the present invention is not limited thereto, and various modifications and improvements by those skilled in the art using the basic concept of the present invention as defined in the following claims also fall within the scope of the present invention.
Claims
Claim 1 A safety inspection system for tunnels inside dam facilities and reservoir water intake facilities based on subscription software and generative AI, comprising: an information acquisition unit that acquires and transmits information regarding tunnels inside dam facilities and reservoir water intake facilities, including an optical camera, a thermal imaging camera, and 3D Lidar; a driving control unit that performs autonomous driving while avoiding obstacles; and an embedded board that stores a program for detecting defects in reservoir water intake facilities by performing AI-based analysis on the information acquired from the shooting robot and performs calculations for analyzing image data and sensing data, wherein the shooting robot includes: a 3D LiDAR sensor; a map generation unit that generates or loads at least one of a point cloud map and a BIM map of an indoor environment using sensing values acquired from the 3D LiDAR sensor; a position recognition unit that estimates a current position based on at least one of the point cloud map and the BIM map generated by the map generation unit; a path planning unit that generates a driving path using the position estimation result of the position recognition unit and destination information on the map, and the driving control unit that performs autonomous driving according to the driving path generated by the path planning unit. Claim 2 A subscription-based software and generative artificial intelligence-based safety inspection system for internal tunnels of dam facilities and reservoir intake facilities, wherein the driving control unit controls a variable 3-axis wheel to perform the autonomous driving, the variable 3-axis wheel is deformable according to the internal condition of the reservoir intake facility and is deformed into a wheel for a dry floor slab, a wheel for a flooded floor slab, or a wheel for a sediment-deposited floor slab, and the camera robot includes a sensor unit that detects at least one of ground condition and obstacle information in real time, and a deformation unit that varies the shape or stiffness of the wheel according to the signal of the sensor unit, and a wheel system for a moving body including the variable 3-axis wheel that maintains a circular stiffness structure when driving on flat ground and operates so that the outer periphery of the wheel is elastically deformed to conform to the shape of the obstacle when driving on obstacles or rough ground, wherein the deformation unit includes at least one of a wire spoke, a chain block, or an origami structure, and includes a variable wheel system that selectively changes the radius, width, or stiffness of the wheel through the control of tension or deployment angle. Claim 3 A subscription-based software and generative artificial intelligence-based safety inspection system for internal tunnels of dam facilities and reservoir water intake facilities, wherein, in claim 1, the shooting robot acquires driving information by including at least one of a 3D LiDAR sensor, an encoder and gyroscope sensor equipped with odometry technology, an accelerometer, a 3-axis laser distance measuring device, an infrared sensor, an ultrasonic sensor, an RF sensor, and a temperature sensor, and the driving control unit performs indoor positioning by recording the position of the robot in real time using the driving information. Claim 4 In paragraph 3, the embedded board includes an artificial intelligence-based 3D point cloud analysis module that analyzes the driving information, and the driving control unit receives control commands from the embedded board and performs obstacle avoidance, forward, backward, and temporary stop according to the obstacle detection result, a subscription-based software and generative artificial intelligence-based safety inspection system for internal tunnels of dam facilities and reservoir water intake facilities. Claim 5 A subscription-based software and generative artificial intelligence-based safety inspection system for internal tunnels and reservoir water intake facilities of a dam facility, wherein the embedded board analyzes information obtained from the imaging robot to detect defects classified as cracks, spalling, deformation, rebar exposure, water intake hole leakage, culvert leakage, culvert scouring, and sediment deposition. Claim 6 A subscription-based software and generative artificial intelligence-based safety inspection system for internal tunnels of dam facilities and reservoir water intake facilities according to claim 1, wherein the embedded board analyzes information obtained from the shooting robot to remove blurred images and low-brightness images, restores an image captured at an angle to an orthophoto using angle and position data, assigns position information and an identifier to the orthophoto, and removes noise included in the orthophoto. Claim 7 In claim 1, the embedded board utilizes a generative artificial intelligence model capable of cross-modal learning that artificially generates and outputs major defects based on image, thermal image, acoustic sensing, and LiDAR data, and automatically outputs artificially generated defects after the AI model understands the characteristics and generation conditions of the major defects included in the user's text input, thereby comprising a subscription-based software and a generative artificial intelligence-based safety inspection system for internal tunnels of dam facilities and reservoir water intake facilities. Claim 8 In claim 1, the embedded board provides a function in which a generative artificial intelligence model automatically generates a 3D mesh using a plurality of superimposed 2D images as input, utilizing a 3D mesh generation model for building a digital twin model, and the subscription software and generative artificial intelligence-based safety inspection system for internal tunnels of dam facilities and reservoir water intake facilities. Claim 9 In claim 1, the embedded board utilizes a generative AI model capable of cross-modal learning that outputs a 3D mesh for building a digital twin model to understand the characteristics of the 3D mesh included in the image and text input provided from the user terminal and the conditions for generating the 3D mesh, and provides the function of generating a low-resolution, coarse mesh in the first step and automatically outputting a high-quality 3D mesh with detailed textures in the second step, thereby providing a subscription-based software and generative AI-based safety inspection system for internal tunnels of dam facilities and reservoir water intake facilities. Claim 10 delete Claim 11 A shooting robot comprising an information acquisition unit that acquires and transmits information regarding tunnels inside dam facilities and reservoir intake facilities, including an optical camera, a thermal imaging camera, and 3D Lidar, and a driving control unit that performs autonomous driving while avoiding obstacles; and an embedded board that stores a program for detecting defects in the reservoir intake facilities by performing AI-based analysis on the information acquired from the shooting robot, and performs calculations for the analysis of image data and sensing data, wherein the embedded board comprises: an image input unit that collects image data; a feature extraction module that extracts feature information from the image data; a deep learning-based detection model that detects crack and defect candidates based on the feature information; an inference engine that receives the detection result of the deep learning-based detection model as input and determines at least one of the type, location, size, and severity of the crack; a decision module that dynamically adjusts at least one of detection parameters and detection strategies according to the judgment result of the inference engine; and an AI agent that autonomously performs defect detection using the feature extraction module, the deep learning-based detection model, the inference engine, and the decision module. A subscription-based software and generative artificial intelligence-based safety inspection system for internal tunnels and reservoir water intake facilities of dam facilities, comprising a result output unit that stores or visualizes the output results of the AI Agent. Claim 12 A shooting robot comprising an information acquisition unit that acquires and transmits information regarding tunnels and reservoir intake facilities inside dam facilities, including an optical camera, a thermal imaging camera, and 3D Lidar, and a driving control unit that performs autonomous driving while avoiding obstacles; and an embedded board that stores a program for detecting defects in reservoir intake facilities by performing AI-based analysis on the information acquired from the shooting robot and performs calculations for the analysis of image data and sensing data, and further comprises a BIM-SLAM-based indoor AI safety inspection unit that performs BIM-linked simultaneous position estimation and mapping (BIM-SLAM) to perform AI-based safety inspection inside tunnels and reservoir intake facilities inside dam facilities, and improves the position estimation accuracy of SLAM and map reliability by utilizing a BIM model containing geometric structure and attribute information of the facility as static reference information, wherein the BIM-SLAM-based indoor AI safety inspection unit includes a BIM data input unit that receives BIM model data containing geometric information and attribute information of the facility; A sensor data acquisition unit that acquires sensor data including at least one of LiDAR data, image data, and inertial data from a sensor for acquiring indoor environment data; a data matching and position estimation unit that aligns structural features including planar elements extracted from the sensor data with structural information corresponding to the walls, floors, and ceilings of a BIM model to estimate the current position and attitude of the system and generate an initial environment map; and a BIM-based map correction unit that utilizes the BIM model as a static standard to compare and correct SLAM drift errors accumulated in the initial environment map, and generates high-precision map data corresponding to the actual environment by excluding at least one of dynamic objects and temporary environmental changes detected from real-time sensor data from the map correction target or managing them separately.A subscription-based software and generative AI-based safety inspection system for internal tunnels of dam facilities and reservoir water intake facilities, comprising an AI safety inspection unit that detects discrepancies between actual environmental data and a BIM model on the high-precision map data, and inputs at least one of image and point cloud data of the area into an AI model to identify and analyze cracks, deformations, obstacles, or safety hazards of the facility.
Citation Information
Patent Citations
The variable wheel
KR1020180089938A
Apparatus for SaaS platform for automatic creation of descriptive AI-based dam facility defect detection, quantification, captioning, drawing, and condition evaluation
KR1020230065188A
Autonomous exploration robot and autonomous exploration method for underground facility exploration
KR1020230111501A
Wheel unit having multi structure for changing stiffness
KR1020240000882A
Wheel module having a subsidiary link
KR1020240009209A
