Intelligent exploration robot and intelligent closed pipeline exploration system including same

Abstract

An intelligent closed pipeline exploration system according to one embodiment of the present invention comprises: an intelligent exploration robot; and an input adjustment device provided to adjust the exploration distance of the intelligent exploration robot when the intelligent exploration robot explores a closed pipeline. The intelligent exploration robot includes: a camera module provided to acquire image data; an acoustic sensor provided to acquire acoustic data generated in the closed pipeline; an artificial intelligence module provided to perform artificial intelligence learning; and a control module electrically connected to the camera module, the acoustic sensor, and the artificial intelligence module. The control module can acquire image data inside the closed pipeline by using the camera module, acquire acoustic data generated in the closed pipeline by using the acoustic sensor, generate refined data by performing data refinement on the acquired image data and acoustic data through pre-processing, generate labeling data through data processing in which labeling is performed on the refined data on the basis of a class defined in advance for a problem area of the closed pipeline, generate training data for the problem area by performing artificial intelligence learning on the labeling data by using the artificial intelligence module, and perform identification and prediction regarding the problem area for newly acquired new image data and new acoustic data on the basis of the training data.

Description

Intelligent exploration robot and intelligent closed pipeline exploration system including the same

[0001] The present disclosure relates to an intelligent exploration robot capable of exploring closed pipelines, such as water pipes, to identify problem areas, and an intelligent closed pipeline exploration system including the same.

[0002] Water is an essential necessity for daily life, and domestic water is supplied to homes, public buildings, and private structures through closed pipelines such as water supply pipes. Since water is supplied through closed pipelines, the design and maintenance of the supply lines are critical. Furthermore, maintenance and repair are extremely important because once installed, the lines are used for a long time, and replacement requires significant time and cost.

[0003] However, due to the nature of closed pipelines through which domestic water flows daily, it is difficult to prevent internal corrosion or the accumulation of foreign matter. Severe corrosion or damage can lead to water waste due to leakage, or in extreme cases, result in accidents such as sinkholes caused by pipe ruptures. Regarding water waste, the monetary value of water leaked from the pipelines over the past few years amounts to trillions of won.

[0004] Accordingly, to manage closed pipelines through which water flows, there is a need for technology that utilizes robotic deployment technology to collect data using embedded cameras and audio, refines and processes the collected data for artificial intelligence learning, and thereby predicts signs of abnormalities in water pipes.

[0005] The problem that the present disclosure aims to solve is to provide an intelligent exploration robot capable of accurately and effectively analyzing and predicting various abnormal signs in a closed pipeline by utilizing image and acoustic data collected through the exploration of a closed pipeline, such as a water supply pipe, and an intelligent closed pipeline exploration system including the same.

[0006] An intelligent closed pipeline exploration system according to one embodiment of the present disclosure comprises an intelligent exploration robot and an input control device configured to adjust the exploration distance of the intelligent exploration robot when the intelligent exploration robot explores a closed pipeline. The intelligent exploration robot comprises a camera module configured to acquire image data, an acoustic sensor configured to acquire acoustic data generated in the closed pipeline, an artificial intelligence module configured to perform artificial intelligence learning, and a control module electrically connected to the camera module, the acoustic sensor, and the artificial intelligence module. The control module acquires image data inside the closed pipeline using the camera module and acquires acoustic data generated in the closed pipeline using the acoustic sensor. It generates refined data by performing data refinement through preprocessing on the acquired image data and acoustic data, generates labeling data through data processing by performing labeling on the refined data based on a predefined class for the problem area of ​​the closed pipeline, and performs artificial intelligence learning on the labeling data using the artificial intelligence module to [indicate] the problem area Training data is generated, and based on the training data, identification and prediction regarding the problem domain can be performed for newly acquired new image data and new sound data.

[0007] According to the embodiments of the present disclosure, problem areas of closed pipelines can be accurately and effectively identified without interruption.

[0008] According to the embodiments of the present disclosure, anomaly inference can be effectively performed based on acquired data.

[0009] According to embodiments of the present disclosure, a basis for predicting the replacement cycle, cleaning cycle, and leakage of closed pipelines can be provided.

[0010] The effects of the present disclosure are not limited to those mentioned above, and other unmentioned effects will be clearly understood by those skilled in the art from the description in the claims.

[0011] FIG. 1 is a drawing showing an intelligent closed pipeline exploration system according to one embodiment of the present disclosure.

[0012] FIG. 2 is a diagram showing the configuration of an intelligent exploration robot according to one embodiment of the present disclosure.

[0013] FIG. 3 is a drawing showing a lidar sensor according to one embodiment of the present disclosure.

[0014] FIG. 4 is a diagram showing the generation flow of a three-dimensional closed pipeline model according to one embodiment of the present disclosure.

[0015] FIG. 5 is a drawing specifically illustrating the generation flow of a three-dimensional closed pipeline model according to one embodiment of the present disclosure.

[0016] FIG. 6 is a drawing showing the identification of a problem area in a three-dimensional closed pipeline model according to one embodiment of the present disclosure.

[0017] FIG. 7 is a diagram illustrating the identification of a problem area based on an artificial intelligence module according to one embodiment of the present disclosure.

[0018] FIG. 8 is a drawing showing an example of a problem area according to one embodiment of the present disclosure.

[0019] FIG. 9 is a drawing showing a digital twin system according to one embodiment of the present disclosure.

[0020] FIG. 10 is a perspective view showing the structure of an intelligent exploration robot according to one embodiment of the present disclosure.

[0021] FIG. 11 is a drawing showing the structure of an intelligent exploration robot according to one embodiment of the present disclosure, viewed from the front.

[0022] FIG. 12 is a drawing showing the bending of an intelligent exploration robot according to one embodiment of the present disclosure.

[0023] Hereinafter, specific details for implementing the present disclosure will be described in detail with reference to the attached drawings. However, in the following description, specific descriptions regarding widely known functions or configurations will be omitted if there is a risk that the gist of the present disclosure may be unnecessarily obscured.

[0024] In the attached drawings, identical or corresponding components are assigned the same reference numerals. Additionally, in the description of the following embodiments, the description of identical or corresponding components may be omitted. However, even if a description of a component is omitted, it is not intended that such component is not included in any embodiment.

[0025] The advantages and features of the disclosed embodiments and the methods for achieving them will become clear by referring to the embodiments described below in conjunction with the accompanying drawings. However, the present disclosure is not limited to the embodiments disclosed below but may be implemented in various different forms, and the embodiments provided are merely to make the present disclosure complete and to fully inform those skilled in the art of the scope of the invention.

[0026] The terms used in this specification will be briefly explained, and the disclosed embodiments will be described in detail. The terms used in this specification have been selected to be as generally used as possible, taking into account their functions in this disclosure; however, these terms may vary depending on the intent of those skilled in the art, case law, the emergence of new technologies, etc. Additionally, in specific cases, terms may be selected at the applicant's discretion, and in such cases, their meanings will be described in detail in the relevant description of the invention. Therefore, the terms used in this disclosure should be defined not merely by their names, but based on their meanings and the content throughout this disclosure.

[0027] In this specification, singular expressions include plural expressions unless the context clearly specifies them as singular. Additionally, plural expressions include singular expressions unless the context clearly specifies them as plural. Throughout the specification, when a part is described as including a certain component, this means that, unless specifically stated otherwise, it does not exclude other components but may include additional components.

[0028] Additionally, the terms 'module' or 'part' as used in the specification refer to software or hardware components, and the 'module' or 'part' performs certain roles. However, the meaning of 'module' or 'part' is not limited to software or hardware. The 'module' or 'part' may be configured to reside in an addressable storage medium or configured to run on one or more processors. Thus, as an example, the 'module' or 'part' may include components such as software components, object-oriented software components, class components, and task components, and at least one of processes, functions, attributes, procedures, subroutines, segments of program code, drivers, firmware, microcode, circuits, data, databases, data structures, tables, arrays, or variables. The components and the functions provided within the 'module' or 'part' may be combined into a smaller number of components and 'modules' or 'parts', or further separated into additional components and 'modules' or 'parts'.

[0029] Throughout the entire specification, when a component is described as being located "on" another component, unless specifically stated otherwise, this includes not only cases where a component is in contact with another component, but also cases where another component exists between the two components.

[0030] Throughout this specification, when a part is described as "comprising" a certain component, this means that, unless specifically stated otherwise, it does not exclude other components but may include additional components.

[0031] Throughout this specification, terms such as "approximately" and "substantially" are used to encompass tolerances when they exist.

[0032] Throughout this specification, the term “combination(s) of these” included in the Markush-type expression means one or more mixtures or combinations selected from the group consisting of the components described in the Markush-type expression, and means including one or more selected from the group consisting of said components.

[0033] Throughout this specification, the description "A and / or B" means "A, or B, or A and B".

[0034] Throughout this specification, when a part is described as being "connected" to another part, this includes not only cases where they are "directly connected," but also cases where they are "electrically connected" with other elements interposed between them.

[0035] Embodiments of the present invention are described below with reference to the attached drawings so that those skilled in the art can easily implement them. However, the present invention may be embodied in various different forms and is not limited to the embodiments described herein. Furthermore, in order to clearly explain the present invention in the drawings, parts unrelated to the explanation have been omitted, and similar parts throughout the specification are denoted by similar reference numerals.

[0036] FIG. 1 shows an intelligent closed pipeline exploration system according to one embodiment of the present disclosure.

[0037] The intelligent closed pipeline exploration system (1) (hereinafter referred to as the 'exploration system') is a system that can accurately and effectively explore a closed pipeline (P) using an intelligent exploration robot (10) to identify problem areas of the closed pipeline (P).

[0038] The exploration system (1) may include an intelligent exploration robot (10) (hereinafter referred to as the 'robot'). The robot (10) can move within the closed pipe (P) to identify the interior (or internal structure) of the closed pipe (P) and identify problem areas within the closed pipe (P). Additionally, the robot (10) can perform three-dimensional closed pipe modeling of the closed pipe (P) using sensors and can identify problem areas more accurately and quickly using artificial intelligence.

[0039] The robot (10) can acquire image data inside the closed pipe (P) using a camera module (e.g., 120). The robot (10) can acquire internal acoustic data of the closed pipe (P) from the acquired image data.

[0040] The robot (10) can explore by flowing through the fluid (e.g., water) inside the closed pipe (P). For example, the robot (10) can effectively move within the fluid through the thrust module (150) described later to adjust the exploration distance, and can also be connected to the coaxial cable (210) described later to adjust the exploration distance.

[0041] The exploration system (1) may include an insertion control device (200). The insertion control device (200) can control the exploration distance of the robot (10) when the robot (10) explores a closed pipe (P). Specifically, the insertion control device (200) may include a coaxial cable (210). The coaxial cable (210) may be physically / electrically connected to the robot (10). The insertion control device (200) can control the exploration distance of the robot (10) using the coaxial cable (210). For example, the insertion control device (200) may have the coaxial cable (210) in a wound state, and the insertion control device (200) may control the exploration distance of the robot (10) by unwinding the coaxial cable (210). However, it is not limited thereto.

[0042] FIG. 2 shows the configuration of an intelligent exploration robot according to one embodiment of the present disclosure, and FIG. 3 shows a lidar sensor according to one embodiment of the present disclosure. The description of the above-described embodiment may be applied to the present embodiment in the same or similar manner.

[0043] The robot (10) may include a LiDAR sensor (100). The LiDAR sensor (100) may be a sensor configured to measure the distance to an object using light (e.g., a laser). Specifically, the LiDAR sensor (100) may emit light at an object and calculate the distance by measuring the time it takes for the light to return. That is, the LiDAR sensor (100) may emit light into the interior of a closed conduit (P) and measure the distance through the time it takes for the light to be reflected back from the internal structure of the closed conduit (P). That is, the robot (10) may acquire distance (or depth) data using the LiDAR sensor (100). Additionally, the robot (10) may acquire three-dimensional point cloud data for the closed conduit (P) using the LiDAR sensor (100), and the point cloud data may be a concept that includes three-dimensional coordinate data in addition to the distance (or depth) data.

[0044] A lidar sensor (100) can be positioned on the front of the robot (10). As the robot (10) moves along the direction of movement, it can identify the internal structure and problem areas of the closed pipe (P) through the lidar sensor (100) positioned on the front. Referring to FIG. 3, the lidar sensor (100) can identify a range of at least 40 degrees when viewed from the side of the robot (10). Additionally, the lidar sensor (100) can identify a range of at least 210 degrees when viewed from above the robot (10).

[0045] The robot (10) may include an inertial sensor (110). The inertial sensor (110) may be configured to sense the position and orientation of the robot (10). Specifically, the inertial sensor (110) may sense the acceleration and angular velocity of the robot (10) to sense the position and orientation of the robot (10) within the closed pipe (P). The robot (10) may obtain position estimation data regarding the position and orientation of the robot (10) using the inertial sensor (110).

[0046] The robot (10) may include a camera module (120). The camera module (120) may be configured to acquire image data. Specifically, the robot (10) may acquire image data within a closed pipe (P) using the camera module (120).

[0047] The robot (10) may include an acoustic sensor (130). The acoustic sensor (130) may be configured to acquire acoustic data generated in the closed pipe (P).

[0048] The robot (10) may include an artificial intelligence module (140). The artificial intelligence module (140) may be configured to perform artificial intelligence learning based on at least one of point cloud data, position estimation data, image data, and acoustic data obtained through a LiDAR sensor (100), an inertial sensor (110), a camera module (120), and an acoustic sensor (130).

[0049] The robot (10) may include a thrust module (150). The thrust module (150) may be configured to provide thrust to enable effective movement when the robot (10) moves within the internal fluid of a closed pipe (P). The thrust module (150) will be described later.

[0050] The robot (10) may include a control module (160). The control module (160) is electrically connected to the lidar sensor (100), inertial sensor (110), camera module (120), acoustic sensor (130), artificial intelligence module (140), and thrust module (150) to perform control over the lidar sensor (100), inertial sensor (110), camera module (120), acoustic sensor (130), artificial intelligence module (140), and thrust module (150).

[0051] FIG. 4 illustrates the generation flow of a three-dimensional closed pipeline model according to one embodiment of the present disclosure, FIG. 5 illustrates the generation flow of a three-dimensional closed pipeline model according to one embodiment of the present disclosure, FIG. 6 illustrates the identification of a problem region in a three-dimensional closed pipeline model according to one embodiment of the present disclosure, FIG. 7 illustrates the identification of a problem region based on an artificial intelligence module according to one embodiment of the present disclosure, and FIG. 8 illustrates an example of a problem region according to one embodiment of the present disclosure. The description of the above-described embodiment may be applied to the present embodiment in the same or similar manner.

[0052] According to S100, the robot (10) can acquire point cloud data using the control module (160). Specifically, the control module (160) can acquire three-dimensional point cloud data regarding the structure of the closed pipe (P) using the LiDAR sensor (100). The point cloud data may be a set of three-dimensional coordinates regarding the internal structure of the closed pipe (P). That is, the control module (160) can acquire the internal structure of the closed pipe (P) in the form of a three-dimensional point cloud using the LiDAR sensor (100). The control module (160) can acquire the point cloud data in real time using the LiDAR sensor (100).

[0053] According to S200, the robot (10) can obtain position estimation data using the control module (160). Specifically, the control module (160) can obtain position estimation data regarding the position and attitude of the robot (10) using the inertial sensor (110). That is, the control module (160) can obtain position estimation data, which is data regarding real-time position and direction changes based on the movement path and attitude change of the robot (10), using the inertial sensor (110). The control module (160) can obtain position estimation data in real time using the inertial sensor (110).

[0054] In S100 to S200, the robot (10) can acquire point cloud data and position estimation data in real time while moving within the closed pipe (P).

[0055] According to S300, the robot (10) can generate a three-dimensional closed pipeline model using the control module (160). Specifically, the control module (160) can generate a three-dimensional closed pipeline model based on point cloud data and location estimation data. Additionally, the control module (160) can identify problem areas in the generated three-dimensional closed pipeline model.

[0056] Prior to the operation according to S300, the robot (10) may perform preprocessing on at least one of the acquired point cloud data and position estimation data. Specifically, the control module (160) may perform noise removal and data optimization on at least one of the point cloud data and position estimation data. For example, the control module (160) may generate more accurate data by removing unnecessary noise during noise removal, and may compress and organize redundant or unnecessary data during data optimization.

[0057] With reference to FIGS. 5 and 6, the creation of a three-dimensional closed pipeline model according to S300 can be described in more detail. The control module (160) can create a plurality of local maps, form a plurality of sub-maps, and combine the plurality of sub-maps to create a three-dimensional closed pipeline model.

[0058] The control module (160) can generate multiple local maps for each of the local sections within the entire section that the robot (10) traveled inside the closed pipe (P), based on point cloud data and position estimation data.

[0059] The control module (160) may use image data when generating multiple local maps. For example, image data for a closed pipe (P) can be acquired using a camera module (120), and multiple local maps can be generated by combining the acquired image data and point cloud data. That is, the control module (160) can generate multiple local maps by applying a visual-depth data association technique and combining point cloud data acquired through a LiDAR sensor (100) and image data acquired through a camera module (120), thereby enabling more precise modeling of the interior of the closed pipe (P). Multiple local maps can be generated in real time while the robot (10) is moving.

[0060] The control module (160) can form multiple sub-maps by combining multiple local maps generated. Specifically, when forming multiple sub-maps, the control module (160) can combine them using a sliding window graph optimization technique to improve the accuracy of combining multiple local maps. That is, the control module (160) can create a consistent 3D closed pipe (P) model by accurately combining multiple local maps through the sliding window graph optimization technique to adjust position errors that occur while the robot (10) moves.

[0061] The control module (160) can generate a three-dimensional closed pipeline model by combining a plurality of sub-maps. Specifically, the control module (160) can generate a three-dimensional closed pipeline model that represents the internal structure and state of the closed pipeline (P) by combining the generated plurality of sub-maps.

[0062] The control module (160) can visualize a three-dimensional closed pipeline model. Specifically, when creating a three-dimensional closed pipeline model, the control module (160) can create a visually displayed model using colors so that it is easy to distinguish between the problem area and the normal area of ​​the closed pipeline (P).

[0063] Referring to FIG. 7, the control module (160) can perform learning on the problem area using the artificial intelligence module (140). Specifically, the control module (160) can generate learning data for the problem area by performing artificial intelligence learning on at least one of point cloud data, location estimation data, image data, and acoustic data using the artificial intelligence module (140). The control module (160) can identify the problem area in the 3D closed pipe model based on the generated learning data.

[0064] Artificial intelligence learning using the artificial intelligence module (140) is performed using various neural network structures, and a neural network suitable for each data type can be applied.

[0065] For example, the artificial intelligence module (140) can perform learning through a method using a Deep Neural Network (DNN). A Deep Neural Network is a basic neural network structure having multiple hidden layers, and each layer can learn increasingly complex features by processing data input from the previous layer. Deep Neural Networks can be useful for processing complex three-dimensional data, such as point cloud data. This neural network structure is composed of a Multilayer Perceptron and can play a major role in analyzing the internal structure of a closed pipe (P) by learning the patterns and correlations of the data input at each layer. Through this, it is possible to learn and predict damaged areas or abnormal signs inside the closed pipe (P).

[0066] The artificial intelligence module (140) can perform learning through a method using a Convolutional Neural Network (CNN). A Convolutional Neural Network may be a neural network that exhibits excellent performance in extracting key features from image data or 3D point cloud data. A Convolutional Neural Network is primarily used to process image data and can process input data through multiple convolution layers and pooling layers. For example, a Convolutional Neural Network can automatically learn and detect problem areas such as damaged parts, cracks, and eroded zones in images inside a closed pipe (P). In the initial stage, it extracts low-level features such as edges or contours, and in subsequent stages, it recognizes more complex structural damage or defects to accurately identify problem areas of the closed pipe (P).

[0067] The artificial intelligence module (140) can perform learning through a method using a Recurrent Neural Network (RNN). A Recurrent Neural Network is a neural network structure that is very advantageous for processing temporally sequential data. It is used to process data that changes over time, such as location estimation data or acoustic data, and can analyze current data while remembering data from previous times. A Recurrent Neural Network can learn continuous or repetitive patterns occurring inside a closed pipe (P) to detect abnormal acoustic signals, such as the sound of leakage or vibration occurring in cracks. In addition, a Long Short-Term Memory (LSTM) network is a type of Recurrent Neural Network that can effectively learn longer sequences by compensating for the problem that Recurrent Neural Networks struggle with maintaining long-term dependencies. Through this, problems that may occur in data observed for a long time in a closed pipe (P) can be effectively predicted.

[0068] The artificial intelligence module (140) can perform learning through a method using deformable convolutional neural networks (DCNN). A deformable neural network is an extension of a convolutional neural network and may be a neural network designed to effectively learn even when the shape of the input data changes. The internal structure of a closed pipe (P) may not have a fixed shape but may have an irregular shape, and a deformable neural network may be used to recognize such changes. This neural network can extract important features even in the deformed areas of the input data and can accurately detect the irregular shape of the damaged pipe or the changed pattern of the crack.

[0069] The artificial intelligence module (140) can perform learning through a method using a Graph Neural Network (GNN). A graph neural network can be a particularly useful neural network for unstructured data with complex structures. It can process graph-shaped data such as point cloud data or pipe network data, and can analyze structural information of the data by learning the relationships between each node and edge. By modeling the three-dimensional structure inside the closed pipe (P) in a graph form, problem areas occurring within the network can be detected more efficiently. The graph neural network exhibits excellent performance in learning complex correlations between data and can model the overall state of the closed pipe (P) more accurately.

[0070] The artificial intelligence module (140) can perform learning through a method using Generative Adversarial Networks (GAN). A Generative Adversarial Network is a structure composed of two neural networks (a generator and a discriminator) and can learn data through the generation and discrimination of data. The Generative Adversarial Network can learn normal and abnormal patterns from data inside a closed pipe (P), and can be very useful for predicting when abnormal patterns appear in new data. For example, after learning the structure of a normal closed pipe (P) through the Generative Adversarial Network, it can detect anomalies in new data and quickly identify problem areas.

[0071] The control module (160) can learn point cloud data, location estimation data, image data, and acoustic data using various neural network structures as described above through the artificial intelligence module (140), and can automatically identify problem areas inside the closed pipeline (P). By using various neural network structures, maintenance work on the closed pipeline (P) can be performed efficiently through an optimal learning method suitable for each data type, and problem areas can be detected early to prevent major damage. The application of such neural network models can maximize the accuracy and efficiency of closed pipeline (P) management.

[0072] Referring to FIGS. 7 and 8, a method for predicting problem areas or abnormal signs in a closed pipeline (P) by performing artificial intelligence learning through the process of collecting, refining, and processing data of the closed pipeline (P) using a camera module (120), an acoustic sensor (130), and an artificial intelligence module (140) equipped in a robot (10) is described.

[0073] The robot (10) can collect image data and acoustic data simultaneously by using a camera module (120) and an acoustic sensor (130) to collect data within the closed pipe (P). The robot (10) can capture image data at 20cm intervals through the camera module (120) to record the overall structure and condition of the closed pipe (P) in detail, and can obtain data from various angles by taking multiple shots including the top, bottom, left, right, and center. In particular, if the diameter of the closed pipe (P) is large, the robot (10) can record the condition of the pipe more precisely through additional shooting, thereby clearly identifying the physical characteristics of the closed pipe (P).

[0074] The robot (10) can collect acoustic data through the acoustic sensor (130). The acoustic data may include internal acoustic data and external acoustic data. Specifically, for example, the robot (10) can acquire internal acoustic data of a closed pipe (P) using the acoustic sensor (130) and synchronize it with image data of a location corresponding to the internal acoustic data. Through this, the robot (10) can combine the image data and the internal acoustic data to perform more comprehensive data analysis. In addition, the robot (10) can also collect external acoustic data, such as external leakage sounds, through the acoustic sensor (130). That is, the robot (10) can secure valid external acoustic data while minimizing external noise.

[0075] The robot (10) can process collected data (e.g., acquired image data and sound data) into high-quality refined data to increase the accuracy of artificial intelligence learning. In the image data refinement, to reduce noise generated in the dark environment inside the closed pipe (P) and underwater environment, the robot (10) can reduce unnecessary noise by applying frequency domain filtering such as Gaussian filtering, median filtering, and bidirectional filtering. In addition, the robot (10) can clearly show image boundaries and details by emphasizing high-frequency components through unsharp mask filtering to express blurry images clearly, and can also apply edge enhancement technology to reduce distortion caused by water flow. In addition, the robot (10) can generate source data for precise analysis by distinguishing the background and foreground through masking techniques and emphasizing specific parts of the closed pipe (P).

[0076] The robot (10) can collect acoustic data by dividing it into a total of 5 seconds, with 2.5 seconds before and after the same location, based on the location of image data captured at intervals of 20 cm within the closed conduit (P). The acoustic data collected in this way can have its clarity of the main signal enhanced by applying a high-pass filter (FIR) and pre-emphasis filtering to reduce unnecessary low-frequency components and emphasize high-frequency components. Additionally, the robot (10) can improve the quality of the acoustic data by maintaining a constant volume through an RMS-based volume normalization technique and reducing the volume difference of the acoustic data through dynamic range compression. The robot (10) can obtain effective acoustic data suitable for detecting abnormal signs in the closed conduit (P) by using a spectrum balancing technique to strengthen specific frequency bands, thereby minimizing noise and emphasizing the main signal.

[0077] The refined data can be converted into data (or labeled data) suitable for artificial intelligence learning using an artificial intelligence module (140) by performing a labeling operation by a robot (10). Specifically, in image data labeling, the condition within the closed pipe (P) is reflected and can be classified into five classes: 'normal area (NORMAL PIPIE)', 'joint (WELD)', 'corrosion (CORROSION)', 'foreign matter (SCALE)', and 'surface abnormality (SURFACE DAMAGE)', as shown in FIG. 8.

[0078] The robot (10) can perform class assignment (or labeling) on ​​image data among refined data. For example, the robot (10) can use an object recognition-based polygon method to designate specific states within an image in the form of polygons and assign a class corresponding to each state. For example, if a joint is found within a closed pipeline (P), the robot (10) can label the part as a 'joint' class to record accurate state information. The above-described labeling can be applied to normal areas, joints, corrosion, foreign substances, and surface abnormalities.

[0079] The robot (10) can perform labeling on acoustic data among refined data. For example, in acoustic data labeling, the robot (10) can classify into two classes, 'Normal (N)' and 'Abnormal (A)', based on a specific pattern in the frequency domain. Specifically, for example, the robot (10) can determine whether a specific pattern appears in the frequency band of the acoustic data and classify it as Abnormal (A) if a bright line continuously appears in the range of 256 to 512 Hz. The robot (10) can classify internal acoustic data and external acoustic data included in the acoustic data, respectively.

[0080] The robot (10) can perform artificial intelligence learning through an artificial intelligence module (140) that includes a semantic segmentation model and a multimodal data classification model using labeling data. Through artificial intelligence learning, the robot (10) can learn abnormal signs (or problem areas) of the closed pipe (P), infer abnormalities in real-time regarding newly acquired image data and acoustic data, and implement an intelligent pipe inspection system that can quickly identify and warn of the condition of the closed pipe (P).

[0081] The robot (10) can perform artificial intelligence learning to predict abnormal signs (or problem areas) inside the closed pipe (P) using an artificial intelligence module (140). The artificial intelligence module (140) implements the artificial intelligence learning process based on a semantic segmentation model and a multimodal data classification model that can comprehensively analyze the state of the closed pipe (P), and through the artificial intelligence learning process, the robot (10) can learn and infer abnormal signs (or problem areas) from image data and acoustic data of the closed pipe (P).

[0082] During the training process of the semantic segmentation model, the robot (10) can analyze image data inside the closed pipeline (P) at the pixel level using a model such as MaskDINO. The MaskDINO model can accurately identify the location and shape of defective objects within the image by utilizing a Transformer-based attention mechanism. Through the semantic segmentation model, the robot (10) can detect defective areas within the image and classify them into five states according to the corresponding class: 'normal', 'joint', 'corrosion', 'sediment / foreign matter', and 'surface abnormality'. The trained data can be used to recognize abnormal signs in the image of the closed pipeline (P) and accurately predict each state.

[0083] The robot (10) can learn image data and acoustic data together through a multimodal data classification model using an artificial intelligence module (140). In this process, the robot (10) can use a Vision Transformer (ViT) model to combine and analyze image data and acoustic data inside a closed pipe (P), and learn the visual and acoustic characteristics of the image data and acoustic data in an integrated manner. The ViT model can learn visual patterns by dividing the input image into patches of a fixed size and encoding each patch into a vector. Through this, the robot (10) analyzes the frequency spectrogram of the acoustic data along with the image data, recognizes abnormal frequency patterns, and learns them comprehensively, thereby more accurately predicting abnormal conditions inside the closed pipe (P).

[0084] After completing artificial intelligence learning, the robot (10) can perform inference in real time on newly collected new image data and new sound data. The robot (10) can detect areas (or problem areas) where abnormal signs appear within the image through a semantic segmentation model and can identify the location and type of defects by performing a comprehensive analysis through a multimodal classification model. By combining and analyzing newly acquired new image data and new sound data, the similarity with previously learned defect patterns is calculated, allowing for the rapid determination (or identification) and prediction of the condition of the closed pipe (P) (e.g., abnormal signs, problem areas).

[0085] The robot (10) can provide immediate feedback on the condition of the closed pipe (P) and can predict the occurrence of abnormal signs early. For example, the robot (10) can generate identification result data that identifies abnormal signs or problem areas inside the closed pipe (P). In addition, if it is predicted that abnormal signs or problem areas will occur inside the closed pipe (P), the robot (10) can generate prediction result data regarding that prediction.

[0086] The robot (10) can merge the aforementioned identification result data and prediction result data into the aforementioned three-dimensional closed pipeline model. Accordingly, the robot (10) can display and implement abnormal signs or problem areas in the three-dimensional closed pipeline model, and can also display and implement areas where abnormal signs or problem areas are expected to occur. That is, the robot (10) can create a three-dimensional closed pipeline model to which the identification result data and / or prediction result data generated based on artificial intelligence learning is applied.

[0087] The robot (10) can generate and transmit to the outside the generated identification result data and prediction result data, and the three-dimensional closed pipeline model data to which the identification result data and prediction result data are applied in three dimensions.

[0088] The movements of the robot (10) described above can be controlled by the control module (160). According to the robot and intelligent closed pipeline exploration system described above, problem areas of closed pipelines can be accurately and effectively identified without interruption, abnormal signs can be inferred effectively based on acquired data, and a basis for predicting the replacement cycle, cleaning cycle, and leakage of closed pipelines can be provided.

[0089] FIG. 9 illustrates a digital twin system according to one embodiment of the present disclosure. The description of the above-described embodiment may be applied to the present embodiment in the same or similar manner.

[0090] The control module (160) can share the three-dimensional closed pipeline model with an external management system (310). Specifically, the control module (160) can share the three-dimensional closed pipeline model, in which the problem area is visualized and displayed, with an external management system. That is, by the control module (160) sharing the three-dimensional closed pipeline model with the external management system (310), a digital twin system (300) can be implemented.

[0091] A digital twin system (300) is a system that creates a digital replica of a physical asset to monitor and analyze the state and operation of the actual asset in real time in a virtual environment. The digital twin system (300) creates an accurate 3D closed pipeline model in a virtual environment based on data collected from the physical system, and the manager can remotely manage and optimize the actual asset through this.

[0092] In the present invention, the control module (160) generates a three-dimensional closed pipeline model based on the internal state of the closed pipeline (P), and the problem area can be displayed in a visualized form in the three-dimensional closed pipeline model. When the control module (160) shares the three-dimensional closed pipeline model with an external management system (310), the external management system receives the three-dimensional closed pipeline model and can monitor the state of the closed pipeline (P) in real time in a digital twin environment. For example, through the three-dimensional closed pipeline model in which the problem area is clearly displayed visually, a manager or maintenance team can accurately determine the internal state of the closed pipeline (P) in real time without directly accessing the physical environment.

[0093] In addition, the digital twin system (300) can enable predictive maintenance based on real-time data. By periodically updating data collected from the closed pipeline (P) by the control module (160) and sharing it with the management system (310), the manager can predict the possibility of damage to the closed pipeline (P) in advance. Through this, problems can be detected in advance, and measures can be taken to prevent large-scale damage or accidents.

[0094] The robot (10) can share the generated identification result data and prediction result data, and the 3D closed pipeline model data to which the identification result data and prediction result data are applied in 3D, with the management system (310), and the same or similar content as described above may be applied to this.

[0095] As described above, a digital twin system (300) can be implemented by the control module (160) sharing the three-dimensional closed pipeline model, identification result data, and prediction result data with an external management system (310). This system can maximize the maintenance efficiency of the closed pipeline (P) and enable a rapid response in the event of a problem.

[0096] FIG. 10 shows the structure of an intelligent exploration robot according to one embodiment of the present disclosure, FIG. 11 shows the structure of an intelligent exploration robot according to one embodiment of the present disclosure from a frontal view, and FIG. 12 shows the bending of an intelligent exploration robot according to one embodiment of the present disclosure. The description of the above-described embodiment may be applied to the present embodiment in the same or similar manner.

[0097] The robot (10) may include a thrust module (150). The thrust module (150) may include a canopy (151). The canopy (151) may be adjusted to unfold or fold in a direction across the direction of movement. At this time, the canopy (151) may enable speed control when the robot (10) moves inside a closed conduit (P). That is, by unfolding or folding the canopy (151) during the movement of the robot (10), resistance can be adjusted to finely control the speed. The canopy (151) may be automatically adjusted as needed, and the efficiency of movement can be increased by folding in narrow spaces and unfolding in wide spaces during movement.

[0098] The thrust module (150) may include a propeller (152). The propeller (152) can control the speed of the robot (10) by providing thrust in the direction of movement or in the opposite direction of movement. The propeller (152) can control the speed by changing the rotation direction of the rotor blades or by adjusting the direction of the propeller (152) itself. For example, by changing the rotation direction of the rotor blades, thrust can be provided in the opposite direction of movement, allowing the speed to be reduced or reversed. Additionally, if the direction of the propeller (152) itself can be adjusted, thrust can be provided in the vertical direction within the fluid inside the closed pipe (P), allowing the robot (10) to move smoothly up and down. Through this, the robot (10) can finely control its movement according to various environmental conditions inside the closed pipe (P).

[0099] The canopy (151) and propeller (152) of the thrust module (150) described above can be controlled by the control module (160).

[0100] The robot (10) may include a body portion (170). The body portion (170) may be composed of a plurality of body portions. For example, it may include a first body portion (171), a second body portion (172), and a third body portion (173). Each body portion may enable the robot (10) to move stably and perform exploration tasks in various environments. The plurality of body portions may be designed so that the robot (10) can move fluidly while maintaining balance even in complex terrain.

[0101] The robot (10) may include a connecting part (180). The connecting part (180) may connect a plurality of body parts (e.g., a first body part (171), a second body part (172), a third body part (173)). The connecting part (180) may be composed of a plurality of connecting parts. For example, it may include a first connecting part (181) and a second connecting part (182). The connecting part (180) may enable flexible connection between each body part, allowing the robot (10) to move smoothly even in parts where the direction changes abruptly or bends inside the closed pipe (P). The connecting part (180) may be designed as an articulated structure or a bendable structure, allowing the robot (10) to move flexibly in narrow spaces or curved paths inside the closed pipe (P). Through this, the robot (10) can adapt to various closed pipe (P) structures and continuously perform exploration work.

[0102] In this way, the robot (10) can control speed and propulsion through the canopy (151) and propeller (152), and can adapt to various environments inside the closed pipe (P) through multiple body parts (170) and bendable connecting parts (180), thereby enabling smooth and efficient exploration.

[0103] The foregoing description of the present disclosure is provided to enable those skilled in the art to practice or use the present disclosure. Various modifications of the present disclosure will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to various variations without departing from the spirit or scope of the present disclosure. Accordingly, the present disclosure is not intended to be limited to the examples described herein, but is intended to be given the broadest possible scope consistent with the principles and novel features disclosed herein.

[0104] Although the present disclosure has been described in relation to some embodiments, it should be understood that various modifications and changes may be made without departing from the scope of the present disclosure as understood by a person skilled in the art to which the present disclosure pertains. Furthermore, such modifications and changes should be considered to fall within the scope of the claims appended to this specification.

Claims

1. Intelligent exploration robot; and When the intelligent exploration robot explores a closed pipeline, it includes an input control device provided to adjust the exploration distance of the intelligent exploration robot. The above intelligent exploration robot is, A camera module configured to acquire image data; An acoustic sensor configured to acquire acoustic data generated in the above-mentioned closed conduit; An artificial intelligence module configured to perform artificial intelligence learning; and It includes the camera module, the acoustic sensor, and a control module electrically connected to the artificial intelligence module. The above control module is, Image data inside the closed pipe is acquired using the above camera module, and Acoustic data generated in the closed pipe is obtained using the above acoustic sensor, and Data refinement is performed on the acquired image data and sound data through preprocessing to generate refined data, and Labeling data is generated through data processing that performs labeling on the above refined data based on a predefined class for the problem area of ​​the above closed pipeline, and Using the above artificial intelligence module, artificial intelligence learning is performed on the above labeling data to generate training data for the above problem domain, and An intelligent closed-pipe exploration system that performs identification and prediction regarding the problem area for newly acquired new image data and new acoustic data based on the above-mentioned training data.

2. In Claim 1, The above artificial intelligence module is, The process of the above artificial intelligence learning is implemented based on a semantic partitioning model and a multimodal data classification model capable of comprehensively analyzing the state of the above closed pipeline, and The above control module is, Using the above artificial intelligence module, the interior of the above closed pipeline is classified into multiple predefined states related to the problem domain through a semantic partitioning model, and An intelligent closed-pipe exploration system that learns by combining the image data and the acoustic data through the multimodal data classification model using the artificial intelligence module.

3. In Claim 2, The above control module is, An intelligent closed pipeline exploration system that identifies and predicts problem areas of a closed pipeline based on similarity with a previously learned problem area by combining and analyzing the new image data and the new sound data based on the above learning data.

4. In Claim 3, The above control module is, In generating the above labeling data, For the above image data, labeling including normal, joint, corrosion, sediment / foreign matter, and surface abnormalities is performed, and For the above acoustic data, labeling including normal and abnormal is performed based on a specific pattern in the frequency domain, and An intelligent closed pipeline exploration system in which a plurality of predefined states related to the problem domain in the above semantic partitioning model include normal, joint, corrosion, sediment / foreign matter, and surface anomaly.

5. In Claim 4, The above control module is, An intelligent closed pipeline exploration system that performs identification and prediction regarding the problem area with respect to the new image data and the new acoustic data, and generates identification result data and prediction result data regarding the problem area.

6. In Claim 5, The above intelligent exploration robot is, LiDAR sensor; and It includes an inertial sensor configured to sense position and attitude, and The above lidar sensor and the above inertial sensor are electrically connected to the control module, and The above control module is, Using the above LiDAR sensor, three-dimensional point cloud data regarding the structure of the closed pipeline is obtained, and Using the inertial sensor, position estimation data regarding the position and attitude of the intelligent exploration robot is obtained, and An intelligent closed pipeline exploration system that generates a three-dimensional closed pipeline model based on the identification result data, the prediction result data, the point cloud data, and the location estimation data.

7. In Claim 6, The above control module is, Based on the above point cloud data and the above position estimation data, a plurality of local maps are generated for each of the local sections within the entire section traveled by the intelligent exploration robot, and The above plurality of local maps are combined to form a plurality of sub-maps, An intelligent closed pipeline exploration system that generates a three-dimensional closed pipeline model by combining the above plurality of sub-maps.

8. In Claim 7, The above control module is, An intelligent closed pipeline exploration system that combines the image data and point cloud data acquired using the camera module to generate a plurality of local maps.

9. In Claim 8, The above control module is, When forming the above plurality of sub-maps, An intelligent closed pipeline exploration system that combines multiple local maps using a sliding window graph optimization technique to improve the combination accuracy of the above-mentioned maps.

10. In Claim 9, The above control module is, An intelligent closed pipeline exploration system that generates a visually displayed model using colors when generating the above 3D closed pipeline model.

11. In Claim 10, The above control module is, An intelligent closed pipeline exploration system capable of sharing the identification result data, the prediction result data, and the three-dimensional closed pipeline model data with an external management system.

Images