Pipeline condition determination device, pipeline condition determination method, and pipeline condition determination program

Abstract

To accurately detect abnormalities occurring inside a pipeline. [Solution] A pipeline condition determination device comprising: a forward-view internal image acquisition unit that acquires a forward-view internal image taken by a self-propelled robot cart equipped with at least one camera capable of imaging the inside of a pipeline while the robot cart travels inside the pipeline and the camera is directed toward the direction of travel of the self-propelled robot cart; an identification unit that identifies abnormalities and abnormal areas occurring inside the pipeline using the forward-view internal image and a trained identification model for identifying at least one abnormality and abnormal area that may occur inside the pipeline; and a superimposed image generation unit that generates an image by superimposing a border line on the forward-view internal image that indicates at least the identified type of abnormality and the range of the abnormal area in a way that makes them distinguishable from other types of abnormalities and ranges of abnormal areas.

Description

Technical Field

[0001] The present invention relates to a pipeline state determination device, a pipeline state determination method, and a pipeline state determination program.

Background Art

[0002] In the above technical field, Patent Document 1 discloses that an image of a specified one frame of an input video and an image obtained by superimposing a video analysis result on the image are displayed in parallel so as to be easily compared, thereby facilitating in-pipe diagnosis (paragraphs

[0153] to

[0157] , FIG. 18, etc. of the same document).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, in the technology described in Patent Document 1, video analysis is performed based on a video captured by a camera directed in the extending direction of the pipe, and in-pipe diagnosis is performed. Therefore, it has been impossible to accurately determine an abnormality occurring inside the pipeline.

Means for Solving the Problems

[0005] To achieve the above object, a pipeline state determination device according to the present invention a forward-view internal image acquisition unit that acquires a forward-view internal image obtained by imaging, with the camera directed in the traveling direction of the self-propelled robot cart, while the self-propelled robot cart equipped with at least one camera capable of imaging the inside of the pipeline travels inside the pipeline; A identification unit that identifies anomalies and abnormal regions occurring inside the pipeline using the forward-view internal image and a trained identification model for identifying at least one anomaly and abnormal region that may occur inside the pipeline, A superimposed image generation unit generates an image in which a border line indicating at least one identified type of anomaly and the range of the anomaly area in a way that allows them to be distinguished from other types of anomalies and ranges of anomalies is superimposed on the forward-view internal image. It is equipped.

[0006] Furthermore, in order to achieve the above objective, the pipeline condition determination method according to the present invention is: A forward-view internal image acquisition step involves a self-propelled robotic cart equipped with at least one camera capable of imaging the inside of a pipeline traveling through the pipeline and acquiring a forward-view internal image captured by the camera in the direction of travel of the self-propelled robotic cart, A selection step to identify an anomaly and anomaly region occurring inside the pipeline, using the forward-view internal image and a trained identification model for identifying at least one anomaly and anomaly region that may occur inside the pipeline; A superimposed image generation step generates an image in which a border line is superimposed on the forward-view internal image to indicate, at least one identified type of anomaly and the extent of the anomaly area in a way that allows them to be distinguished from other types of anomalies and the extent of the anomaly area; Includes.

[0007] Furthermore, in order to achieve the above objective, the pipeline condition determination program according to the present invention is: A forward-view internal image acquisition step involves a self-propelled robotic cart equipped with at least one camera capable of imaging the inside of a pipeline traveling through the pipeline and acquiring a forward-view internal image captured by the camera in the direction of travel of the self-propelled robotic cart, A selection step to identify an anomaly and anomaly region occurring inside the pipeline, using the forward-view internal image and a trained identification model for identifying at least one anomaly and anomaly region that may occur inside the pipeline; A superimposed image generation step generates an image in which a border line is superimposed on the forward-view internal image to indicate, at least one identified type of anomaly and the extent of the anomaly area in a way that allows them to be distinguished from other types of anomalies and the extent of the anomaly area; Have the computer execute it. [Effects of the Invention]

[0008] According to the present invention, it is possible to accurately determine abnormalities occurring inside a pipeline. [Brief explanation of the drawing]

[0009] [Figure 1] This figure illustrates the outline of the determination made by the pipeline condition determination device according to the first embodiment of the present invention. [Figure 2] This is a block diagram illustrating the configuration of a pipeline condition determination device according to the first embodiment of the present invention. [Figure 3] This figure shows an example of a bounding box table in a pipeline condition determination device according to the first embodiment of the present invention. [Figure 4] This is a diagram illustrating the hardware configuration of a pipeline condition determination device according to the first embodiment of the present invention. [Figure 5] This is a flowchart illustrating the processing procedure of the pipeline condition determination device according to the first embodiment of the present invention. [Modes for carrying out the invention]

[0010] The embodiments for carrying out the present invention will be described in detail below with reference to the drawings. However, the configurations, numerical values, processing flow, functional elements, etc., described in the following embodiments are merely examples, and they can be freely modified or changed, and the technical scope of the present invention is not intended to be limited to the following description.

[0011] A pipe damage identification device according to the first embodiment of the present invention will be described with reference to Figures 1 to 5. Figure 1 is a diagram illustrating the overview of the pipe damage identification device according to this embodiment. The pipe condition determination device 100 is a device that displays bounding lines that identify abnormalities and abnormal areas occurring inside a pipe 110, such as a sewer pipe, by superimposing them onto an internal image taken inside the pipe 110.

[0012] In inspections of the inside of pipelines 110, such as sewer pipes, the inspection is carried out using internal images (inner surface images) of the pipeline 110 captured by a camera 121 on a self-propelled robotic trolley 120. Workers use a display installed outside the pipeline 110 to check the images captured by the camera 121 and inspect whether or not there are any abnormalities such as damage to the pipeline 110.

[0013] First, the worker positions the self-propelled robot cart 120 at the inspection start position of the pipeline 110, with the camera 121 facing the inspection target area (the direction of travel of the self-propelled robot cart 120). After that, the self-propelled robot cart 120 starts moving while imaging the inside of the pipeline 110 with the camera 121.

[0014] When the self-propelled robot cart 120 starts moving, the pipeline condition determination device 100 acquires a forward-view internal image 130 (an image focused on the far side of the pipeline 110) from the camera 121, which captures the direction of travel of the self-propelled robot cart 120. Based on the acquired forward-view internal image 130, the pipeline condition determination device 100 identifies any abnormalities that may occur inside the pipeline 110, and if an abnormality has occurred, the area in which the abnormality is occurring. The identification of abnormalities and the area in which the abnormality is occurring is performed using a trained identification model.

[0015] The pipeline state determination device 100 superimposes and displays on the forward-view internal image 130 bounding lines 131, 132, 133 indicating a specified abnormality and the area where the abnormality has occurred. For example, the bounding line 131 indicates the area where "incoming water" has occurred, the bounding line 132 indicates the area where "joint displacement" has occurred, and the bounding line 133 indicates the area where "damage" has occurred. Note that the pipeline state determination device 100 identifies abnormalities and the like using a pre-prepared learned model (a dedicated learned model) for each type of abnormality.

[0016] Finally, when all abnormalities and their occurrence areas are identified, an image 140 in which the bounding lines 131, 132, 133 are superimposed on the forward-view internal image 130 is generated. Terms (character information 135) indicating the name of the abnormality and the like may also be displayed in the areas around the bounding lines 131 to 133. Further, the image 140 may also be displayed together with a distance scale 134 indicating the distance from the starting point.

[0017] Next, the configuration of the pipeline state determination device 100 will be described with reference to FIG. 2. The pipeline state determination device 100 includes a forward-view internal image acquisition unit 201, an identification unit 202, and a superimposed image generation unit 203.

[0018] The forward-view internal image acquisition unit 201 acquires a forward-view internal image 130 obtained by imaging with the camera 121 facing the traveling direction of the self-propelled robot cart 120 while the self-propelled robot cart 120 equipped with at least one camera 121 capable of imaging the inside of the pipeline 110 travels inside the pipeline 110.

[0019] Here, the pipeline 110 (pipe channel) refers to a waterway and is something made for the purpose of water supply and drainage. For example, the pipeline 110 includes a water supply pipe, a sewer pipe, a water supply pipe, a drainage pipe, and the like. Also, the materials of the pipeline 110 include concrete, pottery, iron, etc., and the types of the pipeline 110 include concrete pipes, concrete culvert pipes, ceramic pipes, iron pipes, etc.

[0020] In this embodiment, the pipe diameter of the conduit 110 is assumed to be approximately 450 mm, but it is not limited to this, and conduits 110 of various diameters can be used.

[0021] Camera 121 may be any type of camera, such as a standard camera, a wide-angle camera, or a 360-degree camera. Camera 121 may also have features such as zoom and autofocus. The number of cameras 121 installed is not limited to one; there may be multiple cameras. If multiple cameras 121 are installed, each camera may have different performance characteristics.

[0022] The identification unit 202 identifies abnormalities and abnormal regions occurring inside the conduit 110 using the forward-view internal image 130 and a trained identification model for identifying at least one abnormality and abnormal region that may occur inside the conduit 110.

[0023] First, an abnormality is defined as at least one of the following: damage to the pipeline 110, cracks, water infiltration, protrusion of the connecting pipe, intrusion of tree roots, and mortar adhesion, and includes a condition in which the pipeline 110 cannot perform its expected function.

[0024] The trained specific model is obtained by training the artificial intelligence with forward-view internal images 130 and side-view internal images captured by the camera 121 facing in a direction perpendicular to the direction of travel of the self-propelled robot cart 120.

[0025] The forward-view internal image 130 is an image captured by the camera 121 in the direction of extension of the pipeline 110. In other words, it is an image captured with the camera 121 pointed in the direction of extension of the pipeline 110, and the focus of the camera 121 is set to infinity, for example, in the case of a straight section of the pipeline 110. The forward-view internal image 130 is used to guide the self-propelled robot cart 120 through the pipeline 110 to reach the target position.

[0026] In contrast, the side-view internal image is an image captured by the camera 121 mounted on the self-propelled robot carriage 120 at the location where the anomaly is occurring, and is an image of the anomaly captured from a close distance. Furthermore, the side-view internal image has a higher resolution than the forward-view internal image 130.

[0027] The superimposed image generation unit 203 generates an image 140 by superimposing the outlines 131 to 133, which indicate at least the identified type of anomaly and the range of the anomaly area in a way that distinguishes them from other types of anomalies and the ranges of the anomaly areas, onto the forward-view internal image 130.

[0028] Furthermore, the superimposed image generation unit 203 generates an image in which specific information, including the distance 134, the area of ​​the region indicated by the surrounding lines 131-133, and the rank of the anomaly, is further superimposed. In addition, the superimposed image generation unit 203 generates an image 140 in which character information 135 indicating the type of anomaly is further superimposed on the forward-view internal image. In this way, by superimposing various information onto the image 140, various information can be obtained from a single image 140.

[0029] As described above, by training artificial intelligence using lateral view internal images to determine the type, size, extent, and rank of anomalies, and by combining the results of training with forward view internal images 130 and lateral view internal images, it becomes possible to accurately identify areas where anomalies are occurring simply by driving a self-propelled robotic cart 120 inside the pipeline 110. Thus, because lateral view internal images are images taken from a close distance and are high-resolution images, the type and rank of anomalies can be identified with even greater accuracy.

[0030] Next, an example of the outline table 301 of the pipeline condition determination device 100 will be described with reference to Figure 3. The outline table 301 stores the outline pattern 312 in association with the abnormality type 311. The abnormality type 311 is the type of abnormality and includes items such as damage, cracks, water infiltration, connecting pipes, tree root intrusion, and mortar adhesion. The outline pattern 312 includes the color and line type (solid line, dotted line, etc.) of the outline and the shape of the outline (circular shape, elliptical shape, etc.). The pipeline condition determination device 100 then refers to the outline table 301 and superimposes the outlines 131 to 133 onto the forward-view internal image 130 to generate image 140.

[0031] Referring to Figure 4, the hardware configuration of the pipeline condition determination device 100 will be described. The CPU (Central Processing Unit) 410 is a processor for arithmetic control and realizes the various functional configurations of the pipeline condition determination device 100 shown in Figure 2 by executing programs. The CPU 410 may have multiple processors and may execute different programs, modules, tasks, threads, etc. in parallel. The ROM (Read Only Memory) 420 stores initial data, fixed data such as programs, and other programs. The network interface 430 communicates with other devices via the network. Note that the CPU 410 is not limited to one, and may have multiple CPUs, or may include a GPU (Graphics Processing Unit) for image processing. Furthermore, it is desirable that the network interface 430 has a CPU independent of the CPU 410 and writes or reads transmitted and received data to or from the RAM (Random Access Memory) 440 area. It is also desirable to provide a DMAC (Direct Memory Access Controller) for transferring data between the RAM 440 and the storage 450 (not shown). Furthermore, the CPU 410 recognizes that data has been received or transferred to the RAM 440 and processes the data. The CPU 410 also prepares the processing results in the RAM 440 and leaves subsequent transmission or transfer to the network interface 430 or DMAC.

[0032] RAM 440 is a random access memory used by the CPU 410 as a work area for temporary storage. RAM 440 has a memory area reserved for storing the data necessary to realize this embodiment. Current position data 441 is data about the position of the self-propelled robot cart 120 inside the pipeline 110, for example, data about how far it is from the entrance (starting point) of the pipeline 110. Image data 442 is an image of the inside of the pipeline 110 taken by the camera 121. Anomaly / anomaly area data 443 is data about identified anomalies and anomaly areas. Bounding line data 444 is data about bounding lines 131 to 133 that indicate the identified anomaly area in a way that makes it distinguishable from other anomaly areas.

[0033] The transmitted and received data 446 is data transmitted and received via the network interface 430. The RAM 440 also has an application execution area 447 for running various application modules.

[0034] The storage 450 stores the database, various parameters, and the following data or programs necessary for realizing this embodiment. The storage 450 stores the outline table 301. The outline table 301 is a table that manages the relationship between the abnormality type 311 and the outline pattern 312, as shown in Figure 3.

[0035] The storage 450 further stores a forward-view internal image acquisition module 451, a identification module 452, and a superimposed image generation module 453. The forward-view internal image acquisition module 451 acquires a forward-view internal image 130 captured in the direction of travel of the self-propelled robot cart 120 by a camera 121 mounted on the self-propelled robot cart 120 while the self-propelled robot cart 120 is traveling inside the conduit 110. The identification module 452 is a module that identifies anomalies and anomaly areas occurring inside the conduit 110 using the acquired forward-view internal image 130 and a trained model for identifying anomalies and anomaly areas. The superimposed image generation module 453 is a module that generates an image by superimposing a border line on the forward-view internal image 130 that distinguishes the identified type of anomaly and the extent of the anomaly area from other types of anomalies and anomaly area extents. These modules 451 to 453 are read by the CPU 410 into the application execution area 447 of the RAM 440 and executed. The control program 457 is a program for controlling the entire pipeline condition determination device 100.

[0036] Next, the processing procedure of the pipeline condition determination device 100 will be explained with reference to the flowchart shown in Figure 5. This flowchart is executed by the CPU 410 in Figure 4 using the RAM 440, and realizes the various functional configurations of the pipeline condition determination device 100 in Figure 2.

[0037] In step S501, the forward-view internal image acquisition unit 201 acquires a forward-view internal image 130 by orienting the camera 121 in the direction of travel of the self-propelled robot carriage 120 inside the conduit 110. In step S503, the identification unit 202 uses the acquired forward-view internal image 130 and a trained identification model for identifying at least one abnormality and abnormal area that may occur inside the conduit 110 to determine whether or not it has been able to identify the abnormality and abnormal area occurring inside the conduit 110. If it is determined that it has not been able to identify the abnormality (NO in step S503), the conduit condition determination device 100 returns to step S501. If it is determined that it has been able to identify the abnormality (YES in step S503), the conduit condition determination device 100 proceeds to step S505.

[0038] In step S505, the superimposed image generation unit 203 generates outlines 131 to 133 that distinguish the identified abnormalities and abnormal areas from other abnormalities. In step S507, the superimposed image generation unit 203 generates an image 140 by superimposing the generated outlines 131 to 133 onto the forward-view internal image 130. In step S509, the pipeline condition determination device 100 determines whether the identification of abnormalities, etc., has been completed for the entire length of the pipeline 110. If it is determined that the identification has not been completed (NO in step S509), the pipeline condition determination device 100 returns to step S501. If it is determined that the identification has been completed (YES in step S509), the pipeline condition determination device 100 terminates the process.

[0039] According to this embodiment, by creating such superimposed images, various information can be obtained from a single image, and can therefore be used to estimate the construction period and costs of repair and restoration work on the pipeline 110. Furthermore, since various information is superimposed on the image, it can also be used as supporting documentation to be attached to applications and reports for repair and other construction work.

[0040] Although the present invention has been described above with reference to embodiments, the present invention is not limited to the embodiments described above and can be modified as appropriate. Various modifications to the configuration and details of the present invention can be made that will be understood by those skilled in the art within the scope of the present invention. Furthermore, any system or apparatus that combines the separate features included in each embodiment in any way is also included in the scope of the present invention.

[0041] Furthermore, the present invention may be applied to a system composed of multiple devices or to a single device. Moreover, the present invention is also applicable when an information processing program that realizes the functions of the embodiment is supplied to a system or device and executed by a built-in processor. Therefore, the technical scope of the present invention includes programs installed on a computer to realize the functions of the present invention on a computer, the medium on which the program is stored, the WWW (World Wide Web) server that allows the program to be downloaded, and the processor that executes the program. In particular, at least a non-transitory computer-readable medium containing a program that causes a computer to execute the processing steps included in the above-described embodiment is included in the technical scope of the present invention.

Claims

1. A forward-view internal image acquisition unit acquires a forward-view internal image taken by a self-propelled robot cart equipped with at least one camera capable of imaging the inside of a pipeline, while the robot cart travels inside the pipeline and the camera is positioned in the direction of travel of the self-propelled robot cart. A identification unit that identifies anomalies and abnormal regions occurring inside the pipeline using the forward-view internal image and a trained identification model for identifying at least one anomaly and abnormal region that may occur inside the pipeline, A superimposed image generation unit generates an image in which a border line indicating at least one identified type of anomaly and the range of the anomaly area in a way that allows them to be distinguished from other types of anomalies and ranges of anomalies is superimposed on the forward-view internal image. A pipeline condition determination device equipped with the following features.

2. The pipeline condition determination device according to claim 1, wherein the superimposed image generation unit generates an image in which specific information including distance, area of ​​region indicated by the surrounding line, and rank of abnormality is further superimposed and displayed.

3. The pipe condition determination device according to claim 1, wherein the trained specific model is obtained by training artificial intelligence with the forward-view internal image and the side-view internal image captured with the camera oriented in a direction perpendicular to the direction of travel.

4. The pipeline condition determination device according to claim 3, wherein the abnormality is at least one of the following: damage to the pipeline, cracks, water infiltration, protrusion of a connecting pipe, intrusion of tree roots, and adhesion of mortar.

5. The pipeline condition determination device according to claim 1, wherein the superimposed image generation unit generates an image in which character information indicating the type of abnormality is further superimposed on the forward-view internal image.

6. The pipeline condition determination device according to claim 1, wherein the pipeline is a sewer pipe.

7. A forward-view internal image acquisition step involves a self-propelled robotic cart equipped with at least one camera capable of imaging the inside of a pipeline traveling through the pipeline and acquiring a forward-view internal image captured by the camera in the direction of travel of the self-propelled robotic cart, A selection step to identify an anomaly and anomaly region occurring inside the pipeline, using the forward-view internal image and a trained identification model for identifying at least one anomaly and anomaly region that may occur inside the pipeline; A superimposed image generation step generates an image in which a border line is superimposed on the forward-view internal image to indicate, at least one identified type of anomaly and the extent of the anomaly area in a way that allows them to be distinguished from other types of anomalies and the extent of the anomaly area; A method for determining the condition of a pipeline, including the following:

8. A forward-view internal image acquisition step involves a self-propelled robotic cart equipped with at least one camera capable of imaging the inside of a pipeline traveling through the pipeline and acquiring a forward-view internal image captured by the camera in the direction of travel of the self-propelled robotic cart, A selection step to identify an anomaly and anomaly region occurring inside the pipeline, using the forward-view internal image and a trained identification model for identifying at least one anomaly and anomaly region that may occur inside the pipeline; A superimposed image generation step generates an image in which a border line is superimposed on the forward-view internal image to indicate, at least one identified type of anomaly and the extent of the anomaly area in a way that allows them to be distinguished from other types of anomalies and the extent of the anomaly area; A pipeline condition determination program that is executed by a computer.

Images