In-pipe inspection device, in-pipe inspection method and program

Abstract

To enable output of results of in-pipe inspection for each part.SOLUTION: An in-pipe inspection device comprises an output processing unit that displays, on a display device, information indicating a degree of abnormality for each part of a pipe, on the basis of a video captured by a camera moving inside the pipe.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] An embodiment of the present invention relates to an inside-pipe inspection device, an inside-pipe inspection method, and a program. [Background technology]

[0002] When sediment or rust accumulates inside pipes such as water pipes after many years of use, various abnormalities occur inside the pipes, including the accumulation and adhesion of foreign matter and deterioration and peeling of the inner surface of the pipe.

[0003] To inspect the inside of a pipe such as a water pipe, for example, a camera is inserted into the pipe and the camera is moved around inside the pipe and stopped along the way to capture images of the entire inside of the pipe. Based on the images obtained, a diagnosis is made as to whether or not there is an abnormality inside the pipe, and the severity of the abnormality, etc. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent No. 6532412 [Patent Document 2] Patent Publication No. 2021-162547 [Patent Document 3] JP 2023-055064 A Summary of the Invention [Problem to be solved by the invention]

[0005] Pipes, such as water pipes, are made up of various sections, such as straight sections, joints, perforated sections, T-shaped sections, and bends. The types of abnormalities that are likely to occur and how they progress vary from section to section, and the way in which abnormalities should be responded to when they occur also differs from section to section. For this reason, it is desirable to be able to obtain the results of internal pipe inspections for each section. However, it is not easy to obtain the results of internal pipe inspections for each section from the images of the inside of the pipe that have been taken.

[0006] An object of the present invention is to provide an inside-pipe inspection device, an inside-pipe inspection method, and a program that are capable of outputting the results of an inside-pipe inspection for each portion. [Means for solving the problem]

[0007] The inside-pipe inspection device of the embodiment includes an output processing unit that causes a display device to display information indicating the degree of abnormality for each part of the pipe based on images captured by a camera moving inside the pipe. [Brief description of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram showing an example of the overall configuration of a system including an inside-pipe inspection device according to an embodiment. [Diagram 2] FIG. 2 is a diagram showing an example of the configuration of an inside-pipe inspection device 1 according to an embodiment. [Diagram 3] FIG. 3 is a diagram showing an example of a state in which a camera 4 is inserted into a water pipe 3. As shown in FIG. [Figure 4] FIG. 4 is a diagram conceptually showing the relationship between various parts to be detected and each image frame. [Diagram 5] FIG. 5 is a diagram showing an example in which a substance causing an abnormality (abnormal substance) is present at joint P2 in the direction of movement of camera 4 moving inside the pipe. [Figure 6] FIG. 6 is a diagram showing an example of several successive frame images showing the vicinity of the joint P2 where the abnormal substance exists. [Figure 7] FIG. 7 is a diagram showing an example of settings for various association processes based on the example of the frame image in FIG. [Figure 8] FIG. 8 is a diagram showing an example of an anomaly detection result indicating the presence or absence of an anomaly for each part based on the setting example of FIG. [Figure 9] FIG. 9 is a diagram showing an example of a region map. [Figure 10] FIG. 10 is a diagram showing a first setting example based on the example of the area map in FIG. [Figure 11] FIG. 11 is a diagram showing a second setting example based on the example of the area map in FIG. [Figure 12] FIG. 12 is a diagram showing a third setting example based on the example of the area map in FIG. [Figure 13] FIG. 13 is a diagram showing a fourth setting example based on the example of the area map in FIG. [Figure 14] FIG. 14 is a diagram showing an example of anomaly detection results indicating the presence or absence of anomaly for each part based on the setting examples of FIGS. [Figure 15] FIG. 15 is a diagram showing a method for determining the presence or absence of an abnormality in each of the straight portion A, the joint portion P2, and the straight portion B based on the region map in FIG. 9 and using the settings shown in FIG. [Figure 16] FIG. 16 is a diagram showing a method for determining the presence or absence of an abnormality in each of the straight portion A, the joint portion P2, and the straight portion B based on the region map in FIG. 9 and using the settings shown in FIG. [Figure 17] FIG. 17 is a diagram showing a method for determining the presence or absence of an abnormality in each of the straight portion A, the joint portion P2, and the straight portion B based on the region map in FIG. 9 and using the settings shown in FIG. [Figure 18] FIG. 18 is a diagram showing a method for determining the presence or absence of an abnormality in each of the straight portion A, the joint portion P2, and the straight portion B based on the region map in FIG. 9 and using the settings shown in FIG. [Figure 19] FIG. 19 is a diagram showing a specific example of a method for obtaining an integrated determination result. [Figure 20] FIG. 20 is a diagram showing another specific example of a method for obtaining an integrated determination result. [Figure 21] FIG. 21 is a diagram showing an example of a screen that the output processing unit 50 causes the display device to display. [Figure 22] FIG. 22 is a flowchart showing an example of the operation of the inside of a pipe inspection device according to the embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, an embodiment will be described with reference to the drawings.

[0010] <System configuration> FIG. 1 shows an example of the overall configuration of a system including an inside-pipe inspection device according to an embodiment.

[0011] In this embodiment, the explanation will be given assuming that an inspection is performed inside a water pipe (a drinking water pipe) as an example, however, this embodiment is not limited to this example and can also be applied to the inspection of the inside of various types of pipes including, for example, sewer pipes.

[0012] The pipe inspection device 1 according to this embodiment inspects the inside of a water pipe 3 installed at a site 2 based on images captured by a camera 4 moving inside the pipe as shown in FIG. 1. The images captured by the camera 4 are input to an information terminal (for example, a personal computer, a smartphone, or a tablet). The images are, for example, moving images made up of a plurality of frame images (also simply called "frames"). The information terminal may also store attribute information indicating attributes associated with the camera 4. The captured images (i.e., moving images) are, for example, recorded on a recording medium 5 as video data (files) made up of a plurality of consecutive frame images, or uploaded to and stored in a storage unit of a predetermined server on a network (on the cloud).

[0013] The inside-pipe inspection device 1 imports video data recorded on the recording medium 5, or imports video data stored in the memory of a specified server, and uses these to inspect the inside of the pipe. The attribute information may be imported into the inside-pipe inspection device 1 via the recording medium 5 or the server together with the video data, or may be imported into the inside-pipe inspection device 1 via a route different from the video data, without going through the recording medium 5 or the server.

[0014] An example of the configuration of the inside of a pipe inspection device 1 according to an embodiment is shown in Fig. 2. Fig. 2 is a diagram showing an example of the configuration of the inside of a pipe inspection device 1.

[0015] 2 includes, as various functions, a video input unit 10, an attribute information input unit 11, a camera position detection unit 20, a part detection unit 21, a correspondence processing unit 22, an anomaly detection unit 30, an integrated judgment unit 40, a storage unit 41, a management unit 42, and an output processing unit 50. Some or all of these functions are realized as a program to be executed by one or more computers.

[0016] 2 is merely an example, and is not limited to this example. For example, the inside of the pipe inspection device 1 is not limited to one device, but may be configured by multiple devices, and these multiple devices may exist in different locations. In addition, the various functions described above may be provided in different devices and may exist in different locations.

[0017] For example, the video input unit 10, attribute information input unit 11, camera position detection unit 20, body part detection unit 21, association processing unit 22, abnormality detection unit 30, integrated judgment unit 40, storage unit 41, and management unit 42 may be provided in a predetermined server, and the output processing unit 50 may be provided in an arbitrary information terminal (for example, a personal computer, a smartphone, or a tablet). In this case, the output processing unit 50 may be provided in advance to the arbitrary information terminal side from the predetermined server as a program that realizes a UI (User Interface).

[0018] The image input unit 10 inputs image data (hereinafter sometimes abbreviated as “image”) captured by a camera 4 that captures the inside of a water pipe 3.

[0019] The attribute information input unit 11 inputs attribute information of the camera 4. The attribute information is supplied to the camera position detection unit 20, and is also supplied to the part detection unit 21, the association processing unit 22, the abnormality detection unit 30, and the integrated judgment unit 40 via the management unit 42, and is used as necessary in the processing of each unit.

[0020] The camera position detection unit 20 detects the position of the camera 4 within the pipe, using the video input by the video input unit 10 and the attribute information input by the attribute information input unit 11. Position information indicating the position of the camera 4 is supplied to the part detection unit 21, the association processing unit 22, the abnormality detection unit 30, the integrated judgment unit 40, etc. through the management unit 42, and is used as necessary in the processing of each unit.

[0021] The part detection section 21 detects the type of part of the tube 3 shown in each of a plurality of frame images constituting the video input by the video input section 10 .

[0022] Types of sections include straight sections, junctions, perforated sections, T-shaped sections, and curved sections. Junctions, T-shaped sections, and curved sections are called irregular pipe sections. Straight sections are sections that extend linearly in the longitudinal direction of the pipe. Individual straight sections are connected by irregular pipe sections (junctions, T-shaped sections, or curved sections). Junctions, for example, join two straight sections together. Perforated sections are thin sections of pipe that branch off in either direction halfway through the pipe. T-shaped sections are sections where the pipe branches into two. Curved sections are sections where the pipe is curved.

[0023] For example, the part detection unit 21 performs image analysis on each frame image and infers which part of the pipe 3 is shown on the screen based on the captured features.

[0024] Each type of site has the following characteristics, and these characteristics can be used to detect each type of site.

[0025] Joint: Appears circular in the image.

[0026] Perforation: The image shows a hole leading to a thin tube at the top of the tube.

[0027] T-shaped pipe section: The image shows two branches at the back of the pipe, in a dark area where no light reaches.

[0028] Bends: The image shows a dark area at the back of the tube that is offset in either direction from the center of the tube.

[0029] Straight part: Anything that does not fall into any of the above four parts is shown.

[0030] In addition to general image processing, image recognition and object detection techniques using AI (Artificial Intelligence) may be applied to the part detection process. For example, the accuracy of part detection may be improved by having AI learn the various features described above.

[0031] When a plurality of parts are captured in one frame image, part detection section 21 outputs the plurality of parts as the part detection result.

[0032] The association processing unit 22 performs a process of associating each of the parts of the pipe 3 with each of the multiple frame images constituting the video based on the part detection result of the part detection unit 21. The association process is a process of determining from which frame image (or from which image area within a frame image) an abnormality in each of the various parts should be detected. The specific contents of the process will be described later.

[0033] The abnormality detection unit 30 detects abnormalities for each part of the pipe 3 from a plurality of frame images constituting the video, based on the result of the association process by the association processing unit 22.

[0034] Specifically, the abnormality detection unit 30 judges the presence or absence of an abnormality and the abnormality rank (abnormality level) for each part, and generates the judged result as an abnormality detection result for each detection target. The detection target here refers to any of rust, deposits, partial adhesion of foreign matter, deterioration of anti-corrosion protection on the pipe surface, dirt attached in a film form to the entire inside of the pipe, and floating matter.

[0035] Anomaly detection for each detection target is performed, for example, by image processing based on characteristics such as brightness, color, shape, edges, etc. In addition, the accuracy of anomaly detection may be improved by having AI learn these characteristics.

[0036] Based on the abnormality detection results from the abnormality detection section 30, the integrated judgment section 40 judges the abnormality rank (abnormality level) indicating the degree of abnormality for the five types of detection targets for each part of the pipe 3.

[0037] For example, the integrated judgment unit 40 integrates the abnormality detection results generated by the abnormality detection unit 30 for each part of the pipe 3 through a predetermined process, and generates an integrated judgment result for each part of the pipe 3. The specific contents of the process will be described later. The abnormality detection results and the integrated judgment result are stored in the storage unit 41.

[0038] The storage unit 41 stores the abnormality detection result of the abnormality detection unit 30 and the integrated judgment result of the integrated judgment unit 40.

[0039] The management unit 42 controls and manages each unit so that a series of processes by the attribute information input unit 11, the camera position detection unit 20, the abnormality detection unit 30, the integrated judgment unit 40, and the memory unit 41 are performed appropriately according to a predetermined procedure.

[0040] For example, the management unit 42 supplies the position information of the camera 4 obtained from the camera position detection unit 20 or the attribute information obtained from the attribute information input unit 11 to the part detection unit 21, the association processing unit 22, the abnormality detection unit 30, and the integrated judgment unit 40. The position information and attribute information of the camera 4 can be used to identify the image frame (identify the frame number). Therefore, each of the part detection unit 21, the association processing unit 22, the abnormality detection unit 30, and the integrated judgment unit 40 can associate information related to each process with the frame number.

[0041] The output processing unit 50 causes a display device to display information indicating the degree of abnormality for each part of the pipe 3 based on images captured by the camera 4 moving inside the pipe.

[0042] The output processing unit 50 has, as its main functions, a function for displaying on the display device various kinds of information that can be operated by the user, and a function for displaying on the display device various kinds of information according to the user's operation. The output processing unit 50 acquires the abnormality detection result of the abnormality detection unit 30 and the integrated judgment result of the integrated judgment unit 40 from the storage unit 41, accepts the user's operation, and outputs various kinds of information including the abnormality detection result for each part and the integrated judgment result to the display device according to the user's operation. Details of these functions will be described later.

[0043] <Details of main components> The main elements constituting the inside of pipe inspection device 1 will be described in detail below.

[0044] Video input section 10 The video input unit 10 will now be described in detail.

[0045] The video input unit 10 receives and inputs video of the inside of the pipe taken by the camera 4 from a specified information terminal via a recording medium 5 or a server. For example, as shown in Fig. 1, a video data file can be saved in a recording medium 5 such as an SD card, and the video data can be input by reading the file using a reader that reads the recording medium 5.

[0046] Fig. 3 shows an example of a state in which a camera 4 is inserted into a water pipe 3. As shown in Fig. 3, the camera 4 may be waterproofed as a whole in order to move inside the water pipe 3 at the site 2. The camera 4 may also be equipped with a function to illuminate the inside of the pipe and a rotation mechanism to keep the camera 4 vertical and horizontal at all times. A long cable 6 is connected to the camera 4, and the camera 4 is inserted into the water pipe 3 by pushing the cable 6 from a fire hydrant 7 or the like, and the inside of the water pipe 3 is photographed.

[0047] The image from the camera 4 is supplied to an information terminal outside the fire hydrant 7 via a cable 6, and is recorded on the recording medium 5 as described above, or uploaded to a specified server on the cloud and stored in a specified memory unit. The image may be recorded in any format, and the format is not important. For example, the image may be in the analog NTSC format, or in a digital uncompressed format or a digitally encoded format.

[0048] ·Attribute information input section 11 The attribute information input unit 11 will be described.

[0049] As described above, the attribute information may be imported into the inside pipe inspection device 1 together with the video data via the recording medium 5 or server, or may be imported into the inside pipe inspection device 1 via a route different from that of the video data, without going through the recording medium 5 or server, or may be input from the camera 4 or an input device such as a keyboard or mouse. In this case, information obtained from a sensor or the like in the camera 4 may be used as the attribute information. The attribute information generated in either form may be stored as a data file on a recording medium such as an SD card, and the attribute information may be input by reading the file with a reader that reads the medium.

[0050] The attribute information input by the attribute information input unit 11 includes, for example, information indicating the length (cable length) of the cable 6 when the camera 4 is inserted into the water pipe 3, which becomes the position information of the camera 4 within the pipe.

[0051] The information indicating the cable length can be used in the camera position detection unit 20, which will be described later, by superimposing the information on a predetermined position of the video data and recording it. If the video is recorded in digital format, the information indicating the cable length may be recorded in synchronization with the video data, rather than being superimposed on the video data.

[0052] Other attribute information can also be input in the attribute information input unit 11. For example, attribute information such as the location of the water pipe (place name, latitude and longitude, address, etc.), the type of water pipe, the diameter of the water pipe, the year the water pipe was installed, the date and time of the previous inspection and the diagnosis result at that time, and the model and lens parameters of the camera can be input in the attribute information input unit 11 by recording the information in association with the recording medium that records the video.

[0053] Camera position detection unit 20 The camera position detection unit 20 will now be described in detail.

[0054] The camera position detection unit 20 detects the current position of the camera 4 within the pipe. In the configuration of Fig. 2, it is assumed that the video data and attribute information are input to the camera position detection unit 20 via the video input unit 10 and attribute information input unit 11, and the camera position detection unit 20 detects the position of the camera 4 using the video data and attribute information.

[0055] A method for detecting the position of the camera 4 using video data will be described below.

[0056] The simplest method for detecting the position of camera 4 is to read information indicating the cable length (i.e., position information of camera 4) from the attribute information input by attribute information input unit 11. If the video data is in digital format, this can be achieved by simply embedding information indicating the cable length in association with each image frame in the video data and reading the value.

[0057] If information indicating the cable length is superimposed on the video data, the information can be read using optical character recognition (OCR). In this case, recognition is possible by finding the area where characters are lined up horizontally in each image, without specifying the character display area in the camera 4.

[0058] Body part detection unit 21 The following describes the details of the part detection unit 21. Note that since the processing of the part detection unit 21 also affects the processing of the abnormality detection unit 30, the abnormality detection unit 30 will also be described here.

[0059] As described above, the part detection unit 21 detects the type of part of the pipe 3 shown in each of the multiple frame images constituting the video input by the video input unit 10. The types of parts include straight parts, joints, perforated parts, T-shaped pipe parts, and curved parts, but for ease of understanding, the illustration will be simplified and an example will be given in which detection is performed for two types of parts: straight parts and joints.

[0060] FIG. 4 conceptually shows the relationship between various parts to be detected and each image frame.

[0061] As shown in FIG. 4, when the pipe 3 is viewed section by section, from the left, joint P1, straight section A, joint P2, straight section B, joint P3, straight section C, and joint P4 are arranged.

[0062] An image captured by camera 4 of the inside of such a pipe 3 is regarded as one unit. The image is a series of multiple frame images, and among these multiple frame images, there are one or multiple frame images that respectively capture the above-mentioned joint P1, straight portion A, joint P2, straight portion B, joint P3, straight portion C, and joint P4.

[0063] A judgment section is provided for each part in the multiple frame images that make up the video. The judgment section indicates which frame images in the video should be treated as one unit and from which frame images should be used for anomaly detection (or integrated judgment) for each part, in order to generate the results of the in-pipe inspection for each part.

[0064] Fig. 5 shows an example of a case where a substance causing an abnormality (anomaly substance) is present at joint P2 in the direction of movement of camera 4 moving inside the pipe. Fig. 6 shows an example of several consecutive frame images capturing the vicinity of joint P2 where the abnormal substance is present.

[0065] 5, the camera 4 moves from the left to the right in the order of straight portion Z, joint portion P1, straight portion A, joint portion P2, straight portion B, and joint portion P3. At this time, it is assumed that an abnormal substance such as a rust bump is present in joint portion P2.

[0066] The position of the camera 4 inside the pipe 3 is identified by the camera position detection unit 20 as described above. In this case, for example, the inspection start point may be set to 0.0 m, and the position of the camera 4 may be identified from the travel distance calculated using information on the length of a cable attached to the camera 4 as the camera 4 moves deeper, or from the latitude and longitude on a map. The position information of the camera 4 is ultimately linked to the results of the in-pipe inspection for each part, and is used to indicate the results of each location.

[0067] The part detection unit 21 can identify the frame number of the image frame from the position information (or attribute information) of the camera 4.

[0068] Here, the frame images of frame numbers 1, 2, 3, 4, 5, 6, and 7 are assumed to correspond to the inspection start point of 0.0 m, 1.0 m, 1.5 m, 1.8 m, 2.1 m, 2.5 m, and 3.0 m, respectively, from the inspection start point.

[0069] 6 shows seven frame images (frame numbers 1 to 7) that show the vicinity of the joint P2 where the abnormal substance exists. The conditions of each of the seven frame images (frame numbers 1 to 7) are described below.

[0070] (Frame number 1) The straight portion A is visible, but the joint P2 is not visible. The abnormal substance at the joint P2 is also not visible. At this time, the part detection unit 21 detects the straight portion A as the type of part.

[0071] (Frame number 2) The straight portion A is visible, but the joint P2 is not. However, a small amount of abnormal material caused by the joint P2 is visible. At this time, the part detection unit 21 detects the straight portion A as the type of part (however, depending on the settings in the association processing unit 22 described later, the anomaly detection unit 30 may detect anomalies not in the straight portion A but in the joint P2, or may detect anomalies in both the straight portion A and the joint P2). The anomaly detection unit 30 detects an anomaly in the joint P2 (however, depending on the settings in the association processing unit 22 described later, the anomaly detection may be disabled, and the anomaly detection unit 30 may not detect an anomaly).

[0072] (Frame number 3) The straight portion A is shown, as are the joint P2 and the straight portion B. An abnormal substance caused by the joint P2 is also shown. At this time, the part detection unit 21 detects the straight portion A, the joint P2, and the straight portion B as the types of parts. The abnormality detection unit 30 also detects an abnormality in the joint P2.

[0073] (Frame number 4) Straight portion A, joint P2, and straight portion B are shown, with joint P2 appearing even larger. The abnormal substance caused by joint P2 is also shown even larger. At this time, part detection unit 21 detects straight portion A, joint P2, and straight portion B as part types. Furthermore, anomaly detection unit 30 detects an abnormality in joint P2.

[0074] (Frame number 5) Straight portion A, joint P2, and straight portion B are shown, with joint P2 appearing even larger. The abnormal substance caused by joint P2 is also shown even larger. At this time, part detection unit 21 detects straight portion A, joint P2, and straight portion B as part types. Furthermore, anomaly detection unit 30 detects an abnormality in joint P2.

[0075] (Frame number 6) Straight portion B is visible, but joint portion P2 is not. However, abnormal material caused by joint portion P2 is visible. In this case, part detection unit 21 detects straight portion B as the type of part (however, depending on the settings in association processing unit 22 described later, anomaly detection unit 30 may detect anomaly not in straight portion B but in joint portion P2, or may detect anomaly in both straight portion A and joint portion P2). Anomaly detection unit 30 detects an abnormality in joint portion P2 (however, depending on the settings in association processing unit 22 described later, anomaly detection may be disabled, and anomaly detection unit 30 may not detect an abnormality).

[0076] (Frame number 7) The straight portion B is visible, but the joint P2 is not. The abnormal substance at the joint P2 is also not visible. At this time, the part detection unit 21 detects the straight portion B as the type of part. The abnormality detection unit 30 does not detect an abnormality at the joint P2.

[0077] Correlation processing unit 22 The following describes the details of the association processing unit 22. Note that since the processing of the association processing unit 22 also affects the processing of the anomaly detection unit 30, the anomaly detection unit 30 will also be described here.

[0078] As described above, the association processing unit 22 performs a process of associating each of the multiple frame images constituting the video with any of the parts of the pipe 3 based on the part detection results of the part detection unit 21. The association process exemplified here is a process of determining from which frame image an abnormality in each of the various parts should be detected.

[0079] (Processing when multiple parts are detected from a frame image (part 1)) In the example of FIG. 6, there are frame images with frame numbers 3, 4, and 5 from which multiple parts have been detected (hereinafter, "specific frame images").

[0080] In this way, when there is a specific frame image in which multiple parts are detected from one frame image, the association processing unit 22 selects one part with the highest predetermined priority from the multiple parts, and associates the selected part with the specific frame image. Here, it is assumed that the part with the highest priority is the "junction." In this case, in the example of FIG. 6, the junction 2 will be associated with the frame images of frame numbers 3, 4, and 5. That is, an abnormality shown in the frame images of frame numbers 3, 4, and 5 will be detected as an abnormality in the junction 2.

[0081] In addition, the matching processing unit 22 has the function of selectively implementing one of the various matching processing settings shown below when there is another frame image (frame images with frame numbers 2 and 6 in the example of Figure 6) in which a part different from the part associated with the specific frame image is detected immediately before or after the specific frame image.

[0082] i) Setting to maintain the part associated with the different frame image without changing it (Pattern 1-1-1 <priority order>) ii) Setting to disable anomaly detection for the above other frame images (Pattern 1-1-2 <before and after disabled>) iii) Setting to change the part associated with the different frame image to the same part associated with the specific frame image (Pattern 1-1-3 <Front and rear expansion>) iv) A setting for changing the part associated with the different frame image to both the same part associated with the specific frame image and the part associated with the different frame image (Pattern 1-1-4 <Overlap>).

[0083] The operator can instruct which setting to select through a predetermined operation.

[0084] Fig. 7 shows examples of settings for various association processes based on the example of the frame image in Fig. 6. Fig. 8 shows an example of anomaly detection results indicating the presence or absence of anomalies for each part based on the example of settings in Fig. 7.

[0085] In the example of Figure 7, for each frame image of frame numbers 1, 2, 3, 4, 5, 6, and 7 (images corresponding to camera positions of 0.0 m, 1.0 m, 1.5 m, 1.8 m, 2.1 m, 2.5 m, and 3.0 m from the inspection starting point, respectively), specific examples of four types of settings (patterns) are shown, which correspond to the above-mentioned pattern 1-1-1 <priority order>, pattern 1-1-2 <disable front and rear>, pattern 1-1-3 <extend front and rear>, and pattern 1-1-4 <overlap>.

[0086] In pattern 1-1-1 <priority order> in Fig. 7, straight portion A is associated with frame numbers 1 and 2, joint portion P2 is associated with frame numbers 3, 4, and 5, and straight portion A is associated with frame numbers 6 and 7. An abnormality in the video is detected from each of the frame images of frame number 2 to which straight portion A is associated, frame numbers 3, 4, and 5 to which joint portion P2 is associated, and frame number 6 to which straight portion A is associated. Therefore, in pattern 1-1-1 <priority order> in Fig. 8, the abnormality detection results obtained are that straight portion A is abnormal, joint portion P2 is abnormal, and straight portion B is abnormal.

[0087] Pattern 1-1-1 (priority order) is an effective setting for those who wish to know exactly whether or not there are any abnormalities detected in each part.

[0088] Pattern 1-1-2 <front and rear disabled> in Fig. 7 differs from pattern 1-1-1 <priority order> in Fig. 7 in that abnormality detection is disabled for frame numbers 2 and 6. Therefore, in pattern 1-1-2 <front and rear disabled> in Fig. 8, the abnormality detection results obtained are that straight portion A is normal, joint portion P2 is abnormal, and straight portion B is normal.

[0089] Pattern 1-1-2 (before and after disabled) is an effective setting for those who wish to know only the part that is the source of the abnormality.

[0090] Pattern 1-1-3 <front-rear expansion> in Fig. 7 differs from pattern 1-1-1 <priority order> in Fig. 7 in that straight sections A and B corresponding to frame numbers 2 and 6 are changed to joint section P2, respectively. Therefore, in pattern 1-1-3 <front-rear expansion> in Fig. 8, the abnormality detection result obtained is that straight section A is normal, joint section P2 is abnormal, and straight section B is normal.

[0091] Pattern 1-1-3 (front-rear expansion) is an effective setting for those who want to know not only the area where the abnormality originated but also all areas that are affected by the abnormality when the abnormality spreads from the source to other areas.

[0092] Pattern 1-1-4<overlap> in Fig. 7 differs from pattern 1-1-1<priority order> in Fig. 7 in that not only the straight portion A associated with frame number 2 has been changed to both straight portion A and joint P2, and not only the straight portion B associated with frame number 6 has been changed to both straight portion B and joint P2. Therefore, in pattern 1-1-4<overlap> in Fig. 8, the anomaly detection results obtained are that straight portion A is abnormal, joint P2 is abnormal, and straight portion B is abnormal.

[0093] Pattern 1-1-4 <overlap> is an effective setting for those who wish to know not only the part where the abnormality originated, but also the abnormalities in other parts when the abnormality has spread from the source.

[0094] (Processing when multiple parts are detected from a frame image (part 2)) Next, another example of processing when a plurality of parts are detected from a frame image will be described.

[0095] The method described below is an effective method when multiple parts are detected from a frame image and it is desired to obtain abnormality detection results for each part more precisely than the methods shown in FIGS.

[0096] In this example, the part detection unit 21 further has the function of detecting the types of the multiple parts when multiple parts are captured within a single frame image, and generating an area map showing the area of ​​each part detected within the frame image.

[0097] The correspondence processing unit 22 further has a function of, when there is a specific frame image in which multiple parts are detected from one frame image, using an area map to set multiple partial areas within the specific frame image corresponding to each of the multiple parts.

[0098] The anomaly detection unit 30 further has a function of detecting anomalies in each part of the pipe from the plurality of partial regions. The anomaly detection of each detection target is performed, for example, based on characteristics such as brightness, color, shape, and edge by image processing. In addition, the accuracy of anomaly detection may be improved by having AI learn these characteristics.

[0099] An example of the region map is shown in Fig. 9. In the region map of Fig. 9, a plurality of portions such as a straight portion A, a joint portion P2, and a straight portion B are displayed.

[0100] The partial regions of the straight portion A, the joint P2, and the straight portion B may be set with the boundaries X and Y as the boundaries, or the regions of the straight portion A, the joint P2, and the straight portion B may be set as they are as partial regions regardless of the boundaries X and Y. The boundary X is set on the front side of the joint P2 (the side closer to the camera), and the boundary Y is set on the back side of the joint P2 (the side farther from the camera).

[0101] Boundary X is set to have a certain width in front of joint P2. Similarly, boundary Y is set to have a certain width on the back side of joint P2. Each width is determined according to the imaging magnification of camera 4 and the range in which anomalous matter is generated that develops from the joint.

[0102] In the example of the area map in FIG. 9, a first abnormality detection result (abnormality detection (1)) is obtained from straight portion A (the side closer to joint P2). A second abnormality detection result (abnormality detection (2)) is obtained from joint P2. A third abnormality detection result (abnormality detection (3)) is obtained from straight portion B (the side closer to joint P2). A fourth abnormality detection result (abnormality detection (4)) is obtained from straight portion B (the side farther from joint P2).

[0103] (Various patterns related to subregions) The correspondence processing unit 22 has a function of using the area map to identify a specific part (e.g., joint P2) having the highest predetermined priority among the multiple parts detected from the specific frame image, and other parts (e.g., straight parts A and B) other than the specific part.

[0104] Furthermore, the association processing unit 22 has a function of selectively performing any one of the various association processing settings shown below.

[0105] i) A setting in which a partial area of ​​the specific part is the same area as the area in which the specific part is shown, and a partial area of ​​the other part is the same area as the area in which the other part is shown (Pattern 1-2-1 <Normal>) ii) A setting in which the partial area of ​​the specific part is the same area as the area of ​​the specific part, and the partial area of ​​the other part is reduced by a certain amount so that an area in which abnormality detection is disabled is formed between the partial area of ​​the specific part and the partial area of ​​the other part (Pattern 1-2-2 <periphery disabled>) iii) A setting in which a partial region of the specific part is a region in which the specific part is enlarged by a certain amount, and a partial region of the other part is a region in which the other part is reduced by a certain amount (Pattern 1-2-3 <peripheral expansion>) iv) A setting in which the partial area of ​​the specific part is enlarged by a certain amount from the area in which the specific part is shown so that part of the partial area of ​​the specific part overlaps with part of the partial area of ​​the other part, and the partial area of ​​the other part is set to the same area as the area in which the other part is shown (Pattern 1-2-4 <Overlap>).

[0106] The operator can instruct which setting to select through a predetermined operation.

[0107] Figures 10 to 13 show examples of settings for various kinds of association processing based on the example of the region map in Figure 9. Also, Figure 14 shows an example of anomaly detection results indicating the presence or absence of anomalies for each part based on the setting examples in Figures 10 to 13. Furthermore, Figures 15 to 18 show a method for determining the presence or absence of anomalies in each of the straight portion A, the joint portion P2, and the straight portion B based on the region map in Figure 9 using the four types of settings described in Figures 10 to 13. The shaded areas in each figure indicate areas where anomaly detection is invalid (invalid areas).

[0108] In the examples of Figures 10 to 13, specific examples of four types of settings (patterns) are shown that correspond to the above-mentioned pattern 1-2-1 <normal>, pattern 1-2-2 <periphery disabled>, pattern 1-2-3 <periphery expanded>, and pattern 1-2-4 <overlap>.

[0109] In pattern 1-2-1 <Normal> in Figure 10, boundaries X and Y are not used, the partial area of ​​joint P2 is set to the same area in which joint P2 is reflected, and the partial areas of straight sections A and B are set to the same area in which straight sections A and B are reflected, respectively.

[0110] 15, the partial area of ​​the joint P2 is set to the same area as the area where the joint P2 is reflected (area excluding the invalid area), and the presence or absence of an abnormality in the joint P2 is determined from the partial area. Also, the partial areas of the straight portions A and B are set to the same area as the area where the straight portions A and B are reflected (area excluding the invalid area), and the presence or absence of an abnormality in the straight portions A and B is determined from the partial areas.

[0111] Therefore, in pattern 1-2-1 <normal> in Figure 14, the following abnormality detection results are obtained: abnormality detection (1) from straight portion A, abnormality detection (2) from joint P2, and abnormality detection results (3) and (4) from straight portion B.

[0112] The pattern 1-2-1 (normal) setting is an effective setting for those who wish to know exactly whether or not there are any abnormalities detected in each area.

[0113] 11, the boundaries X and Y are used, and the partial areas of the straight portion A, the joint P2, and the straight portion B are set based on the boundaries X and Y. In this case, the partial areas of the straight portions A and B are set by reducing the areas in which the straight portions A and B are reflected by a certain amount by the boundaries X and Y, and the partial area of ​​the joint P2 is set to the same area as the area in which the joint P2 is reflected.

[0114] 16, the partial areas of the straight parts A and B are set to areas (areas excluding invalid areas) that are a fixed reduction of the areas in which the straight parts A and B are reflected by the boundaries X and Y, and the presence or absence of abnormalities in the straight parts A and B is determined from the partial areas. Also, the partial area of ​​the joint P2 is set to the same area (area excluding invalid areas) as the area in which the joint P2 is reflected, and the presence or absence of abnormalities in the joint P2 is determined from the partial area.

[0115] Therefore, in the pattern 1-2-2 <periphery invalid> in FIG. 14, the abnormality detection results obtained are normal from the straight portion A, abnormality detection (2) from the joint P2, and abnormality detection (4) from the straight portion B.

[0116] The pattern 1-2-2 (surrounding area invalid) setting is effective for those who wish to know only the area that is the source of the abnormality.

[0117] 12, the boundaries X and Y are used, and the partial regions of the straight portion A, the joint portion P2, and the straight portion B are set based on the boundaries X and Y. In this case, the joint portion P2 is set by the boundaries X and Y to have a certain amount of expansion of the region in which the joint portion P2 is reflected, and the partial regions of the straight portions A and B are set by the boundaries X and Y to have a certain amount of reduction of the region in which the straight portions A and B are reflected.

[0118] 17, the joint P2 is set as an area (excluding the invalid area) that is a certain amount larger than the area where the joint P2 is reflected by the boundaries X and Y, and the presence or absence of an abnormality in the joint P2 is determined from the partial area. Similarly, the partial areas of the straight portions A and B are set as areas (excluding the invalid area) that are a certain amount smaller than the area where the straight portions A and B are reflected by the boundaries X and Y, and the presence or absence of an abnormality in the straight portions A and B is determined from the partial area.

[0119] Therefore, in pattern 1-2-3 <peripheral expansion> in Figure 14, the abnormality detection results obtained are normal from straight portion A, abnormality detection results (1), (2), and (3) from joint P2, and abnormality detection (4) from straight portion B.

[0120] Pattern 1-2-3 (extended periphery) is an effective setting for determining not only the area where the anomaly originated but also all areas that are affected by the anomaly when the anomaly has spread from the source to other areas.

[0121] 13, the boundaries X and Y are used, and the partial areas of the straight portion A, the joint portion P2, and the straight portion B are set based on the boundaries X and Y. In this case, the partial area of ​​the joint portion P2 is set to a certain amount of expansion of the area in which the joint portion P2 is reflected by the boundaries X and Y, and the partial areas of the straight portions A and B are set to the same area as the area in which the straight portions A and B are reflected.

[0122] 18, the partial area of ​​the joint P2 is set to an area (excluding the invalid area) that is a certain amount larger than the area where the joint P2 is reflected by the boundaries X and Y, and the presence or absence of an abnormality in the joint P2 is determined from the partial area. Also, the partial areas of the straight portions A and B are set to the same area (excluding the invalid area) as the area where the straight portions A and B are reflected, and the presence or absence of an abnormality in the straight portions A and B is determined from the partial area.

[0123] Therefore, in pattern 1-2-4 <overlap> in Figure 14, the following abnormality detection results are obtained: abnormality detection (1) from straight portion A, abnormality detection results (1), (2), and (3) from joint P2, and abnormality detection results (3) and (4) from straight portion B.

[0124] Pattern 1-2-4 <overlap> is an effective setting for those who want to know not only the part where the abnormality originated, but also the abnormalities in other parts when the abnormality spreads from the source.

[0125] In this example, a case has been shown in which partial regions are set in the region map and then anomaly detection is performed from the partial regions, but alternatively, anomaly detection may be performed on the entire image frame and then anomaly detection results for each part may be obtained based on the region map.

[0126] Anomaly detection unit 30 The abnormality detection unit 30 will now be described in detail.

[0127] The abnormality detection unit 30 detects the presence or absence of an abnormality in each detection target for each portion of the pipe 3, and determines the level of the abnormality. The details thereof have been described above.

[0128] In this embodiment, the detection targets are assumed to be (1) rust, (2) deposits, (3) partial adhesion of foreign matter and deterioration of corrosion protection on the pipe surface, (4) film-like dirt adhering to the entire inside of the pipe, and (5) floating objects. The method of detecting each detection target will be described below.

[0129] (1) Chorus Rust occurs when the inner surface of a tube corrodes and rusts, and often appears reddish brown in color. Since the inner surface of a tube does not normally turn reddish brown, it is possible to detect rust by finding reddish brown areas through color detection. The level of abnormality of the chorus may be determined based on, for example, the shape of the inside of the pipe or the blockage rate from the video data.In addition to the method using the blockage rate as an index, other values ​​may be used as the index, such as the total area (number of pixels) of the chorus on the screen, the size of the chorus chunks by labeling, or the area of ​​the chorus in a predetermined area to be determined.

[0130] (2) Sediment In deposit detection, foreign matter accumulated at the bottom of the pipe is detected as deposit. The level of abnormality of the deposit may be determined based on, for example, the area or volume. In addition to the method using the area or volume as an index, the size of the area where the deposit exists or the number of lumps obtained by labeling may also be used as an index.

[0131] (3) Partial adhesion of foreign matter and deterioration of the corrosion protection of the pipe surface A normal water pipe has a uniform surface color, so there are no edge-like features. On the other hand, when abnormalities occur such as partial adhesion of foreign matter or deterioration of the corrosion protection of the pipe surface (hereinafter referred to as foreign matter adhesion and inner surface deterioration), edge-like features tend to appear in the image. Therefore, by using the edges detected by edge processing, it is possible to detect foreign matter adhesion and inner surface deterioration and to determine the level of abnormality.

[0132] (4) Dirt that has formed a film-like coating on the entire inside of the pipe It is difficult to detect film-like dirt that has adhered to the entire inside of a pipe (hereinafter referred to as film-like adhesions) using the method described in (3) above. Therefore, by calculating the average brightness and color of the inner surface of the pipe and calculating the difference from the normal state, it is possible to detect film-like adhesions and determine their abnormality level. The level of abnormality of the film-like deposit may be determined based on, for example, the amount of change in brightness from the normal state. In addition to the method using the amount of change in brightness as an index, the size of the line traced by the camera 4, for example, the thickness, length, area, or cumulative value of edge strength, may also be used as an index.

[0133] (5) Floating objects Floating objects are foreign objects contained in the water flowing inside the pipe, and are reflected in the form of optical flow. Floating objects can be detected from the optical flow. The abnormality level of the floating matter may be determined, for example, such that a state in which there are no floating matter is normal (eg, level S), and the abnormality level is increased (worsened) as the amount of floating matter increases.

[0134] Integrated Judgment Section 40 As described above, the integrated judgment unit 40 judges an abnormality rank (abnormality level) indicating the degree of abnormality for five types of detection targets for each part of the pipe 3 based on the abnormality detection results of the abnormality detection unit 30. The abnormality levels are classified into multiple stages (for example, S (normal) and five stages of A, B, C, and D) according to the degree of movement. For example, the integrated judgment unit 40 integrates the abnormality detection results generated by the abnormality detection unit 30 for each part of the pipe 3 by a predetermined process, and generates an integrated judgment result for each part of the pipe 3.

[0135] In general, anomaly detection results for each frame image may contain noise or sudden false positives. Here, we explain a method to reduce errors in integrated judgment based on noise and false positives, using three setting examples (patterns 1-3-1, 1-3-2, and 1-3-3).

[0136] Fig. 19 shows a specific example of a method for obtaining an integrated judgment result, and Fig. 20 shows another specific example of a method for obtaining an integrated judgment result.

[0137] Figure 19, similar to the case of Figure 4, shows a specific example of a method for obtaining an integrated judgment result for straight section A from information in which joint P1, straight section A, joint P2, straight section B, joint P3, straight section C, and joint P4 are arranged from left to right, and each part is associated with frame numbers 1 to 16.

[0138] Unlike Figure 19, Figure 20 shows a specific example of a method for obtaining an integrated judgment result for joint P2 from information associated with frame numbers 1 to 16 in a form in which portions of the areas of joint P1, straight section A, joint P2, straight section B, joint P3, straight section C, and joint P4 overlap.

[0139] For example, in the example of pattern 1-3-1 <mode> in Figure 19, from the perspective of removing noise to suppress false detections, the mode A1 of the values ​​indicating the abnormality level of each of the frame images of frame numbers 2, 3, and 4 (the frame images of frame numbers 2 to 5 excluding frame number 5) and the mode B1 of the values ​​indicating the abnormality level of each of the frame images of frame numbers 3, 4, and 5 (the frame images of frame numbers 2 to 5 excluding frame number 2) are obtained, and the worst value of the obtained modes A1 and B1 is determined as the integrated judgment result.

[0140] In addition, in the example of pattern 1-3-2 <average value> in Figure 19, from the perspective of removing noise to suppress false detections, an average value A2 of the values ​​indicating the abnormality level of each of the frame images of frame numbers 2, 3, and 4 (frame images of frame numbers 2 to 5 excluding frame number 5) and an average value B2 of the values ​​indicating the abnormality level of each of the frame images of frame numbers 3, 4, and 5 (frame images of frame numbers 2 to 5 excluding frame number 2) are calculated, and the worst value of the calculated average values ​​A2 and B2 is determined as the integrated judgment result.

[0141] In the example of pattern 1-3-3 <worst value> in FIG. 19, the worst value among the abnormality levels obtained from the frame image group of frame numbers 2, 3, 4, and 5 is set as the integrated judgment result.

[0142] In the example of pattern 1-3-1 <mode> in Figure 20, from the perspective of removing noise to suppress false detections, the mode A1 of the values ​​indicating the abnormality level of each of the frame images of frame numbers 4, 5, and 6 (the frame images of frame numbers 4 to 7 excluding frame number 7) and the mode B1 of the values ​​indicating the abnormality level of each of the frame images of frame numbers 5, 6, and 7 (the frame images of frame numbers 4 to 7 excluding frame number 4) are calculated, and the worst of the calculated modes A1 and B1 is determined as the integrated judgment result.

[0143] In addition, in the example of pattern 1-3-2 <average value> in Figure 20, from the perspective of removing noise to suppress false detections, an average value A2 of the values ​​indicating the abnormality level of each of the frame images of frame numbers 4, 5, and 6 (frame images of frame numbers 4 to 7 excluding frame number 7) and an average value B2 of the values ​​indicating the abnormality level of each of the frame images of frame numbers 5, 6, and 7 (frame images of frame numbers 4 to 7 excluding frame number 4) are calculated, and the worst value of the calculated average values ​​A2 and B2 is determined as the integrated judgment result.

[0144] In the example of pattern 1-3-3 <worst value> in FIG. 20, the worst value among the abnormality levels obtained from the frame image group of frame numbers 4, 5, 6, and 7 is set as the integrated judgment result.

[0145] That is, the integrated judgment unit 40 has a function of selectively implementing any one of the various settings for integrated judgment shown below.

[0146] i) For each part of the pipe 4, from among the corresponding frame image groups (for example, frame numbers 2, 3, 4, 5 in the example of FIG. 19), some frame images are selectively removed to select various frame image groups (for example, frame numbers 2, 3, 4 and frame numbers 3, 4, 5), and for each selected frame image group, the most frequent value (for example, A1, B1) of the value indicating the abnormality level is obtained, and the worst value among the obtained most frequent values ​​is set as the integrated judgment result (1-3-1). ii) For each part of the pipe 4, from among the corresponding frame image groups (e.g., frame numbers 2, 3, 4, 5), some frame images are selectively removed to select various frame image groups (e.g., frame numbers 2, 3, 4 and frame numbers 3, 4, 5), and for each selected frame image group, an average value (e.g., A2, B2) of values ​​indicating an abnormality level is calculated, and the worst value among the calculated average values ​​is set as the integrated judgment result (1-3-2). iii) Setting (1-3-3) that sets the worst abnormality level obtained from each of the frame images (eg, frame numbers 2, 3, 4, and 5) associated with each part of the pipe 4 as the integrated judgment result.

[0147] The operator can instruct which setting to select through a predetermined operation.

[0148] Note that preventing sudden false positives and detecting localized abnormalities are contradictory. Therefore, which of the three setting examples mentioned above (patterns 1-3-1, 1-3-2, and 1-3-3) should be adopted depends on how the results are used. For example, when selecting parts to be replaced, it is desirable to adopt pattern 1-3-1, which has fewer false positives and does not need to detect localized abnormalities that do not require immediate action. When inspecting the deterioration of a wide area, it is desirable to adopt pattern 1-3-2, which gives average results. When investigating localized abnormalities based on reports from water users, it is desirable to adopt pattern 1-3-3.

[0149] Output processing unit 50 As described above, the output processing unit 50 acquires the abnormality detection results of the abnormality detection unit 30 and the integrated judgment result of the integrated judgment unit 40 from the memory unit 41, accepts user operations, and outputs various information including the abnormality detection results for each part and the integrated judgment result to the display device in accordance with the user operations.

[0150] FIG. 21 shows an example of a screen that the output processing unit 50 causes the display device to display.

[0151] 21 displays an input image 61, an abnormality detection result 62, a bar 63 showing the abnormality judgment result for rust, a bar 64 showing the abnormality judgment result for deposits, a bar 65 for seeking the video, a marker 66 showing the current position of the video being displayed, an integrated judgment result 67 for the part at the current position of the rust, and an integrated judgment result 68 for the part at the current position of the deposits. Note that, in order to avoid complicating the illustration, the bars showing the abnormality judgment results for parts other than rust and deposits are omitted.

[0152] In this embodiment, a bar 70 is further displayed indicating the names of various parts in the image (straight portion 1, joint portion 1, straight portion 2, joint portion 2, straight portion 3). Information indicating the abnormality detection result of each part may be displayed near the part where each part name is displayed.

[0153] A bar 65 at the bottom of the screen labeled "Video" is used to seek through the video, and any frame image in the video can be confirmed by moving a marker 66. The name of the body part associated with the displayed frame image can be confirmed from the position of the marker 66 on a bar 70.

[0154] In this embodiment, pattern selection items 71, 72, and 73, and boundary selection items 74 and 75 are further provided.

[0155] Selection item 71 allows one of the above-mentioned patterns 1-1-1, 1-1-2, and 1-1-3 to be selected and set. Selection item 72 allows one of the above-mentioned patterns 1-2-1, 1-2-2, and 1-2-3 to be selected and set. Selection item 73 allows one of the above-mentioned patterns 1-3-1, 1-3-2, and 1-3-3 to be selected and set.

[0156] Selection item 74 makes it possible to select and set the position of the aforementioned boundary X. Selection item 75 makes it possible to select and set the position of the aforementioned boundary Y. Each value is set to 0 as the default value (reference value), and the aforementioned width can be adjusted by increasing or decreasing the value in the + or - direction.

[0157] Although not shown in the example in Figure 21, it is also possible to create play, pause, frame-by-frame buttons, etc. to play the video automatically.

[0158] The left side of bar 65 corresponds to the beginning of the image, and the right side corresponds to the last frame of the image, and above them, for each abnormality detection item, abnormal points in the image are displayed in correspondence with bars 63 and 64. The vertical stripes indicate the positions of images with abnormality level B, and the diagonal hatched positions indicate the positions of images with abnormality level D.

[0159] Input image 61 is the original image of the input image corresponding to the "current position of the video", and anomaly detection result 62 is an image obtained by superimposing the results detected by anomaly detection unit 30 and integrated judgment unit 40. In this example image, deposits accumulated at the bottom of a pipe have been detected, and in the anomaly detection result 62 image, the area of ​​the deposits is enclosed in a frame to clearly show where the abnormality is located.

[0160] In addition, the position and type of anomalies detected by the anomaly detection unit 30 and the integrated judgment unit 40 may be indicated by using symbols, rectangles, transparent color overlays, etc. on the image of the anomaly detection result 62 to make them easier to understand.

[0161] The integrated judgment results for the corresponding parts are displayed in the rust integrated judgment result 67 and the deposit integrated judgment result 68. In this example image, there is no abnormality in the rust, so the integrated judgment result is S (normal), and deposits are present, so the integrated judgment result is B.

[0162] Although each integrated judgment result is basically displayed as the worst value of each individual abnormality level, it is not limited to this, and the output contents on the screen may adopt an average value or a specific abnormality, etc. In addition, attribute information input by the attribute information input unit 11 may be displayed together.

[0163] (operation) Next, an example of the operation of the inside of a pipe inspection device according to the embodiment will be described with reference to the flowchart in Fig. 22. Note that the individual processes shown in Fig. 22 are not necessarily performed in the order of the step numbers, and can be changed as appropriate.

[0164] The video input unit 10 inputs video data captured by the camera 4 while moving inside the water pipe 3, and the attribute information input unit 11 inputs attribute information indicating attributes associated with the camera 4 (step S1). At the same time, the camera position detection unit 20 uses the video data input by the video input unit 10 and the attribute information input by the attribute information input unit 11 to detect the position of the camera 4 inside the pipe.

[0165] Next, the type of part of the tube 3 shown in each of the plurality of frame images constituting the video input by the video input unit 10 is detected (step S2).

[0166] Next, the association processing unit 22 performs a process of associating any one of the parts of the pipe 3 with each of the multiple frame images constituting the video, based on the part detection result by the part detection unit 21 (step S3).

[0167] Next, based on the matching processing results of the matching processing unit 22, the abnormality detection unit 30 detects abnormalities for each part of the pipe 3 from the multiple frame images that make up the video, determines the presence or absence of abnormality and the abnormality level for each detection target, and generates the determined results as abnormality detection results for each part (step S4).

[0168] Next, the integrated determination unit 40 integrates the abnormality detection results generated for each part by the abnormality detection unit 30, and generates an integrated determination result indicating the level of abnormality inside the pipe (step S5).

[0169] Next, the integrated judgment unit 40 stores and holds the integrated judgment result together with the abnormality detection result in the storage unit 41 (step S6).

[0170] Finally, the output processing unit 50 acquires the abnormality detection result of the abnormality detection unit 30 and the integrated judgment result of the integrated judgment unit 40 from the memory unit 41, accepts user operation, and outputs various information including the abnormality detection result for each part and the integrated judgment result to the display device in accordance with the user operation (step S7).

[0171] As described above in detail, according to the embodiment, it is possible to output the results of an intraductal inspection for each site. [Explanation of symbols]

[0172] 1...pipe inspection device, 2...site, 3...water pipe, 4...camera, 5...recording medium, 10...video input section, 11...attribute information input section, 12...setting section, 20...camera position detection section, 21...part detection section, 22...corresponding processing section, 30...anomaly detection section, 40...integrated judgment section, 41...memory section, 42...management section, 50...output processing section.

Claims

1. and an output processing unit that displays on a display device information indicating the degree of abnormality for each part of the pipe based on images captured by a camera moving inside the pipe. Pipe inspection equipment.

2. A part detection unit detects the type of part of the pipe shown in each of the plurality of frame images constituting the video. The pipe inspection device according to claim 1 .

3. a matching processing unit that matches each of the plurality of frame images with any one of the parts of the pipe based on a part detection result of the part detection unit; an anomaly detection unit that detects an anomaly for each part of the pipe from the plurality of frame images based on a result of the association process by the association processing unit; Further comprising: The pipe inspection device according to claim 2.

4. Further, an integrated determination unit is provided that determines the degree of abnormality for each part of the pipe based on the abnormality detection result of the abnormality detection unit. The pipe inspection device according to claim 3.

5. The association processing unit When a specific frame image exists in which a plurality of body parts are detected from one frame image, a single body part having a highest predetermined priority is selected from the plurality of body parts, and the selected body part is associated with the specific frame image. The pipe inspection device according to claim 3.

6. The association processing unit When there is another frame image in which a part different from the part associated with the specific frame image is detected immediately before or after the specific frame image, i) a setting for maintaining the part associated with the different frame image without changing it; ii) a setting for disabling anomaly detection for the other frame image; iii) changing a part corresponding to the different frame image to the same part corresponding to the specific frame image; iv) changing a region associated with the different frame image to both the same region associated with the specific frame image and a region associated with the different frame image; It is possible to selectively implement any one of the following: The pipe inspection device according to claim 5.

7. The association processing unit When a specific frame image in which a plurality of body parts are detected exists, a plurality of partial regions corresponding to the plurality of body parts are set in the specific frame image, The anomaly detection unit is Detecting an abnormality in each part of the pipe from the plurality of partial regions. The pipe inspection device according to claim 3.

8. The association processing unit Identifying a specific part having a highest predetermined priority among the plurality of parts detected from the specific frame image and another part other than the specific part; i) setting a partial area of ​​the specific part to be the same area as an area in which the specific part is shown, and a partial area of ​​the other part to be the same area as an area in which the other part is shown; ii) A setting in which the partial area of ​​the specific part and the partial area of ​​the other part are reduced by a certain amount compared to an area in which the other part is shown, and the area in which the specific part is shown is the same area as the area in which the specific part is shown, so that an area in which abnormality detection is disabled is formed between the partial area of ​​the specific part and the partial area of ​​the other part; iii) setting a partial area of ​​the specific part to be a certain amount of enlargement of an area in which the specific part is shown, and a partial area of ​​the other part to be a certain amount of reduction of an area in which the other part is shown; iv) setting the partial region of the specific part to be a region obtained by enlarging the region in which the specific part is shown by a certain amount so that a part of the partial region of the specific part overlaps with a part of the partial region of the other part, and setting the partial region of the other part to be the same region as the region in which the other part is shown; It is possible to selectively implement any one of the following: The pipe inspection device according to claim 7.

9. The integrated determination unit is i) selecting various frame image groups obtained by selectively removing some frame images from the frame image groups associated with each part of the pipe, determining the most frequent value of the value indicating the abnormality level for each selected frame image group, and setting the worst value of each of the determined most frequent values ​​as the integrated judgment result; ii) for each part of the pipe, selecting various frame image groups obtained by selectively removing some frame images from the corresponding frame image groups, calculating an average value of values ​​indicating an abnormality level for each selected frame image group, and setting the worst value among the calculated average values ​​as an integrated judgment result; iii) setting the worst value among the abnormality levels obtained from the corresponding frame images for each part of the pipe as the integrated judgment result; It is possible to selectively implement any one of the following: The pipe inspection device according to claim 4.

10. Based on the images captured by the camera moving inside the pipe, the output processing unit displays information indicating the abnormality status for each part of the pipe on the display device. An in-pipe inspection method comprising:

11. On one or more computers, A function that displays information on abnormal conditions for each part of the pipe on a display device based on images taken by a camera moving inside the pipe. A program to achieve this.

Images