In-tube inspection device, in-tube inspection method, and program

Abstract

To enable an appropriate result to be easily acquired even the case that contaminants exist in a tube.SOLUTION: According to an embodiment, an in-tube inspection device includes a contaminant detection part for detecting contaminants in a tube on the basis of a video obtained by performing photographing by a camera moving in the tube, and generating information showing at least an amount of the contaminants, an abnormality detection part for detecting abnormality of the tube on the basis of the video to output its abnormality detection result, and outputting information showing a degree of reliability representing the goodness of visibility in the tube determined according to the amount of contaminants, and an integration determination part for outputting a determination result showing a degree of abnormality of the tube on the basis of the abnormality detection result and the degree of reliability.SELECTED DRAWING: Figure 7

Description

[Technical Field]

[0001] An embodiment of the present invention relates to an inside-pipe inspection device, an inside-pipe inspection method, and a program. [Background technology]

[0002] When sediments and rust accumulate inside pipes such as water pipes after many years of use, foreign matter accumulates and adheres, and the inner surface of the pipe deteriorates and peels off, causing various abnormalities inside the pipe.

[0003] To inspect the inside of a pipe such as a water pipe, for example, a camera is inserted into the pipe and the camera is moved around inside the pipe, stopping at various points along the way to capture images of the entire pipe. Based on the images obtained, it is possible to determine whether there are any abnormalities in the pipe and the extent of the abnormalities. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Laid-Open No. 2006-114962 [Patent Document 2] Japanese Patent Application Publication No. 2017-054024 [Patent Document 3] Japanese Patent Application Laid-Open No. 2016-161597 Summary of the Invention [Problem to be solved by the invention]

[0005] When conducting inspections based on images taken with a camera inside a pipe such as a water pipe, the following factors can obstruct the view, making it difficult to accurately grasp the extent of any abnormalities in the pipe and resulting in difficult to obtain accurate inspection results.

[0006] - Impact of impurities (floating matter) moving along with the water flow inside the pipe The effect of impurities (sediments) accumulated at the bottom of the pipe being blown up by the movement of the camera The problem to be solved by the invention is to provide an in-pipe inspection device, an in-pipe inspection method, and a program that can easily obtain appropriate inspection results even when impurities are present in the pipe. [Means for solving the problem]

[0007] The pipe inspection device of the embodiment includes an impurity detection unit that detects impurities inside the pipe based on images captured by a camera moving inside the pipe and generates information indicating at least the amount of the impurities; an abnormality detection unit that detects abnormalities in the pipe based on the images and outputs the abnormality detection results, as well as information indicating a reliability that represents the quality of visibility inside the pipe, which is determined according to the amount of the impurities; and an integrated judgment unit that outputs a judgment result that indicates the degree of abnormality in the pipe based on the abnormality detection result and the reliability. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram showing an example of the overall configuration of a system including an inside-pipe inspection device according to an embodiment. [Figure 2] FIG. 2 is a diagram showing an example of how a camera is inserted into a water pipe to capture images of the inside of the pipe. [Figure 3] FIG. 3 is a diagram showing the basic structure of a water pipe and examples of abnormalities and impurities that may exist inside the pipe. [Figure 4] FIG. 4 shows examples of two types of suspended matter flowing inside a pipe. [Figure 5] FIG. 5 is a diagram conceptually showing the movement of floating objects shown in the video. [Figure 6] FIG. 6 is a conceptual diagram showing the movement of the floating sediment seen on the video. [Figure 7] FIG. 7 is a diagram showing an example of the configuration of an inside-pipe inspection device. [Figure 8] FIG. 8 is a diagram showing an example of a process for generating impurity distribution information from a frame image in which impurities are captured. [Figure 9] FIG. 9 is a diagram showing an example of a calculation procedure for determining a motion vector. [Figure 10] FIG. 10 is a diagram showing the concept of a method for determining a motion vector from the correlation coefficient shown in FIG. [Figure 11] FIG. 11 is a diagram showing the difference in calculation results between when the center coordinates and peak coordinates shown in FIG. 10 match and when they do not. [Figure 12] FIG. 12 is a diagram showing an example of a technique for detecting the motion vectors of impurities appearing in a plurality of predetermined detection regions on a video screen. [Figure 13] FIG. 13 is a diagram showing an example of a table used to determine the type of impurity based on the motion vector detected from each of the plurality of detection areas in FIG. [Figure 14] FIG. 14 shows some specific examples of anomaly detection processing methods. [Figure 15] FIG. 15 is a diagram showing an example of the operation of the anomaly detection unit to perform different processes depending on the amount and type of impurities. [Figure 16] FIG. 16 is a diagram illustrating an example of information including an anomaly detection result and a reliability obtained by the integrated determination unit from the anomaly detection unit. [Figure 17] FIG. 17 is a diagram showing an example of a processing procedure for obtaining an integrated determination result for each detection target using the information shown in FIG. [Figure 18] FIG. 18 is a diagram showing an example of a table showing the maximum value of the detection result of each detection target for each reliability level. [Figure 19] FIG. 19 is a diagram showing an example of a screen that the output processing unit causes the display device to display. [Figure 20] FIG. 20 is a diagram showing an example of a screen when a frame image showing flying deposits is displayed. [Figure 21] FIG. 21 is a flowchart showing an example of the overall operation of the inside of a pipe inspection device. DETAILED DESCRIPTION OF THE INVENTION

[0009] Hereinafter, embodiments will be described with reference to the drawings.

[0010] <System configuration> FIG. 1 shows an example of the overall configuration of a system including an inside-pipe inspection device according to an embodiment.

[0011] In this embodiment, the explanation will be given assuming that an inspection is being performed inside a water pipe (a drinking water pipe) as an example, but this embodiment is not limited to this example and can also be applied to the inspection of the inside of various types of pipes, including sewer pipes, for example.

[0012] The inside-pipe inspection device 1 according to this embodiment inspects the inside of a water pipe 3 installed at a work site 2, for example, as shown in FIG. 1 , based on images captured by a camera 4 that captures images while moving inside the pipe. The images captured by the camera 4 are imported into an information terminal (for example, a personal computer, a smartphone, or a tablet). The images are, for example, moving images made up of a plurality of frame images (also simply referred to as "frames"). The information terminal may also store attribute information indicating attributes associated with the camera 4. The captured images (i.e., moving images) are recorded on a recording medium 5 as video data (files) made up of, for example, a plurality of consecutive frame images, or are uploaded and stored in the memory of a specified server on a network (on the cloud). The captured images may also be supplied to the inside-pipe inspection device 1 in real time simultaneously with being recorded on the recording medium 5 or uploaded to the specified server.

[0013] The inside-pipe inspection device 1 imports video data recorded on the recording medium 5, or imports video data stored in the memory of a specified server, or imports video data from the camera 4 in real time, and inspects the inside of the pipe using the video data. The attribute information may be imported into the inside-pipe inspection device 1 together with the video data via the recording medium 5 or server, or may be imported into the inside-pipe inspection device 1 via a route different from the video data, without going through the recording medium 5 or server.

[0014] FIG. 2 shows an example of how a camera 4 is inserted into a water pipe 3 to take pictures of the inside of the pipe.

[0015] For example, as shown in Figure 2, a camera 4 attached to the end of a cable 6 is inserted through an insertion port 7 of a water pipe 3 installed at a work site 2, and the camera takes pictures while moving inside the pipe. In this case, the position of the maximum reach within the pipe is determined by the constraints of the length of the cable 6 attached to the camera 4. During filming, the camera 4 is advanced in the outward direction by pushing out the cable 6, and after reaching the maximum reach 9, the camera 4 is retreated in the return direction by pulling back the cable 6. If the camera 4 hits an obstacle within the pipe and becomes unable to proceed, the maximum reach may be determined regardless of the length of the cable 6.

[0016] The video captured by camera 4 is a series of multiple frame images, and the smallest inspection unit is a frame image. If the video output from camera 4 is in interlaced format, it is also possible to inspect in field image units, or in frame image units after deinterlacing.

[0017] However, depending on the purpose of the inspection, inspection on a frame image basis may provide inappropriate results due to being too detailed. For example, if you want to know when it's time to renew equipment, it is appropriate to inspect the water pipe section by section to obtain inspection results. Types of sections include straight sections, joints, perforated sections, T-shaped sections, and bends. A straight section is a section that extends linearly in the longitudinal direction of the pipe. Individual straight sections are connected by irregularly shaped pipe sections (joints, T-shaped sections, or bends). A joint joins, for example, two straight sections together. A perforated section is a thin section of pipe that branches off in either direction midway through the pipe. A T-shaped section is a section where the pipe branches into two. A bend is a curved section of the pipe.

[0018] Figure 3 shows the basic structure of a water pipe 3 and examples of abnormalities and impurities that may exist inside. Figure 3(a) is a view of the water pipe 3 seen in its longitudinal direction, and Figure 3(b) is a perspective view showing an overview of the water pipe 3.

[0019] The water pipe 3 is a pipe (for example, a ductile cast iron pipe) with a seal coat C applied to the inner surface of the pipe, and has a structure in which multiple straight sections A, B, ... are joined at joints P. Abnormalities such as rust R can occur in the water pipe 3. Impurities can also be present inside the pipe. Impurities include floating matter F that moves with the water flow inside the pipe, and sediment D that has accumulated at the bottom of the pipe and has been stirred up (hereinafter referred to as "floating sediment DF"). Depending on the amount of impurities, they can reduce the visibility of the inside of the pipe in the video, making it difficult to correctly identify abnormalities in the water pipe 3.

[0020] Floating matter F can be divided into two types depending on the direction of flow inside the pipe. Figure 4 shows examples of two types of floating matter F flowing inside the pipe.

[0021] The water inside the water pipe 3 does not always flow in a fixed direction, but may reverse direction or stop flowing over time. Therefore, the movement of the floating matter F also changes depending on the flow of the water. Here, the direction of the floating matter F moving toward the camera 4 is referred to as the "camera direction 101," and the direction of the floating matter F moving away from the camera 4 is referred to as the "opposite direction 102." In order to correctly understand the movement of the floating matter F itself through the image from the camera 4, it is important to check the direction in which the floating matter F is moving in the image as well as the direction in which the camera 4 is moving.

[0022] Figure 5 conceptually shows the movement of floating matter F as seen in the video. Figure 6 conceptually shows the movement of floating sediment DF as seen in the video.

[0023] When there is a flow of water in the water pipe 3, the floating matter F moves along with the flow of water, as shown in FIG.

[0024] On the other hand, when the camera 4 comes into contact with the bottom of the pipe of the water pipe 3, for example, the sediment that has settled at the bottom is stirred up, as shown in FIG. 6, and the stirred up impurities DF are generated.

[0025] Sediment DF initially moves from bottom to top. It then mixes uniformly, moves from top to bottom, and then falls back to the bottom. If there is a water flow when sediment DF occurs, the two phenomena of water flow and sediment DF will occur simultaneously, and the sediment DF will exhibit complex movements.

[0026] <Configuration of the pipe inspection device 1> FIG. 7 shows an example of the configuration of the inside of the pipe inspection device 1.

[0027] 7 includes, as various functions, a video input unit 10, an attribute information input unit 11, a camera position movement detection unit 20, an impurity detection unit 21, an abnormality detection unit 30, an integrated judgment unit 40, a storage unit 41, a management unit 42, and an output processing unit 50. Some or all of these functions are realized as a program to be executed by one or more computers.

[0028] 7 is merely an example and is not limited to this example. For example, the inside of the pipe inspection device 1 is not limited to one device, but may be configured by multiple devices, and these multiple devices may be located in different locations. Furthermore, the various functions described above may also be provided in different devices and located in different locations.

[0029] For example, the video input unit 10, attribute information input unit 11, camera position movement detection unit 20, impurity detection unit 21, anomaly detection unit 30, integrated determination unit 40, storage unit 41, and management unit 42 may be provided in a predetermined server, and the output processing unit 50 may be provided in an arbitrary information terminal (for example, a personal computer, a smartphone, or a tablet). In this case, the output processing unit 50 may be provided in advance to the arbitrary information terminal from the predetermined server as a program that realizes a UI (User Interface).

[0030] The video input unit 10 inputs video data (hereinafter sometimes abbreviated as "video") captured by a camera 4 that captures the inside of a water pipe 3.

[0031] The attribute information input unit 11 inputs attribute information of the camera 4. The attribute information is supplied to the camera position movement detection unit 20, and also supplied to the impurity detection unit 21, the anomaly detection unit 30, and the integrated judgment unit 40 via the management unit 42, and is used as necessary in the processing of each unit.

[0032] The attribute information input by the attribute information input unit 11 includes, for example, information indicating the length of the cable 6 when the camera 4 is inserted into the water pipe 3 (cable length), which serves as position information for the camera 4 within the pipe. The information indicating the cable length can be used for detecting the camera position in the camera position movement detection unit 20 or for display on the screen in the output processing unit 50 by recording it superimposed at a predetermined position in the video data.

[0033] The camera position and movement detection unit 20 detects the position and movement of the camera 4 within the pipe using the video input by the video input unit 10 and the attribute information input by the attribute information input unit 11. The position information indicating the position of the camera 4 and the movement information indicating the movement of the camera 4 are supplied to the impurity detection unit 21, the abnormality detection unit 30, the integrated judgment unit 40, etc. via the management unit 42 and are used as necessary in the processing of each unit.

[0034] The position of camera 4 can be detected, for example, by reading information indicating the cable length (i.e., position information of camera 4) from the attribute information input by attribute information input unit 11. Alternatively, information indicating the cable length may be embedded in the video data in association with each frame, and the value may be read. The movement of camera 4 can be determined, for example, from the optical flow of an object shown in the video. The movement of camera 4 can be determined by detecting the optical flow over the entire image, finding the vanishing point from the direction of the flow, and calculating a flow vector with the vanishing point as its starting point or end point.

[0035] The impurity detection unit 21 detects impurities in the pipe (including determining whether or not there are impurities, determining the amount and distribution of impurities, determining the movement of impurities, determining the type of impurities, etc.) for each one or more frame images based on the video input by the video input unit 10, and generates various information related to the impurities.

[0036] The detection of impurities in the pipe may be performed using AI (Artificial Intelligence), which can determine the presence or absence of impurities, the type of impurities, and the amount and distribution of impurities based on the video.

[0037] The "amount of impurities" here may refer to the number of impurities in a unit area, or it may refer to the concentration of the impurities (concentration of impurities relative to water per unit volume). In this case, the AI ​​may be trained in advance to learn the relationship between the impurities shown in the frame image and their concentrations, allowing the concentration to be calculated with high accuracy.

[0038] The impurity detection unit 21 further has the following various functions.

[0039] For example, the impurity detection unit 21 has a function of generating impurity distribution information for each of one or more frame images, the impurity distribution information individually indicating a value corresponding to the amount of impurities appearing in each unit area of ​​the frame image. The impurity distribution information is sent to the anomaly detection unit 30.

[0040] Each unit area refers to an image area divided into a grid-like pattern at regular intervals in the frame image. The size of the image area can be changed as appropriate. The specific process for generating impurity distribution information will be described later.

[0041] The "value according to the amount of impurities" is expressed in multiple levels, for example, level 1, 2, or 3. In this case, the level increases from 1 to 3, indicating a greater amount of impurities. A unit area with no impurities may be expressed as 0.

[0042] The impurity distribution information may be generated as more simplified information. For example, instead of individually indicating a value (e.g., level 1, 2, or 3) corresponding to the amount of impurities appearing in each unit area of ​​the frame image as described above, the impurity distribution information may be generated as information simply indicating a value (e.g., level 1, 2, or 3) corresponding to the amount of impurities appearing in the entire frame image. Furthermore, the information generated in this manner may be sent to the anomaly detection unit 30 instead of the impurity distribution information.

[0043] The impurity detection unit 21 also has the function of detecting the movement of impurities based on multiple frame images, and further determining whether the impurities are floating deposits DF or floating matter F based on the movement, and outputting information indicating the type (impurity type information). The movement of the impurities is expressed, for example, by a motion vector indicating the direction and magnitude of the movement. The impurity type information is sent to the anomaly detection unit 30. The process of calculating the motion vector and the process of determining the type of impurities will be described in detail later.

[0044] The anomaly detection unit 30 detects anomalies in the water pipe 3 for each one or more frame images based on the video input by the video input unit 10 (including implementing an appropriate anomaly detection processing method according to the type and amount of impurities, determining the presence or absence of anomalies, and determining the level of anomaly), and outputs the anomaly detection result as well as information indicating the reliability of visibility within the pipe, which is determined according to the amount of impurities (for example, based on impurity distribution information). In other words, the anomaly detection unit 30 outputs the anomaly detection result and information indicating the reliability for each one or more frame images. The anomaly detection result and information indicating the reliability are sent to the integrated judgment unit 40.

[0045] The anomaly detection results are expressed as a level indicating the degree of anomaly for each detection target. The level is expressed as one of the values, for example, 0, 1, 2, 3, or 4. In this case, a level of 0 indicates no anomaly, and levels 1 through 4 indicate increasing anomaly levels.

[0046] The detection targets referred to here are five types: (1) rust, (2) deposits, (3) floating matter, (4) partial adhesion of foreign matter and deterioration of corrosion protection on the pipe surface, and (5) dirt that has adhered to the entire inside of the pipe in the form of a film. In the following, for ease of understanding, the detection targets will be explained as three types: (1) rust, (2) deposits, and (3) floating matter.

[0047] The reliability is expressed as a value between 0 and 100, for example. As the amount of impurities increases and visibility inside the pipe becomes poorer, the visibility of the anomaly detection result also becomes poorer, and the reliability decreases. A reliability of 0 means that the amount of impurities exceeds the reference value, making it impossible to determine the presence or absence of an anomaly or to determine the level of an anomaly, and corresponds to a case where a large amount of sediment has been stirred up (a large amount of sediment that exceeds the reference value). For a frame image with a reliability of 0, an anomaly cannot be detected and the anomaly detection result is invalid. For a frame image that does not show any impurities, the reliability is 100.

[0048] When the reliability is determined based on the impurity distribution information, the reliability may be determined, for example, as a value (e.g., any value between 0 and 100) corresponding to the total of the level values ​​(e.g., 0, 1, 2, or 3) indicated in each unit area of ​​the impurity distribution information, or as a value corresponding to the total of the level values ​​indicated only in the unit areas in which an abnormality is detected.

[0049] The reliability may be determined using, for example, impurity distribution information as well as impurity type information. Since visibility inside the pipe may differ depending on the type of impurity, the reliability may be adjusted depending on the type of impurity.

[0050] The abnormality detection unit 30 further has the following various functions.

[0051] For example, the abnormality detection unit 30 has the function of determining the shooting position of each frame image based on each frame image of the video input by the video input unit 10 and information indicating the position and movement of the camera 4 sent from the camera position movement detection unit 20, detecting abnormalities for each detection target at each shooting position, and generating the results as abnormality detection results for each detection target.

[0052] Anomaly detection for each detection target can be performed, for example, by image processing based on features such as brightness, color, shape, and edges. The accuracy of anomaly detection can be improved by having AI learn these features.

[0053] Furthermore, the anomaly detection unit 30 has a function of changing the processing method for anomaly detection (selecting an appropriate processing method) depending on the type of impurity determined by the impurity detection unit 21. Specific examples of the processing method will be described later.

[0054] The integrated judgment unit 40 generates an integrated judgment result indicating the degree of abnormality of each detection target based on the abnormality detection result and reliability generated by the abnormality detection unit 30 for each one or more frame images for each part of the water pipe 3.

[0055] The integrated judgment result is expressed as a rank indicating the degree of abnormality of each detection target for each part of the water pipe 3. The rank is indicated by, for example, S, A, B, C, or D. In this case, the rank S indicates no abnormality, and the ranks from A to D indicate increasing degrees of abnormality.

[0056] The integrated judgment unit 40 further has the following functions.

[0057] For example, the integrated judgment unit 40 has a function of determining the integrated judgment result by giving priority to the worst abnormality detection result among the abnormality detection results obtained from a plurality of frame images for each part of the water pipe 3.

[0058] Specifically, when there is a frame image whose reliability is equal to or greater than a certain value, the integrated judgment unit 40 determines the integrated judgment result for that frame image using the worst anomaly detection result obtained from that frame image. On the other hand, when there is a frame image whose reliability is less than a certain value other than 0, the integrated judgment result for that frame image is determined using at least the worst anomaly detection result obtained from a frame image whose reliability is equal to or greater than the certain value. Frame images whose reliability is 0 are not used in the integrated judgment process. A specific example of the process for generating the integrated judgment result will be described later.

[0059] The storage unit 41 stores various information including video including individual frame images, attribute information, the presence or absence and type of impurities, abnormality detection results, reliability, and integrated judgment results.

[0060] The management unit 42 controls and manages each unit so that a series of processes by the attribute information input unit 11, camera position movement detection unit 20, impurity detection unit 21, anomaly detection unit 30, integrated judgment unit 40, and memory unit 41 are performed appropriately according to a predetermined procedure.

[0061] For example, the management unit 42 supplies the position information of the camera 4 obtained from the camera position / motion detection unit 20 or the attribute information obtained from the attribute information input unit 11 to the impurity detection unit 21, the anomaly detection unit 30, and the integrated judgment unit 40. The position information and attribute information of the camera 4 can be used to identify the frame image (identify the frame number). Therefore, each of the impurity detection unit 21, the anomaly detection unit 30, and the integrated judgment unit 40 can associate information related to each process with the frame number.

[0062] The output processing unit 50 has, as its main functions, a function of displaying on the display device various types of information that can be operated by the user, and a function of displaying on the display device various types of information according to the operation of the user.

[0063] The output processing unit 50 acquires various information from the memory unit 41, including video including individual frame images, attribute information, the presence or absence and type of impurities, abnormality detection results, reliability, integrated judgment results, etc., accepts user operations, and outputs various information to the display device in accordance with the user operations.

[0064] <Details of the processing of the main components> The processing of the main elements that make up the inside of the pipe inspection device 1 will be described in detail below.

[0065] Impurity detection unit 21 The processing performed by the impurity detection unit 21 will now be described in detail.

[0066] As described above, the impurity detection unit 21 has the function of generating impurity distribution information for each of one or more frame images, which individually indicates a value corresponding to the amount of impurities reflected in each unit area of ​​the frame image.

[0067] FIG. 8 shows an example of a process for generating impurity distribution information from a frame image showing impurities.

[0068] Fig. 8(a) shows an example of a frame image (frame image at a cable length of 100 m) showing the floating sediment DF as an impurity. Fig. 8(b) shows an example of impurity distribution information generated from the frame image shown in Fig. 8(a).

[0069] The frame image shown in Figure 8(a) includes areas where the sediment is blown up (DF) and areas where it is not. In areas where the sediment is blown up (DF), the concentration of impurities is often not uniform.

[0070] Therefore, as shown in Fig. 8(b), impurity distribution information indicating a value corresponding to the amount of impurities (here, one of levels 1, 2, or 3 indicating the level of concentration) is generated for each unit area of ​​a predetermined size for the frame image shown in Fig. 8(a). This makes it possible to grasp the degree of poor or good visibility inside the pipe corresponding to the distribution of impurities shown in the frame image for each unit area.

[0071] As described above, the impurity detection unit 21 has the function of detecting the movement of impurities based on a plurality of frame images, determining based on the movement whether the impurities are stirred-up sediments DF or floating matter F, and outputting impurity type information indicating the type. The movement of the impurities is expressed, for example, by a motion vector indicating the direction and magnitude of the movement.

[0072] Fig. 9 shows an example of a calculation procedure for determining a motion vector. Fig. 10 shows the concept of a method for determining a motion vector from the correlation coefficient shown in Fig. 9. Fig. 11 shows the difference in calculation results between when the center coordinates and peak coordinates shown in Fig. 10 match and when they do not. Note that the method for calculating a motion vector is not limited to the example shown here, and other methods may be adopted.

[0073] Known methods for determining the movement of objects in video include optical flow, block matching, and phase-only correlation. Here, we will show an example of a method using phase-only correlation to calculate the motion vector of an object that appears in a predetermined detection area (image area for detecting the movement of the object) on the video screen. Any number of detection areas can be set anywhere on the video screen.

[0074] As shown in Figure 9, the current frame image (current frame) and the frame image one frame before (previous frame) are used as input information to calculate a motion vector. Specifically, two-dimensional Fourier transform 111 and Fourier transform 112 are performed on the current frame and the previous frame, respectively. Complex conjugate 113 is performed on the result obtained by Fourier transform 111. Multiplication 114 of the result obtained by complex conjugate 113 and the result obtained by Fourier transform 112 is performed. An inverse Fourier transform 115 is performed on the result obtained by multiplication 114 (corresponding to a composite image), thereby obtaining information indicating a two-dimensional correlation coefficient (corresponding to a correlation intensity function). Furthermore, peak value calculation 116 is performed using a four-quadrant coordinate plane based on the information indicating this correlation coefficient, and a motion vector is obtained as shown below.

[0075] The maximum value (peak) of the correlation coefficient can be expressed on a four-quadrant coordinate plane. In this case, the position (peak coordinate) corresponding to the maximum value (peak) of the correlation coefficient is found on the four-quadrant coordinate plane as shown in Figure 10. From the positional relationship between this peak coordinate and the center coordinate of the coordinate plane, the vector of the correlation coefficient, i.e., the motion vector indicated by the arrow in Figure 10, can be found.

[0076] The coincidence of the center coordinate and the peak coordinate (or a difference of less than a certain amount) means that the impurity is not moving. Specifically, as shown in Fig. 11(a), when the peak coordinate is (0,0) with respect to the center coordinate (0,0), it can be seen that the position of the impurity shown in the current frame has not moved from the position of the impurity shown in the previous frame.

[0077] On the other hand, a mismatch between the center coordinates and the peak coordinates (a difference of a certain amount or more) means that the impurity has moved. Specifically, as shown in Figure 11(b), if the peak coordinates are (2,3) with respect to the center coordinates (0,0), it can be seen that the position of the impurity shown in the current frame has moved by 2 in the X direction and 3 in the Y direction from the position of the impurity shown in the previous frame. In this case, the motion vector is expressed as, for example, (2,3).

[0078] 9 shows an example in which a reliability is generated together with a motion vector. This reliability is set to a value corresponding to the height of the peak, for example. However, the reliability does not necessarily have to be generated on the impurity detection unit 21 side. Instead, it may be generated on the anomaly detection unit 30 side. Furthermore, the reliability may be determined based on impurity distribution information as described above.

[0079] Next, an example of a method for determining the type of impurities will be described with reference to FIGS.

[0080] Fig. 12 shows an example of a method for detecting the motion vectors of impurities appearing in a plurality of predetermined detection areas on a video screen. Fig. 13 shows an example of a table used to determine the type of impurity based on the motion vectors detected from each of the plurality of detection areas in Fig. 12.

[0081] As shown in Fig. 12, two-dimensional coordinates are predetermined for the video screen. In the example of Fig. 12, the origin of the two-dimensional coordinates is located in the upper left corner, and the X axis indicates positive values ​​in the rightward direction, and the Y axis indicates positive values ​​in the downward direction.

[0082] 12, three detection areas 103, 104, and 105 are predefined on the video screen. Detection area 103 is located to the left of the center of the screen, detection area 104 is located below the center of the screen, and detection area 105 is located to the right of the center of the screen.

[0083] From each detection area, the motion vector of the impurity can be detected from the current frame and the previous frame. Then, by determining whether each motion vector corresponds to "X<0," "X=0," or "X>0" on the X axis, and whether it corresponds to "Y<0," "Y=0," or "Y>0" on the Y axis, the type of impurity can be determined.

[0084] For example, as shown in Fig. 12, if the detection result (detected motion vector) of detection area 103 indicates "X<0", "Y=0", the detection result of detection area 104 indicates "X=0", "Y>0", and the detection result of detection area 105 indicates "X>0", "Y=0", it is understood that the impurity corresponds to "floating matter" and that the floating matter is moving in camera direction 101. In Fig. 13, the detection results of each detection area in this case and the determination result of the corresponding type are expressed in the column of "floating matter (camera direction 101)".

[0085] Furthermore, if the detection result of detection area 103 indicates "X>0" and "Y=0", the detection result of detection area 104 indicates "X=0" and "Y<0", and the detection result of detection area 105 indicates "X<0" and "Y=0", it can be seen that the impurities correspond to "floating matter" and that the floating matter is moving in the opposite direction 102.

[0086] Furthermore, if the detection results of detection area 103 and detection area 105 both indicate "X≠0", "Y<0" (upward direction) or "Y>0" (downward direction), it is understood that the impurities correspond to "floating deposits." In this case, detection area 104 is not considered because the movement of impurities changes depending on the situation.

[0087] When the camera 4 is moving, the motion component of the camera 4 is added to the detected motion vector of the processing result, so only the motion of the impurities is calculated by canceling out the motion component of the camera using the result of the camera position motion detection unit 20. Alternatively, the motion of the impurities may be calculated only when the camera 4 is stationary.

[0088] Anomaly detection unit 30 The details of the processing performed by the abnormality detection unit 30 will be described.

[0089] As described above, the anomaly detection unit 30 has a function of changing the processing method for anomaly detection (selecting an appropriate processing method) depending on the type of impurity determined by the impurity detection unit 21.

[0090] FIG. 14 shows some specific examples of anomaly detection processing methods.

[0091] FIG. 14 shows an example of a table showing various treatment methods, as well as their effectiveness for "floating matter (but only if it floats continuously)" and "floating sediment (but only if it floats temporarily)." Note that if the floating sediment is not temporary but is floating continuously (for example, if it exceeds the reference value), anomaly detection cannot be performed, and therefore it is not included here. Also, if the floating matter is not continuous but is floating temporarily, it is equivalent to a state where there is almost no floating matter, and therefore it is not included here.

[0092] In FIG. 14, "◯" indicates that it is valid. "△" indicates that it may be valid in some cases. "×" indicates that it is not valid (invalid).

[0093] The processing methods shown in FIG. 14 include the following.

[0094] (1) Select a frame image with minimal contamination This processing method detects abnormalities in frame images where the amount of impurities is below a certain level. It is effective when the amount of suspended impurities changes over a short period of time. It is effective when the amount of floating matter changes over time, but is ineffective when the amount is constant.

[0095] (2) Select an area with few impurities in one frame image. This processing method involves detecting anomalies in areas (one or more unit areas) where the amount of impurities is below a certain level. It is effective when the distribution of stirred-up impurities is uneven depending on the location. It is effective when the amount of floating matter varies depending on the area. It is not effective when the amount of floating matter is uniform across the entire screen.

[0096] (3) Weighting the detection results for each frame according to the amount of impurities This processing method involves weighting the anomaly detection results obtained for each frame image according to the amount of impurities. If there is little change in the amount of impurities over time or in the amount depending on the location, it is effective to weight the detection results for times and locations with less change in the amount.

[0097] (4) Detect after performing defogging process This processing method involves performing defogging (image processing to reduce the loss of visibility due to impurities) before detecting anomalies. If the amount of airborne impurities is not large, the loss of visibility due to impurities can be reduced by defogging. Specifically, a combination of techniques such as brightness and contrast correction and smoothing of multiple frames to remove particulate impurities is applied. However, if there are no impurities, defogging is not performed.

[0098] (5) When the camera is stopped, select a frame image with minimal interference. This processing method indicates that during the camera stopped period, anomaly detection is performed on frame images with less than a certain amount of impurities. During the camera stopped period, multiple identical detection results are obtained at that camera position (shooting position), so the detection result with the highest reliability (with the least amount of impurities) is used.

[0099] (6) Disable detection when there are many impurities This processing method indicates that abnormality detection is invalidated when the amount of impurities is above a certain level. When the amount of suspended matter is extremely large, abnormality detection cannot be performed, so abnormality detection is invalidated.

[0100] (7) Prioritize abnormality detection on the camera's outbound journey over its inbound journey. This processing method indicates that priority is given (e.g., weighting) to abnormality detection on the camera's outbound journey over abnormality detection on the inbound journey. If the camera causes foreign matter to fly up, the amount of foreign matter seen in the video on the outbound journey is relatively less than that on the inbound journey, so the results from the outbound journey are given priority.

[0101] Therefore, for "floating matter (but continuously floating matter)," more reliable detection results can be obtained by adopting a combination of the processing methods (3), (4), and (6) marked with a "○" or at least one of these. Also, depending on the situation, adopting a combination of processing methods that further includes (1) and (2) marked with a "△" or at least one of these increases the number of processing method options, thereby increasing the degree of freedom in selecting a processing method.

[0102] For "floating sediments (however, only temporary floating sediments)," more reliable detection results can be obtained by adopting a combination of the processing methods marked with "○" (1), (2), (3), (4), (5), and (7), or by adopting at least one of these.

[0103] 14 is an example, and is not limited to this example. For example, the combination of processing methods to be adopted may be changed as appropriate depending on the content of the impurity distribution information.

[0104] FIG. 15 shows an example of the operation in which the abnormality detection unit 30 performs different processes depending on the amount and type of impurities.

[0105] Here, an example of operation will be shown in which processing such as detecting an abnormality and determining reliability is performed for each frame for one portion of the water pipe 3.

[0106] In step S1, the anomaly detection unit 30 acquires a frame image used to detect an anomaly, impurity distribution information used to confirm the presence and amount of impurities and determine the reliability, impurity type information used to confirm the type of impurity, etc.

[0107] In step S2, the anomaly detection unit 30 checks whether or not there are any impurities based on the impurity distribution information. If there are no impurities, the process proceeds to step S3. If there are impurities, the process proceeds to step S4.

[0108] If there is no impurity (No in step S2), in step S3, the anomaly detection unit 30 sets the reliability to 100 and performs anomaly detection on the entire range of the frame image.

[0109] On the other hand, if impurities are present (Yes in step S2), in step S4, the abnormality detection unit 30 determines whether or not there is a large amount of impurities and whether or not this corresponds to the floating of sediment, based on the impurity distribution information and the impurity type information. If this corresponds, the process proceeds to step S5. If this does not correspond, the process proceeds to step S6.

[0110] If the amount of impurities is large and corresponds to the floating of sediments (Yes in step S4), in step S5, the abnormality detection unit 30 sets the reliability to 0 and determines not to detect an abnormality.

[0111] On the other hand, if the amount of impurities is large and does not correspond to the floating of sediments (No in step S4), in step S6, the abnormality detection unit 30 determines whether the type of impurities corresponds to "floating matter" or "floating of sediments" based on the impurity type information, and selects a processing method previously associated with the corresponding type. Here, if there is a predetermined processing (for example, "defogging processing") that should be executed before abnormality detection, that processing is executed first.

[0112] In step S7, the anomaly detection unit 30 determines whether the area in which the impurities exist is a part of the frame image or the entire frame image based on the impurity distribution information. If it is a part of the frame image, the process proceeds to step S8. If it is the entire frame image, the process proceeds to step S9.

[0113] If the area where the impurities exist is only part of the frame image (No in step S7), in step S8, the anomaly detection unit 30 performs anomaly detection on the area with few impurities. In this case, from among the processing methods selected in step S6, an anomaly detection method suitable for anomaly detection on an area with few impurities is used. In addition, the anomaly detection unit 30 determines a reliability (a value between 1 and 99) based on the impurity distribution information.

[0114] On the other hand, if the range in which impurities are present is the entire frame image (Yes in step S7), in step S9, the anomaly detection unit 30 performs anomaly detection on the entire range. In this case, from among the processing methods selected in step S6, an anomaly detection method suitable for anomaly detection on the entire range is used to perform the anomaly detection. In addition, the anomaly detection unit 30 determines a reliability (a value between 1 and 99) based on the impurity distribution information.

[0115] In step S10, the abnormality detection unit 30 determines whether to end the abnormality detection process for the relevant part depending on whether the part changes in the next frame image (for example, whether it changes to a different pipe). If the part does not change, the process does not end and proceeds to step S1, where the abnormality detection process is performed for the next frame image. If the part changes, the abnormality detection process for the relevant part ends.

[0116] Integrated Judgment Section 40 The processing of the integrated judgment unit 40 will now be described in detail.

[0117] As described above, the integrated judgment unit 40 generates an integrated judgment result indicating the degree of abnormality of each detection target based on the abnormality detection result and reliability generated by the abnormality detection unit 30 for each one or more frame images for each part of the water pipe 3.

[0118] 16 shows an example of information including an anomaly detection result and reliability that the integrated determination unit 40 acquires from the anomaly detection unit 30. Hereinafter, the anomaly detection result may be abbreviated as detection result.

[0119] The information shown in Fig. 16 shows an example of anomaly detection results and reliability for each frame image related to one part. Specifically, for each frame image, the frame number, camera position, reliability (a value between 0 and 100), and the detection result (a level of 0, 1, 2, 3, or 4) for each detection target (floating matter, rust, deposits) are shown. Note that information related to frame images with frame numbers 4 to 8 corresponds to the results when a large amount of deposits were blown up (amount exceeding the reference value), and indicates a reliability of "0," so this information will not be used.

[0120] Based on this information, the integrated judgment unit 40 obtains an integrated judgment result (rank of S, A, B, C, or D) for each detection target on a part-by-part basis. However, the integrated judgment result will differ between a frame image with a reliability above a certain level and a frame image with a reliability below a certain level (excluding 0).

[0121] It is desirable that the integrated judgment result indicates the worst result for each part. Therefore, as described above, the integrated judgment unit 40 has a function of determining the integrated judgment result by giving priority to the worst abnormality detection result among the abnormality detection results obtained from the multiple frame images for each part of the water pipe 3.

[0122] The following correspondence exists between the level (0, 1, 2, 3, 4) of the anomaly detection result and the rank (S, A, B, C, D) of the integrated judgment result.

[0123] 0→S 1→A 2→B 3→C 4→D Basically, the rank of the integrated judgment result is determined from the level of the worst anomaly detection result according to such a correspondence relationship. However, if there is a frame image with a reliability below a certain level, the rank of the integrated judgment result is not necessarily determined according to the correspondence relationship. The rank of the integrated judgment result may be determined based on the adjusted level.

[0124] FIG. 17 shows an example of a processing procedure for obtaining an integrated determination result for each detection target using the information shown in FIG.

[0125] In step S11, the integrated judgment unit 40 obtains the maximum value (worst level) of the detection results of each detection target for each reliability level based on the information shown in FIG. 16 (however, those with a reliability level of 0 are not included).

[0126] 16, when focusing on the reliability, the frame images can be classified into those with a reliability of "50" (frame numbers 1, 2, 3, 11, 12, ...) and those with a reliability of "25" (frame numbers 9, 10, ...). However, for ease of understanding, information from frame number 13 onwards will be ignored.

[0127] If we focus only on the frame images showing a reliability of "50," we can see that the maximum detection result for "floating objects" is "2," the maximum detection result for "rust" is "3," and the maximum detection result for "deposits" is "4."

[0128] If we focus only on the frame images showing a reliability of "25," we can see that the maximum detection result for "floating objects" is "4," the maximum detection result for "rust" is "0," and the maximum detection result for "deposits" is "0."

[0129] An example of a table summarizing these results is shown in Figure 18. The table in Figure 18 shows the maximum detection results for each detection target (floating particles, rust, and sediment) for each reliability level (for reliability levels of 50 and 25).

[0130] The classified reliability levels are not limited to "50" and "25" as shown in the table of Fig. 18, but may be changed as appropriate. For example, a certain range may be given to each of "50" and "25" and changed to "50 or more but less than 99" or "1 or more but less than 49", or the like, or the classification may be changed to a finer range than this.

[0131] Next, in steps S12, S13, and S14, the integrated determination unit 40 performs an integrated determination of each detection target (floating matter, rust, and deposits).

[0132] As mentioned above, it is desirable for the integrated judgment result to show the worst result for each part. Therefore, if, among the anomaly detection results of multiple frame images showing the part, the anomaly detection result shown by a frame image with a reliability above a certain level (for example, 50 or higher) shows the worst level (maximum value), the rank corresponding to that level is used as the integrated judgment result.

[0133] On the other hand, if the anomaly detection result indicated by a frame image with a reliability below a certain level (for example, less than 30) indicates the worst level, the reliability is too low and the rank corresponding to that level cannot be used as the integrated judgment result as is. In this case, the rank corresponding to the worst level indicated by a frame image with a reliability above a certain level (for example, 50 or more) is used as the integrated judgment result, or the rank corresponding to the average of the worst level indicated by a frame image with a reliability above a certain level (for example, 50 or more) and the worst level indicated by a frame image with a reliability below a certain level (for example, less than 30) is used as the integrated judgment result. In this case, the average of both may be changed to a weighted average, or the calculation method may be changed as appropriate.

[0134] In the example table of Figure 18, for a frame image with a reliability of "50," the maximum detection result for "floating matter" is "2," the maximum detection result for "rust" is "3," and the maximum detection result for "deposits" is "4." If the ranks corresponding to these levels are used as the integrated judgment results, the ranks of the integrated judgment results will be "B," "C," and "D," respectively.

[0135] On the other hand, in a frame image with a reliability of "25," the maximum value of the detection results for "floating objects" is "4," the maximum value of the detection results for "rust" is "0," and the maximum value of the detection results for "deposits" is "0." These levels of reliability are too low, so the ranks corresponding to these levels cannot be used as integrated judgment results. In this case, the maximum value of each detection result shown in a frame image with a reliability of "50" is used.

[0136] For example, by using the average or weighted average of both of the above, for "floating matter," the integrated assessment result is adopted as rank "C," which corresponds to level "3," obtained by averaging levels "2" and "4." For "rust," the integrated assessment result is adopted as rank "A" (or "B"), which corresponds to level "1" (or "2"), obtained by weighting levels "3" and "0," rather than using level "1.5" (a non-integer value) obtained by averaging levels "3" and "0." For "sediment," the integrated assessment result is adopted as rank "B," which corresponds to level "2," obtained by averaging levels "4" and "0."

[0137] Output processing unit 50 The processing of the output processing unit 50 will now be described in detail.

[0138] As described above, the output processing unit 50 acquires various information from the memory unit 41, including video including individual frame images, attribute information, the presence or absence and type of impurities, abnormality detection results, reliability, integrated judgment results, etc., accepts user operations, and outputs various information to the display device in accordance with the user operations.

[0139] FIG. 19 shows an example of a screen that the output processing unit 50 causes the display device to display.

[0140] The screen 60 shown in Figure 19 displays an input image display section 61, an abnormality detection result image display section 62, bars 63, 64, and 65, a reliability gradation image display section 67, a bar 68 and a marker 69, an abnormality area frame 70, a reliability display section 71, a floating matter rank display section 72, a rust rank display section 73, a deposit rank display section 74, and a floating matter display section 75.

[0141] The input image display unit 61 displays individual frame images (input images) of the video input by the video input unit 10. Information indicating the cable length is also superimposed on the input image.

[0142] The abnormality detection result image display unit 62 displays an image (an abnormality detection result image) in which information indicating the abnormality detection result is superimposed on the input image. In this example of the abnormality detection result image, an abnormality area frame 70 that surrounds the area where the abnormality was detected (in this example, the area where deposits have accumulated at the bottom of the pipe) is superimposed to make it easier to confirm the area where the abnormality was detected. In addition, similarly, symbols, rectangles, transparent colors, etc. may be superimposed on the image of the abnormality detection result image display unit 62 to make it easier to understand the area where the abnormality was detected and the type of abnormality.

[0143] The bar 63 indicates whether or not there is any floating matter that changes along the time axis of the video, so that it can be identified for each frame image. The left end of the bar 63 corresponds to the start time of the video, and the right end corresponds to the end time of the video.

[0144] The bar 64 displays the presence or absence of a transition that changes along the time axis of the video so that it can be identified for each frame image. The left end of the bar 64 corresponds to the start time of the video, and the right end corresponds to the end time of the video.

[0145] The bar 65 indicates whether or not there is sediment, which changes along the time axis of the image, so that it can be identified for each frame image. The left end of the bar 65 corresponds to the start time of the image, and the right end corresponds to the end time of the image.

[0146] The reliability grayscale image display section 67 displays the reliability, which indicates the degree of visibility inside the pipe as it changes along the time axis of the image, for each frame image using grayscale. The grayscale display is displayed, for example, superimposed on the entire display of the bars 63, 63, 65. Frame images with lower reliability are displayed in darker colors, and frame images with higher reliability are displayed in lighter colors. This makes it easy to understand which frame images are suitable for anomaly detection.

[0147] The bar 68 and marker 69 are used to seek to any frame image in the video, and also indicate the temporal position of the currently displayed frame image within the entire video. The left end of the bar 68 corresponds to the start time of the video, and the right end corresponds to the end time of the video. The frame image corresponding to the position of the marker 69 is displayed in the input image display area 61 and the anomaly detection result image display area 62.

[0148] The abnormality area frame 70 indicates the abnormal area in the abnormality detection result image displayed in the abnormality detection result image display section 62 .

[0149] The reliability display section 71 displays the reliability value corresponding to the frame image currently being displayed.

[0150] The floating object rank display section 72 displays the integrated judgment result of the floating object corresponding to the currently displayed frame image (integrated judgment result for the relevant part) as a rank. The rank is determined taking into consideration the reliability.

[0151] The rust rank display section 73 displays the integrated rust judgment result (integrated judgment result for the relevant part) corresponding to the currently displayed frame image as a rank. The rank is determined taking into consideration the reliability.

[0152] The deposit rank display section 74 displays the integrated judgment result of the deposit corresponding to the currently displayed frame image (the integrated judgment result for the corresponding part) as a rank. The rank is determined taking into consideration the reliability.

[0153] The flying up display section 75 indicates whether or not there is flying up of sediment in the frame image being displayed.

[0154] Although not shown in the example in Figure 19, it is possible to make the video play automatically by placing play, pause, and frame-by-frame buttons.

[0155] In the display example of FIG. 19, when the bars 63, 63, and 65 are viewed, it can be seen that the frame image corresponding to the position of the marker 69 has floating matter, no rust, and sediment.

[0156] Correspondingly, it can be seen that floating matter, but not rust, is shown in the input image of input image display unit 61 and the abnormality detection result image of abnormality detection result image display unit 62. Correspondingly, it can be seen that floating matter rank C is shown in floating matter rank display unit 72, rust rank S (no abnormality) is shown in rust rank display unit 73, and deposit rank B is shown in deposit rank display unit 74.

[0157] In the example display of Fig. 19, no floating of sediment has occurred, so the floating display section 75 indicates "no floating," but if floating had occurred, the floating display section 75 would indicate "present" floating. An example display of the screen 60 in this case is shown in Fig. 20.

[0158] FIG. 20 shows an example of a screen when a frame image showing flying sediments is displayed.

[0159] 20 differs from the display example of Fig. 19 in that a reliability graph 80 is displayed instead of the reliability grayscale image display portion 67. The reliability graph 80 displays the degree of reliability that changes in the time axis direction of the video in the form of a graph.

[0160] In the example display of Figure 20, the reliability graph 80 shows that the reliability for the position of marker 69 is at almost the lowest level. In this case, the splash-up display section 75 indicates that splash-up is "present." Looking at the bars 63, 63, 65, it can be seen that the frame image for the position of marker 69 is indicated by a decorative display N of a special color or pattern to indicate that a large amount of splash-up of sediment exceeding the reference value has occurred.

[0161] Correspondingly, the input image in input image display section 61 displays a large amount of floating sediment, and the abnormality detection result image in abnormality detection result image display section 62 displays information indicating "detection not possible." In this case, reliability display section 71 displays a reliability of "0." No rank is displayed in floating matter rank display section 72, rust rank display section 73, or sediment rank display section 74.

[0162] (operation) Next, an example of the overall operation of the inside of pipe inspection device 1 according to the embodiment will be described with reference to the flowchart of FIG.

[0163] In step S101, the video input unit 10 inputs video data from the camera 4 that took pictures while moving inside the water pipe 3, the attribute information input unit 11 inputs attribute information indicating attributes associated with the camera 4, and the camera position movement detection unit 20 detects the position and movement of the camera 4 inside the pipe using the video data input by the video input unit 10 and the attribute information input by the attribute information input unit 11.

[0164] In step S102, the impurity detection unit 21 detects impurities in the pipe (including determining the presence or absence of impurities, determining the amount and distribution of impurities, determining the movement of impurities, determining the type of impurities, etc.) for each of one or more frame images based on the video input by the video input unit 10, and generates impurity type information and impurity distribution information.

[0165] In step S103, the abnormality detection unit 30 detects abnormalities in the water pipe 3 for each one or more frame images based on the video input by the video input unit 10 (including implementing an appropriate abnormality detection processing method according to the type and amount of impurities, determining the presence or absence of an abnormality, determining the level of abnormality, etc.), and generates the abnormality detection result, as well as information indicating the reliability of the visibility within the pipe, which is determined based on the impurity distribution information.

[0166] In step S104, the integrated judgment unit 40 generates an integrated judgment result indicating the degree of abnormality of each detection target based on the abnormality detection result and reliability generated by the abnormality detection unit 30 for each one or more frame images for each part of the water pipe 3.

[0167] In step S105, the storage unit 41 stores and preserves various information including video including individual frame images, attribute information, the presence or absence and type of impurities, abnormality detection results, reliability, and integrated judgment results.

[0168] In step S106, the output processing unit 50 acquires various information from the memory unit 41, including video including individual frame images, attribute information, the presence or absence and type of impurities, abnormality detection results, reliability, integrated judgment results, etc., accepts user operations, and outputs and displays various information on the screen of the display device in accordance with the user operations.

[0169] As described above in detail, according to the embodiment, appropriate test results can be easily obtained even when impurities are present in the tube.

[0170] For example, if there are impurities such as floating sediments or floating objects inside the pipe, not only can the abnormality detection results and integrated judgment results be confirmed on the display device screen, but the occurrence status of the impurities and the reliability indicating the visibility inside the pipe depending on the amount of impurities can also be confirmed, allowing for a more accurate understanding of the situation inside the pipe.

[0171] Furthermore, in anomaly detection, an appropriate processing method is implemented depending on the type and amount of impurities, so that accurate anomaly detection results can be obtained even when impurities are present.

[0172] Furthermore, the integrated determination result reflects the reliability according to the amount of impurities, so that it is possible to obtain an integrated determination result with high accuracy even when impurities are present. [Explanation of symbols]

[0173] 1...pipe inspection device, 2...site, 3...water pipe, 4...camera, 5...recording medium, 10...video input unit, 11...attribute information input unit, 20...camera position movement detection unit, 21...impurity detection unit, 30...anomaly detection unit, 40...integrated judgment unit, 41...memory unit, 42...management unit, 50...output processing unit, A, B...straight section, C...seal coat (surface paint), D...deposit, DF...floating deposit, F...floating matter, P...joint, R...rust.

Claims

1. an impurity detection unit that detects impurities in the pipe based on an image captured by a camera moving inside the pipe and generates information indicating at least the amount of the impurities; an abnormality detection unit that detects abnormalities in the pipe based on the image, outputs the abnormality detection result, and outputs information indicating the reliability of visibility inside the pipe, which is determined according to the amount of impurities; an integrated determination unit that outputs a determination result indicating the degree of abnormality of the pipe based on the abnormality detection result and the reliability; An internal pipe inspection device comprising:

2. The device further includes an output processing unit that displays at least the anomaly detection result, the reliability, and information indicating the reliability on a display device. The pipe inspection device according to claim 1 .

3. The abnormality detection unit outputting information indicating the anomaly detection result and the reliability for each of one or more frame images; The integrated determination unit generating the determination result based on the abnormality detection result and the reliability output for each of one or more frame images for each portion of the pipe; The pipe inspection device according to claim 1 .

4. The integrated determination unit determining the determination result by preferentially using the worst abnormality detection result among the abnormality detection results obtained from a plurality of frame images for each portion of the pipe; The pipe inspection device according to claim 3.

5. The integrated determination unit If there is a frame image whose reliability is equal to or greater than a certain value, the determination result for that frame image is determined using the worst anomaly detection result obtained from that frame image; When there is a frame image in which the reliability indicates a value other than 0 and less than a certain value, the determination result for that frame image is determined using the worst anomaly detection result obtained from at least a frame image in which the reliability indicates a value equal to or greater than a certain value. The pipe inspection device according to claim 4.

6. The impurity detection unit includes: generating, for each of one or more frame images, impurity distribution information that individually indicates a value corresponding to the amount of the impurities appearing in each unit area of ​​the frame image; The abnormality detection unit determining the reliability for each of one or more frame images using the impurity distribution information; The pipe inspection device according to claim 1 .

7. The impurity detection unit includes: Detecting the movement of the impurities based on a plurality of frame images, determining whether the impurities are floating sediments or floating matter based on the movement, and outputting information indicating the type. The pipe inspection device according to claim 1 .

8. The abnormality detection unit A method for detecting an anomaly is changed depending on the type. The pipe inspection device according to claim 7.

9. The abnormality detection unit When the impurities correspond to the floating matter, At least one of a processing method of weighting the anomaly detection results obtained for each frame image according to the amount of impurities, a processing method of performing image processing to reduce the decrease in visibility due to the impurities and then performing anomaly detection, and a processing method of invalidating anomaly detection when the amount of impurities is equal to or greater than a certain level is adopted. The pipe inspection device according to claim 8.

10. The abnormality detection unit When the impurities correspond to the floating matter, At least one of a processing method for detecting anomalies in a frame image in which the amount of impurities is less than a certain level, a processing method for detecting anomalies in an area in one frame image in which the amount of impurities is less than a certain level, a processing method for weighting the anomaly detection results obtained for each frame image according to the amount of impurities, a processing method for detecting anomalies after performing image processing to reduce the decrease in visibility due to the impurities, and a processing method for invalidating anomaly detection when the amount of impurities is equal to or greater than a certain level is adopted. The pipe inspection device according to claim 8.

11. The abnormality detection unit When the impurities correspond to the floating sediments, At least one of the following is adopted: a processing method for detecting anomalies in frame images in which the amount of impurities is less than a certain level; a processing method for detecting anomalies in areas in one frame image in which the amount of impurities is less than a certain level; a processing method for weighting the anomaly detection results obtained for each frame image according to the amount of impurities; a processing method for detecting anomalies after performing image processing to reduce the decrease in visibility due to the impurities; a processing method for detecting anomalies in frame images in which the amount of impurities is less than a certain level while the camera is stopped; and a processing method for prioritizing anomaly detection on the outbound journey of the camera over anomaly detection on the return journey. The pipe inspection device according to claim 8.

12. The impurity detection unit detects impurities in the pipe based on an image captured by a camera moving inside the pipe, and generates information indicating at least the amount of the impurities; An abnormality detection unit detects an abnormality in the pipe based on the image, outputs the abnormality detection result, and outputs information indicating the reliability of visibility inside the pipe, which is determined according to the amount of impurities; outputting a determination result indicating the degree of abnormality of the pipe based on the abnormality detection result and the reliability by an integrated determination unit; An intra-pipe inspection method comprising:

13. On one or more computers, A function of detecting impurities in a pipe based on an image captured by a camera moving inside the pipe and generating information indicating at least the amount of the impurities; A function of detecting an abnormality in the pipe based on the image, outputting the abnormality detection result, and outputting information indicating the reliability of visibility inside the pipe determined according to the amount of impurities; a function of outputting a determination result indicating the degree of abnormality of the pipe based on the abnormality detection result and the reliability; A program to achieve this.

Images