Image processing device and image processing method

Abstract

To provide an image processing device capable of identifying the factor of an abnormality that occurs in a structure.SOLUTION: An image processing device includes a joint position acquisition unit configured to acquire, from an image of a structure constituted by a plurality of members, positions of joints between the plurality of members, an abnormality detection unit configured to detect, from the image, an abnormality occurring in the structure and the position of the abnormality, and an abnormality factor identification unit configured to identify an occurrence factor of the abnormality based on the positions of the joints and the position of the abnormality.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an image processing device and an image processing method. [Background technology]

[0002] In recent years, technology has been proposed to photograph structures (such as the inner walls of sewer pipes or tunnel walls) with a camera and detect abnormalities in the structures from the captured images in order to check for abnormalities such as cracks or damage.

[0003] For example, Patent Document 1 listed below discloses a method in which, in the case of a sewer pipe, the defect is assumed to exist at a pipe joint extending circumferentially around the pipe, and a predetermined area extending forward and backward in the axial direction of the pipe joint is cut out and defect detection and discrimination is performed on the cut-out image. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2007-24746 A Summary of the Invention [Problem to be solved by the invention]

[0005] However, in the method described in Patent Document 1, pipe joints are extracted by calculating the edge strength of the image, and if abnormalities such as free lime or water leakage occur at the pipe joint, the edge strength becomes weak and it becomes difficult to extract the pipe joint.

[0006] Furthermore, in the case of a structure in which an abnormality (free lime, water leakage, etc.) has occurred, it is necessary not only to determine whether or not the abnormality exists, but also to identify the cause of the abnormality (damage, cracks, joints, etc.).

[0007] One aspect of the present invention has been made in view of the above-mentioned problems, and has an object to provide an image processing device capable of identifying the cause of an abnormality occurring in a structure. [Means for solving the problem]

[0008] An image processing device according to one embodiment of the present invention includes a seam position acquisition unit that acquires the positions of seams between a structure composed of a plurality of components from an image captured of the structure, an abnormality detection unit that detects an abnormality occurring in the structure and the position of the abnormality from the image, and an abnormality cause identification unit that identifies a cause of the abnormality based on the position of the seam and the position of the abnormality.

[0009] An image processing method according to one embodiment of the present invention includes steps of obtaining positions of joints between a structure composed of a plurality of components from an image of the structure, detecting an abnormality occurring in the structure and the position of the abnormality from the image, and identifying a cause of the abnormality based on the position of the joint and the position of the abnormality. [Brief description of the drawings]

[0010] [Figure 1] FIG. 1 is a diagram illustrating an example of a configuration of an image processing device according to an embodiment. [Diagram 2] FIG. 13 is a diagram for explaining the imaging of a tubular structure by an imaging device. [Figure 3A] 1 is an example of an image captured of the inside of a tubular structure composed of multiple members. [Figure 3B] 13 is an example of a result of edge extraction. [Figure 3C] This is an example of a mask. [Figure 4] 13 is an example of position information of a seam. [Diagram 5] FIG. 1 is a diagram showing an example of an abnormality occurring in a structure. [Figure 6] This is an example of information on the location and cause of an anomaly. [Figure 7A] FIG. 2 is a side view of the tubular structure. [Figure 7B] 13 is an example of seam position information before the seam is added. [Figure 7C] 13 is an example of seam position information after a seam is added. [Figure 8] 13 is an example of anomaly location and cause information after a cause has been added. [Figure 9] 4 is a flowchart illustrating an example of an image processing method according to an embodiment. [Figure 10] FIG. 13 is a diagram showing an example of displaying the cause of occurrence of a first abnormality on a display unit. [Figure 11A] This is an example of an image of a concrete wall. [Figure 11B] 13 is an example of seam position information before the seam is added. [Figure 11C] 13 is an example of seam position information after a seam is added. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Hereinafter, embodiments will be described with reference to the drawings. In the drawings, the same or equivalent elements are designated by the same reference numerals, and duplicated explanations will be omitted.

[0012] FIG. 1 is a diagram illustrating an example of a configuration of an image processing device according to an embodiment.

[0013] The image processing device 101 includes a processing unit 111, a storage unit 121, and a display unit 131. The image processing device 101 is, for example, a computer such as a personal computer (PC) or a server.

[0014] The processing unit 111 performs processes such as control of the image processing device 101. The processing unit 111 has an image acquisition unit 112, a seam position acquisition unit 113, an abnormality detection unit 114, an abnormality cause identification unit 115, and a display control unit 116. The processing unit 111 is realized by, for example, a CPU (Central Processing Unit), a GPU (Graphic Processing Unit: image processing device), an FPGA (Field Programmable Gate Array: an integrated circuit whose circuit can be rewritten after manufacture), or an ASIC (Application Specific Integrated Circuit: an integrated circuit for a specific application).

[0015] The image acquisition unit 112 acquires an image of a structure composed of multiple members. The acquired image is, for example, a video including multiple images (frames) captured continuously, or one or more images. The structure composed of multiple members is, for example, a tubular structure such as a sewer pipe, a water pipe, a gas pipe, a pipeline, or a chimney.

[0016] Here, imaging of a tubular structure will be described.

[0017] FIG. 2 is a diagram for explaining the imaging of a tubular structure by the imaging device.

[0018] The imaging device 201 captures images of the inside of a tubular structure. Examples of tubular structures include sewer pipes, water supply pipes, gas pipes, pipelines, and tunnels. In the following description, the axis parallel to the depth direction of the piping is the Z axis, and the axes perpendicular to the Z axis are the X axis and the Y axis. In the embodiment, the tubular structure is disposed in the horizontal direction, and the X axis and the Z axis are aligned along the horizontal direction, and the Y axis is aligned along the vertical direction. First, the configuration of the imaging device 201 will be described.

[0019] The imaging device 201 captures images of the inside of the tubular structure multiple times while moving in the depth direction of the tubular structure to obtain multiple frames of images. FIG. 2 is a schematic diagram of the imaging device 201 moving inside the tubular structure 203 in a cross section (YZ cross section) in the depth direction (longitudinal direction) of the tubular structure 203. As shown in FIG. 2, the imaging device 201 includes a camera 211 and a moving body 212 that is mounted with the camera 211 and can move inside the tubular structure 203 in the depth direction. The moving body 212 includes a light that irradiates illumination light toward the imaging direction of the camera 211. As shown in FIG. 2, the camera 211 is installed at a position at a predetermined height of the moving body 212 so that the optical axis 204 of the camera 211 is parallel to the traveling direction of the moving body 212. The camera 211 captures an imaging range (angle of view) 205 centered on the optical axis 204.

[0020] The photographing device 201 is disposed inside the tubular structure 203 such that the optical axis 204 of the camera 211 is aligned with the Z-axis. The photographing device 201 photographs a plurality of images at different positions in the depth direction of the tubular structure 203. The photographing device 201 transmits the images to the image processing device 101 connected via a wireless or wired network, and the image processing device 101 stores the received images in the storage unit 121. The photographing device 201 may also store the photographed images in a storage device provided inside the moving body 212.

[0021] Returning to Figure 1, we will continue with the explanation.

[0022] The image acquisition unit 112 acquires, for example, an image of a structure stored in the storage unit 121. The image processing device 101 may be connected to the image capturing device 201 via a wireless or wired network, or may be connected to the image capturing device 201 via a cable, and acquire an image of the structure from the image capturing device 201.

[0023] The joint position acquisition unit 113 acquires the positions of joints between a plurality of members from an image of a structure made up of a plurality of members. Here, detection of the joint positions will be described.

[0024] Fig. 3A is an example of an image captured inside a tubular structure composed of multiple members. Image 301 in Fig. 3A is an image of one frame constituting a video.

[0025] Tubular structures such as sewer pipes are constructed by connecting circular pipes of a certain length. The joints between the circular pipes are called seams. In the case of the tubular structure 203, the seams 302 appear as circles on the image 301.

[0026] The seam position acquisition unit 113 performs edge extraction on the image 301 acquired by the image acquisition unit 112, and if the obtained edge is circular, detects the seam as being present on the image 301. For example, the seam position acquisition unit 113 calculates edge strength using a Sobel filter or the like in the edge extraction, and extracts pixels with edge strength equal to or greater than a threshold value as edges.

[0027] FIG. 3B is an example of the result of edge extraction.

[0028] In the edge extraction result 303 shown in FIG. 3B, the extracted edges are drawn in white and the rest of the image is drawn in black, with seams and dirt on the underside of the structure being extracted as edges.

[0029] FIG. 3C is an example of a mask.

[0030] The seam position acquisition unit 113 applies a circular mask 304 as shown in Fig. 3C to the edge extraction result 303, and determines that the extracted edge is circular if a threshold number or more of edge pixels are extracted at a position in the edge extraction result 303 that corresponds to the circular mask 304 shown in white in Fig. 3C. The seam position acquisition unit 113 determines that the image (frame) in which the extracted edge is determined to be circular is the image (frame) in which a seam is detected.

[0031] FIG. 4 is an example of the seam position information.

[0032] The seam position acquisition unit 113 sets an ID (seam ID) for identifying each detected seam, sets the number (frame number) of the image (frame) in which the seam is detected as the position [frame] indicating the position of the seam, and generates seam position information in which the seam ID and the position [frame] are associated with each other. Note that the frame numbers are assigned in order, for example, starting from 1, from the first frame of the video. As shown in FIG. 4, in the seam position information 401, seam ID=1 is associated with position [frame]=200, seam ID=2 is associated with position [frame]=400, seam ID=3 is associated with position [frame]=600, ... respectively. The seam position information 401 in FIG. 4 indicates that seams have been detected in, for example, the 200th, 400th, and 600th frames from the start of the video.

[0033] The seam position acquisition unit 113 outputs the generated seam position information 401 to the abnormality cause identification unit 115. The seam position acquisition unit 113 may further cause the storage unit 121 to store the position information 401.

[0034] Returning to Figure 1, we will continue with the explanation.

[0035] The anomaly detection unit 114 detects an anomaly occurring in a structure composed of a plurality of members and the position where the anomaly occurs from an image of the structure. The position where the anomaly occurs is, for example, the number of the frame where the anomaly is detected.

[0036] The anomaly detection unit 114 detects anomalies occurring in a structure using, for example, a method of detecting anomalies using a trained model that has been machine-learned using images containing anomalies as learning data, or a method of detecting anomalies based on changes in the brightness values ​​of the image.

[0037] FIG. 5 is a diagram showing an example of an abnormality occurring in a structure.

[0038] 5, free lime 501 spreads in the circumferential direction of the inner wall of the tubular structure, and a crack 502 has occurred on the right side. Since the inner wall without an abnormality and the free lime 501 and crack 502 have different characteristics in terms of brightness values, etc., the abnormality detection unit 114 can detect an abnormality by a method using machine learning or a method based on a change in brightness values.

[0039] Next, the anomaly detection unit 114 identifies the type of anomaly for the detected anomaly. The type of anomaly is identified, for example, in the same manner as in the detection of anomalies, by using a trained model that has been machine-learned using images in which anomalies are classified by type as learning data. The anomaly detection unit 114 may also identify the type of anomaly based on a change in the luminance value of the image.

[0040] In the embodiment, anomalies are classified into anomalies that require the identification of the cause of occurrence (first anomalies) and anomalies that do not require the identification of the cause of occurrence (second anomalies). Anomalies that require the identification of the cause of occurrence are, for example, free lime, water leakage, and tree root intrusion, and anomalies that do not require the identification of the cause of occurrence are, for example, damage and cracks. For example, when anomalies such as free lime, water leakage, and tree root intrusion are detected, if it is possible to identify whether the abnormality is caused by damage, a crack, a joint, or a combination of damage and joints, it can be useful for determining the deterioration state of a structure and formulating a repair plan. In addition, the abnormality cause identification unit 115 automatically identifies the cause of occurrence, thereby reducing the burden on the inspector.

[0041] FIG. 6 shows an example of information on the location and cause of anomaly.

[0042] The anomaly detection unit 114 detects anomalies occurring in the structure from the images (frames) acquired by the image acquisition unit 112, sets an ID (anomaly ID) for identifying each detected anomaly, and sets the number (frame number) of the image (frame) in which the anomaly is detected as the position [frame] indicating the position of the anomaly. Note that the frame numbers are assigned, for example, starting from 1 in order from the first frame of the video. Furthermore, the anomaly detection unit 114 identifies the type of the detected anomaly, and determines whether or not the detected anomaly is an anomaly that requires the cause of occurrence to be identified according to the identified type of anomaly. For example, if the type of anomaly is free lime, water leakage, or tree root intrusion, it is determined to be an anomaly that requires the cause of occurrence to be identified, and if the type of anomaly is breakage or crack, it is determined to be an anomaly that does not require the cause of occurrence to be identified.

[0043] The anomaly detection unit 114 generates anomaly position and cause information 601 as shown in FIG. 6 in which anomaly ID, position [frame], type, cause identification, and cause are associated with each other, and outputs the information to the anomaly cause identification unit 115. The anomaly ID is an ID for identifying each detected anomaly, the position [frame] indicates the position of the detected anomaly, and the type indicates the type of the detected anomaly. The cause identification indicates whether or not the detected anomaly is an anomaly that requires identification of the cause of occurrence, with "Yes" indicating that identification of the cause of occurrence is required and "No" indicating that identification of the cause of occurrence is not required. The cause indicates the cause of the detected anomaly. Note that the cause of the anomaly is identified by the anomaly cause identification unit 115 described later, so at this point, the cause in the anomaly position and cause information 601 is blank (-).

[0044] 6, the first row of the anomaly location and cause information 601 corresponds to anomaly ID=1, position [frame]=800, type=free lime, cause identified=yes, and cause=blank (-). This indicates that the anomaly corresponding to anomaly ID=1 was detected in the 800th frame of the video, the type of the anomaly is free lime, and the cause of the anomaly needs to be identified.

[0045] Returning to Figure 1, we will continue with the explanation.

[0046] The abnormality cause identification unit 115 identifies the cause of an abnormality (first abnormality) whose cause needs to be identified, based on the seam position information 401 acquired from the seam position acquisition unit 113 and the abnormality position and cause information 601 acquired from the abnormality detection unit 114.

[0047] First, the abnormality factor identifying unit 115 calculates the interval between the seams from the positions of the seams, and adds the positions of the seams so that the intervals between the seams become regular. Here, the details of the process of adding the positions of the seams will be described with reference to FIGS. 7A to 7C.

[0048] 7A is a diagram of the tubular structure 203 as a structure viewed from the side, and shows the positions of seams 302-i (i=1 to (N-1)) detected by the seam position acquisition unit 113 in the tubular structure 203. In Fig. 7A, as in Fig. 2, the axis parallel to the depth direction of the tubular structure 203 is the Z axis, and the axes perpendicular to the Z axis are the X axis and the Y axis.

[0049] FIG. 7B is an example of seam position information.

[0050] As shown in seam position information 701 in Fig. 7B, N-1 seams have been detected by seam position acquisition unit 113. The seam corresponding to seam ID=i in Fig. 7B is seam 302-i in Fig. 7A.

[0051] Here, the position where a seam is detected is defined as Ln, and the difference in the positions of adjacent seams (Ln-Ln-1) is defined as the seam interval. n indicates the ordinal seam, and is defined as the value of the seam ID here. The position of a seam is defined as the position [frame] in the seam position information that corresponds to the seam ID of the seam. For example, the position L1 of the seam with seam ID=1 is 200, and the position L2 of the seam with seam ID=2 is 400, and the interval between these seams is 200 (=400-200).

[0052] When joints are arranged at equal intervals in a structure, the joint intervals calculated at each location are equal. The abnormality factor identification unit 115 calculates the intervals of each joint detected by the joint position acquisition unit 113, and calculates the median value Lmed of the calculated intervals of each joint. Here, the rule for arranging the joints is set in advance, for example, to be equal intervals. The abnormality factor identification unit 115 detects joints that were not detected by the joint position acquisition unit 113 due to abnormalities such as free lime or water leakage, assuming that the joints are arranged at equal intervals with the median value Lmed, and adds the detected joints to the position information of the joints. The abnormality factor identification unit 115 calculates the positions of the undetected joints based on the median value Lmed.

[0053] For example, the abnormality factor identification unit 115 detects a location where the seams are not regularly arranged, i.e., not evenly spaced, based on the seam position information and the seam interval rule, and performs a process of adding seams. The location where the seams are not regularly arranged is a location where the edge of the seam is difficult to obtain due to an abnormality such as free lime or water leakage that occurs around the seam, and the seam was not detected. The abnormality factor identification unit 115 calculates this location and adds a seam to this location, thereby making it possible to add a seam that could not be detected by the seam position acquisition unit 113. The location where the seams are not regularly arranged is, for example, a location where the seam interval is Pmed±α or more. α is a predetermined value, and can be set, for example, to 5% of the seam interval Pmed. The abnormality factor identification unit 115 detects a location where the seam interval is Pmed±α or more based on the seam position information, and adds a seam to the detected location. Specifically, for example, the information on the seam at the detected location is added to the seam position information.

[0054] For example, in the seam position information 701 of Fig. 7B, the median seam spacing Lmed is 200, and the spacing between the seam corresponding to seam ID = 3 and the seam corresponding to seam ID = 4 is 400, so the abnormality factor identifying unit 115 detects the area between the seam corresponding to seam ID = 3 and the seam corresponding to seam ID = 4 as a location 702 (see Fig. 7A) where seams are not regularly arranged. The abnormality factor identifying unit 115 adds information about the detected location to the seam position information 701. The abnormality factor identifying unit 115 calculates the position of the seam to be added based on the median Lmed, and, for example, determines the position of the seam to be added to a location where the spacing between the seam to be added and an adjacent seam is the median Lmed.

[0055] In seam position information 704 after adding seam information shown in Fig. 7C, information on seam 703 is added between seam ID=3 and seam ID=4 in seam position information 701 in Fig. 7B, and the position [frame] of the added seam 703 is set to 800 so that the distance to the adjacent seam with seam ID=3 is the median Lmed=200. As a result, in seam position information 704 shown in Fig. 7C, the seams are spaced at equal intervals of 200. Also, in seam position information 704 shown in Fig. 7C, the anomaly cause identification unit 115 reassigns the seam IDs after adding the seam information, so that seam IDs=4 to N-1 in Fig. 7B become seam IDs=5 to N in Fig. 7C, respectively.

[0056] In this way, by adding joints so that they are spaced at equal intervals, even if the joints are filled in by free lime or water leakage and become difficult to see in the image, and the joints are not detected, by adding the position of the joints, it is possible to identify that the joints are the cause of the first abnormality.

[0057] Next, a method for identifying the cause of an anomaly will be described. Anomaly cause identifying unit 115 identifies the cause of a first anomaly as at least one of a seam detected at the same position as the first anomaly and a second anomaly.

[0058] A specific method for identifying the cause of an abnormality using the abnormality position / cause information 601 shown in FIG. 6 and the seam position information 704 shown in FIG. 7C will be described.

[0059] As shown in anomaly position and cause information 601 in Fig. 6, free lime and a crack are detected at position [frame] = 800. Since it is necessary to identify the cause of free lime, anomaly cause identification unit 115 identifies a joint and a second anomaly detected at the same position [frame] as the cause of the occurrence. For example, as shown in joint position information 704 in Fig. 7C, there is a joint at position [frame] = 800, and there is a second anomaly, a crack, at position [frame] = 800 in anomaly position and cause information 601 in Fig. 6, so anomaly cause identification unit 115 identifies the causes of the occurrence of free lime, which is the first anomaly, as being the joint and a crack.

[0060] 6, a leak and damage are detected at position [frame]=900. Since it is necessary to identify the cause of a leak, the fault cause identifying unit 115 identifies the damage detected at the same position [frame] as the cause of the leak. Note that in the joint position information 704 shown in FIG. 7C, a joint is not detected at position [frame]=900, so the fault cause identifying unit 115 does not identify the joint as the cause of the leak.

[0061] In this way, there are cases where the cause of occurrence is multiple, such as joints and cracks, and cases where it is a single piece of damage. Since there is often not just one cause but multiple causes in severely deteriorated structures, if it is possible to identify all secondary abnormalities such as joints, cracks, and damage detected in the same position as causes of occurrence, the user can consider the compound effects when checking the results and determining the degree of deterioration or when making repair plans.

[0062] The anomaly cause identification unit 115 adds the identified cause of the anomaly to the anomaly position / cause information 601 acquired from the anomaly detection unit 114. For example, the identified cause of the anomaly is added to the anomaly position / cause information 601 in Fig. 6 to generate anomaly position / cause information 801 as shown in Fig. 8, and stored in the storage unit 121. In the anomaly position / cause information 801 in Fig. 8, the cause of free lime corresponding to anomaly ID = 1 is a joint and a crack detected in the same position [frame] as the free lime. Also, the cause of water leakage corresponding to anomaly ID = 3 is a breakage detected in the same position [frame].

[0063] Returning to Figure 1, we will continue with the explanation.

[0064] The display control unit 116 causes the display unit 131 to display information based on the abnormality position / cause information 801, images recorded in the storage unit 121, and the like.

[0065] The storage unit 121 stores programs, data, and the like used by the image processing device 101. The storage unit 121 is, for example, a semiconductor memory such as a static random access memory (SRAM), a dynamic random access memory (DRAM), a read only memory (ROM), or a flash memory, a magnetic storage device such as a hard disk drive (HDD), or an optical storage device such as an optical disk device. The storage unit 121 stores, for example, images of a structure composed of multiple members, position and cause information of an abnormality, and the like.

[0066] Display unit 131 displays the processing results output from processing unit 111 and images recorded in storage unit 121. Display unit 131 is, for example, a liquid crystal display or an organic EL (ElectroLuminessence) display.

[0067] FIG. 9 is an example of a flowchart of an image processing method according to the embodiment.

[0068] In step S901, the image acquisition unit 112 acquires an image of a structure composed of a plurality of members. Specifically, for example, the image acquisition unit 103 acquires a video including a plurality of images (frames) of the inside of a tubular structure such as a sewer pipe.

[0069] In step S902, the seam position acquisition unit 113 detects a seam appearing in each image (frame) constituting the acquired moving image, generates position information of the seam, and outputs the position information of the seam to the abnormality cause identification unit 115. For example, the seam position acquisition unit 106 generates seam position information 401 shown in FIG.

[0070] In step S903, the abnormal area detection unit 114 detects an abnormality occurring in the structure and the position where the abnormality occurs for each image (frame) that constitutes the video, generates abnormality position and cause information 601, and outputs it to the abnormality cause identification unit 115.

[0071] In step S904, the abnormality cause identification unit 115 identifies the cause of an abnormality (first abnormality) whose cause needs to be identified based on the seam position information 401 acquired from the seam position acquisition unit 113 and the abnormality position and cause information 601 acquired from the abnormality detection unit 114, and adds the identified abnormality cause to the abnormality position and cause information 601.

[0072] In step S905, display control unit 116 causes display unit 131 to display an image (frame) of the position where the first abnormality was detected, and further causes display unit 131 to display the first abnormality and the cause of the first abnormality.

[0073] FIG. 10 is a diagram showing an example of displaying the cause of the first abnormality on the display unit.

[0074] As shown in abnormality position / cause information 801 in Fig. 8, free lime is detected as a first abnormality in a frame corresponding to position [frame] = 800. As shown in Fig. 10, the display control unit 116 causes the display unit 116 to display a frame corresponding to position [frame] = 800 of the video, and further causes the display unit 116 to display a solid-line rectangle and text indicating the detected free lime 1001. The display control unit 116 also causes the display unit 131 to display a dashed-line rectangle and text indicating a joint 1002 and a crack 1003, which are causes of the free lime that is the first abnormality. This makes it possible to present the cause of occurrence to the user in an easily understandable manner.

[0075] According to the image processing device of the embodiment, the cause of an abnormality occurring in a structure can be identified.

[0076] (Variation 1) In the above embodiment, the abnormality factor identifying unit 115 has determined that the seams are arranged at equal intervals with the median value Pmed, but the seams may be arranged at intervals according to an arbitrary rule. For example, an arithmetic progression, a geometric progression, or a plurality of rule patterns may be stored in the storage unit 121, and the abnormality factor identifying unit 115 may determine the rule that best matches the calculated seam interval among the rules stored in the storage unit 121 as the seam rule, and add seams based on the seam rule. The match may be determined by checking whether the difference between the seam interval calculated according to the rule and the detected seam interval is equal to or less than a predetermined value, or whether the rate at which the seam interval calculated according to the rule and the detected seam interval match is equal to or more than a predetermined value.

[0077] By automatically determining the spacing rules for seams, seams can be added even when they are arranged according to an arbitrary rule, and it can be determined whether the cause of the problem is the seams.

[0078] (Variation 2) In the above embodiment, the abnormality cause identifying unit 115 represents the position of a joint or an abnormality by the frame number of a video frame, but the position may be represented by a distance using distance information in a structure linked to the video. For example, in the case of a tubular structure such as a sewer pipe, the frame and the distance can be replaced by setting the pipe opening to 0 mm and inputting the distance D mm from the pipe opening and the corresponding frame of the video taken at that position.

[0079] (Variation 3) In the above embodiment, the anomaly cause identifying unit 115 identifies at least one of the seam or the second anomaly detected at the same position as the first anomaly as the cause of the first anomaly, but a margin may be provided for the position. For example, when the first anomaly is detected at position (M), at least one of the seam or the second anomaly detected within a range of position M±T may be identified as the cause of the first anomaly. By providing a margin T for the position, it is possible to absorb a slight difference in the timing at which the anomaly or seam is detected, making it easier to identify the cause of the anomaly.

[0080] (Variation 4) In the above embodiment, the structure is a tubular structure. However, the structure may be a flat structure such as a concrete wall.

[0081] In the above-described embodiment, the seam position acquisition unit 113 detects one seam from one image (frame) that constitutes a video, but it is also possible to detect multiple seams from one image and add seams so that the seams become regular.

[0082] An example in which a plurality of joints are detected in an image will be described with reference to Figures 11A to 11C. Figure 11A is an example of an image of a concrete wall surface.

[0083] Structures such as concrete walls have joints such as formwork streaks, and so a single image shows multiple joints 111-1 to 111-4, as shown in image 1101 in Fig. 11A. In Fig. 11A, if vertical axis 1102 of image 1101 is taken as the Y-axis direction, joints 111-1 to 111-4 are located at positions Y coordinate = 100, 200, 300, and 500. In image 1101, the joint is not clearly visible at position Y coordinate = 400 because there is an abnormality such as free lime and water leakage on the wall surface, shown in gray.

[0084] Since formwork streaks appear as straight lines, the joints can be detected by extracting edges and applying a Hough transform to the extracted edges.

[0085] The image acquisition unit 112 acquires an image 1101, and the seam position acquisition unit 113 performs edge extraction on the image 1101 and applies a Hough transform to the extracted edges to detect seams.

[0086] The seam position acquisition unit 113 sets an ID for each detected seam, and acquires the seam position (position [coordinate]) as coordinates in the vertical axis direction 1102 of the image 1101. The seam position acquisition unit 113 generates seam position information that associates the seam ID with the position [coordinate]. The seam position acquisition unit 113 generates, for example, seam position information 1103 shown in FIG. 11B. The seams corresponding to seam IDs 1 to 4 in FIG. 11B are seams 111-1 to 111-4 in FIG. 11A, respectively.

[0087] In Fig. 11B, the joint between joint ID3 and joint ID4 cannot be detected due to anomalies such as free lime occurring on the wall surface and water leakage. The anomaly cause identification unit 115 calculates the median joint interval Lmed, and since the median joint interval Lmed is 100 in Fig. 11B, information on a joint at position [coordinate] = 400 is added between joint ID3 and joint ID4 in the joint position information 1103. As a result, the joint position information 1103 becomes position information 1104 shown in Fig. 11C. The joint position information 1104 is an example in which joints are added so that the joint intervals are regular, i.e., equal, and the joint IDs are reassigned after the joints are added.

[0088] The anomaly cause identifying unit 115 identifies the cause of an anomaly (first anomaly) whose cause needs to be identified, based on the seam position information 1104 after the seam is added.

[0089] In this way, even if there are multiple seams in an image, by adding seams based on the rules for seam spacing, it is possible to determine whether a seam is the cause of an abnormality when inspecting structures such as concrete walls.

[0090] (Variation 5) In the above-described embodiment, the seam position acquisition unit 113 detects the seams of the structure from the video, but the seams may be detected from a drawing of the structure or the like. In addition, the user may input the position of the seam using an input unit (not shown) included in the image processing device 101. Even if the seam is not input due to a mistake in the drawing or a detection mistake by the user, the image processing device 101 can automatically calculate the distance between the seams and add the seam, thereby reducing the burden on the inspector.

[0091] (Example of software implementation) The control block of the image processing device 101 (particularly the image acquisition unit 112, seam position acquisition unit 113, abnormal area detection unit 114, and abnormality cause identification unit 115 of the processing unit 111, and the display control unit 116) may be realized by a logic circuit (hardware) formed in an integrated circuit (IC chip) or the like, or may be realized by software.

[0092] In the latter case, the image processing device 101 includes a computer that executes instructions of a program, which is software that realizes each function. This computer includes, for example, at least one processor (control device) and at least one computer-readable recording medium that stores the program. This computer also includes hardware such as an OS (Operating System) and peripheral devices. In the computer, the processor reads the program from the recording medium and executes it, thereby achieving the object of the present invention.

[0093] The processor may be, for example, a CPU (Central Processing Unit). The recording medium may be a "non-transient tangible medium", for example, a portable medium such as a ROM (Read Only Memory) or a CD-ROM, a disk such as a tape, a flexible disk or a magneto-optical disk, a card, a semiconductor memory, a programmable logic circuit, or a hard disk built into a computer system. The computer-readable recording medium may be one that dynamically holds a program for a short period of time, such as a network such as the Internet or a communication line when transmitting a program via a communication line such as a telephone line, or one that holds a program for a certain period of time, such as a volatile memory inside a computer system that is a server or client.

[0094] The image processing device 101 may further include a RAM (Random Access Memory) for expanding the program. The program may be supplied to the computer via any transmission medium capable of transmitting the program (such as a communication network or a broadcast wave). One aspect of the present invention may also be realized in the form of a data signal embedded in a carrier wave, in which the program is embodied by electronic transmission.

[0095] Also, a part or all of the image processing device 101 may be realized as an LSI, which is typically an integrated circuit. In this case, the control block of the image processing device 101 may be individually chipped, or a part or all of it may be integrated and chipped. The integrated circuit method is not limited to LSI, and may be realized by a dedicated circuit or a general-purpose processor. Furthermore, if an integrated circuit technology that can replace LSI appears due to advances in semiconductor technology, it is also possible to use an integrated circuit based on that technology.

[0096] In addition, in the above-described embodiments, the control lines and information lines indicate those that are considered necessary for explanation, and not all control lines and information lines are necessarily shown in the product, and all components may be interconnected.

[0097] The present invention is not limited to the above-described embodiment and may be modified, and the above configurations may be replaced with substantially the same configurations, configurations that achieve the same effects, or configurations that can achieve the same purpose. [Explanation of symbols]

[0098] 101 Image processing device 111 Processing section 112 Image acquisition unit 113 Seam position acquisition section 114 Abnormality detection unit 115 Abnormality Cause Identification Department 116 Display control unit 121 Storage section 131 Display section

Claims

1. a joint position acquisition unit that acquires positions of joints between a plurality of members from an image of a structure composed of the plurality of members; an anomaly detection unit that detects an anomaly occurring in the structure and a position of the anomaly from the image; an abnormality factor identification unit that identifies a cause of the abnormality based on the position of the seam and the position of the abnormality; Equipped with The abnormality cause identification unit is Calculating the distance between the joints from the positions of the joints; Calculating a position of a seam other than the position of the seam acquired by the seam position acquisition unit based on the calculated seam interval; an image processing device that identifies a cause of the abnormality based on the position of the seam, the position of the other seam, and the position of the abnormality.

2. A joint position acquisition unit that acquires positions of joints between a plurality of members from an image of a structure composed of the plurality of members; an anomaly detection unit that detects an anomaly occurring in the structure and a position of the anomaly from the image; an abnormality factor identification unit that identifies a cause of the abnormality based on the position of the seam and the position of the abnormality; Equipped with The abnormality cause identifying unit is an image processing device that identifies a seam within a predetermined range from the position of the abnormality as a cause of the abnormality.

3. The abnormality detected by the abnormality detection unit is either a first type of abnormality that requires identification of a cause of occurrence or a second type of abnormality that does not require identification of a cause of occurrence, The image processing apparatus according to claim 1 , wherein the abnormality cause identification unit identifies a cause of the first type of abnormality among the abnormalities detected by the abnormality detection unit.

4. The image processing device according to claim 1 , wherein the abnormality factor identifying unit calculates the positions of the other seams based on a median value of the intervals between the seams.

5. Acquiring positions of joints between a plurality of members from an image of a structure composed of the plurality of members; Detecting an anomaly occurring in the structure and a position of the anomaly from the image; Identifying a cause of the abnormality based on the position of the seam and the position of the abnormality. Processing is provided. The process of specifying Calculating the distance between the joints from the positions of the joints; Calculating the position of a seam other than the position of the seam acquired in the acquiring process based on the calculated seam interval; An image processing method for identifying a cause of the abnormality based on the position of the seam, the position of the other seam, and the position of the abnormality.

6. Obtaining positions of joints between a plurality of members from an image of a structure composed of the plurality of members; Detecting an anomaly occurring in the structure and a position of the anomaly from the image; Identifying a cause of the abnormality based on the position of the seam and the position of the abnormality. Processing is provided. The identifying process is an image processing method for identifying a seam within a predetermined range from the position of the abnormality as the cause of the abnormality.

Images