Structure inspection support device, structure inspection support system, and structure inspection support program

Abstract

To provide a structure inspection support apparatus, a structure inspection support system and a structure inspection support program which clearly show a detection result of a defect spot of a structure.SOLUTION: A structure inspection support apparatus 1 comprises: a storage device which stores a discriminator that receives input of an image of a structure and outputs a probability that a defect exists in each spot of the structure corresponding to each region of the image; and a processing device which executes processing of outputting a defect probability map being data indicating a probability in each region of an image of the designated structure by inputting the image of the designated structure to the discriminator, and processing of correcting the defect probability map by specifying a distortion spot in the image of the designated structure, reducing resolution of a region corresponding to the distortion spot in the output defect probability map on the basis of the specified distortion spot and raising a probability in the region where the resolution is reduced according to a prescribed algorithm.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a structure inspection support device, a structure inspection support system, and a structure inspection support program. [Background technology]

[0002] Inspection of buildings or structures has been conventionally performed by performing image processing on captured images of the buildings or structures. In this case, the angle of view of the image capture device is generally adjusted to face the target object directly, so that the target object can be easily identified in the image. For example, when detecting abnormalities such as corrosion or cracks on a concrete wall, an image is captured at an angle of view facing the concrete wall directly.

[0003] On the other hand, with the recent advancement of drone technology, specifically the improvement of flight range (time or distance) and the increase in payload, the method of inspecting by mounting a camera on a drone is becoming more widespread (Non-Patent Document 1).

[0004] Furthermore, to improve the efficiency of inspection work, there is a technology that takes multiple images, matches feature points between the images, synthesizes a wide-area image, and then inspects the divided images before synthesis (Patent Document 1). [Prior art documents] [Non-patent literature]

[0005] [Non-Patent Document 1] "Bridge Inspection System Using Spherical Drones," Infrastructure Technology Overview, Strategic Innovation Imagination Program (SIP), January 2019 [Patent documents]

[0006] [Patent Document 1] Japanese Patent Application Publication No. 2020-126590 Summary of the Invention [Problem to be solved by the invention]

[0007] When performing inspections using a drone-mounted camera, the drone must fly stably while maintaining a certain distance from the surface to be inspected. However, when a drone is used to enter a narrow structure for inspection, the open space around the flying drone is limited, and air turbulence occurs inside the structure as the drone flies, making it difficult to maintain a certain distance from the surface to be inspected. Therefore, for example, if the shape of the enclosed structure is a vertical hole, camera photography must be performed using a hanging jig, and if it is a horizontal hole, camera photography must be performed while rotating a vehicle-type or crawler-type robot.

[0008] However, because the inside of a confined structure is narrow, it is difficult to adjust the angle of view by adjusting the jig or turning the robot.

[0009] In addition, to shorten inspection time, a camera with as wide an angle of view as possible (ideally, a 360° full-circle view) It is desirable to reduce the number of inspection images taken by using a 360-degree panoramic camera capable of taking panoramic photographs. However, when the camera's angle of view is widened, the characteristics of the lens cause structures on the outer edge of the captured image to become distorted, making it difficult to detect the object being inspected.

[0010] For example, Figure 21 shows the inspection targets of the sewer manhole and its nearby underground buried structures, as well as 1 and 2, and examples of images of the underground buried structure photographed with a standard wide-angle camera. First, (A) in the figure is a perspective view of an underground buried structure 201. As shown in the figure, this underground buried structure 201 is composed of a manhole 200 extending vertically downward from the ground surface (not shown), an inspectable room 210 having a substantially rectangular parallelepiped internal space that is connected to the lower end of the manhole 200 and its ceiling, and sewer pipes 220 and 230 that are connected to the inspectable room 210 and extend substantially horizontally to the left and right from two opposing side walls of the inspectable room 210. A footrest 211 is attached to the back wall 235 of the inspectable room 210, to which the sewer pipes 220 and 230 are not connected, allowing workers to climb up and down.

[0011] (B) in the same figure is an example of an image 240 taken by a normal wide-angle camera of the underground buried structure 201 from the front direction 250 of the inspected room 210 (the direction opposite the footrest metal fittings 211, perpendicular to the extension direction of the sewer pipes 220, 230). As shown in this image 240, a wide-angle camera can only capture a portion of the left and right sides from the front of the lens visual axis. Therefore, in order to inspect the entire inspected room 210, it is necessary to rotate the camera multiple times (for example, when the front is 0°, it is 90° to the right). The inspection work takes longer because it is necessary to take photographs (a total of four times, at horizontal rotations of 180°, 180°, and 270°).

[0012] Next, Fig. 22 shows an example of the underground buried structure 201 and an image of the underground buried structure 201 taken by a panoramic camera. As shown in (A) of the same figure, the inside of the inspected room 210 is rotated 360° on a substantially horizontal plane by a panoramic camera (360-degree camera, celestial sphere camera, or omnidirectional camera) and panoramic photographed (reference numeral 300). In this case, a panoramic image 310 such as that shown in (B) of the same figure is obtained. Specifically, the manhole 200 and the footrest 211 are photographed in the leftmost region 311 and the rightmost region 312 of this panoramic image 310. In this way, a panoramic camera has the advantage that the entire periphery of the structure to be inspected can be inspected with a single image, thereby shortening the inspection work time.

[0013] On the other hand, the upper end region 320 and the lower end region 330 of the panoramic image 310 are stretched, resulting in a loss of resolution. As a result, the defect detection results in areas with reduced resolution, resulting in "the detected position being unclear and the detected portion being less likely to be a defect (defect probability)." Therefore, when using the panoramic image 310, there is a problem in that it becomes difficult to detect an abnormality in the structure being inspected in areas with low resolution.

[0014] As a technology for performing inspections using distorted images, Patent Document 1 proposes a method for capturing multiple images, matching feature points between the captured images, synthesizing the images into a panoramic image, and detecting defects on the segmented images before the synthesis. However, this method assumes that defect detection is performed on segmented images with minimal distortion, making it difficult to perform defect detection on full-circle panoramic images captured using existing panoramic cameras, which contain significant distortion. In other words, a dedicated camera system compatible with Patent Document 1 is required, making it difficult to perform defect detection on full-circle panoramic images captured using existing general-purpose panoramic cameras.

[0015] The present invention has been made in consideration of the above circumstances, and its purpose is to provide a structure inspection support device, a structure inspection support system, and a structure inspection support program that are capable of clearly displaying the detection results of defective areas in a structure. [Means for solving the problem]

[0016] One aspect of the present invention for solving the above problem is a storage device that stores a classifier that receives an image of a structure, and outputs the probability of a defect existing in each location of the structure corresponding to each region of the image, in association with the region; and a defect probability that, when an image of a specified structure is input to the classifier, the defect probability is data indicating the probability in each region of the image of the specified structure. The structure inspection support device is equipped with a processing device that executes the following processes: a process of outputting a defect probability map; and a process of identifying distorted locations in an image of the specified structure, reducing the resolution of the area of ​​the distorted location in the output defect probability map based on the identified distorted locations, and increasing the probability in the area with reduced resolution in accordance with a predetermined algorithm, thereby correcting the defect probability map. [Effects of the Invention]

[0017] According to the present invention, the detection results of defective portions of a structure can be clearly displayed. [Brief explanation of the drawings]

[0018] [Figure 1] 1 is a diagram illustrating an example of the configuration of a structure inspection support device according to a first embodiment. [Figure 2] FIG. 4 is a flowchart illustrating an example of a structure inspection support process according to the first embodiment. [Figure 3] FIG. 10 is a diagram showing an example of defect learning image data. [Figure 4] FIG. 10 is a diagram showing an example of inspection image data input to a classifier. [Figure 5] FIG. 10 is a diagram illustrating an example of defect probability map data. [Figure 6] FIG. 10 is a diagram showing an example of an image showing the detection result of defect portions in an underground buried structure when no correction is made by the defect distribution display process. [Figure 7] FIG. 10 is a flowchart illustrating details of a defect distribution display process. [Figure 8] FIG. 10 is a diagram illustrating an example of image distortion data. [Figure 9] 10A and 10B are diagrams illustrating an example of a change in resolution of an inspection target image when the resolution is changed based on grid information. [Figure 10] FIG. 10 is a diagram showing an example of a defect detection result screen according to the first embodiment. [Figure 11] FIG. 10 is a diagram illustrating an example of a hardware configuration and functions of a structure inspection support device according to a second embodiment. [Figure 12] FIG. 10 is a flowchart illustrating an example of a structure inspection support process according to the second embodiment. [Figure 13] FIG. 10 is a diagram showing an example of inspection image data that is read by the structure inspection support device in the defect probability inference process in the second embodiment. [Figure 14] FIG. 10 is a flowchart illustrating an example of a defect distribution display process according to the second embodiment. [Figure 15] FIG. 10 is a flowchart illustrating details of mixed resolution conversion processing. [Figure 16] FIG. 10 is a diagram illustrating an example of distance data. [Figure 17]FIG. 10 is a diagram illustrating an example of a change in image resolution due to a resolution conversion process according to the second embodiment. [Figure 18] FIG. 10 is a diagram showing an example of a defect probability map output by the defect probability inference process according to the second embodiment. [Figure 19] FIG. 10 is a diagram showing an example of a defect detection result screen displayed by the defect distribution display processing according to the second embodiment. [Figure 20] FIG. 10 is a diagram illustrating an example of the configuration of a structure inspection support system according to a third embodiment. [Figure 21] 1A and 1B are diagrams showing examples of an image of a sewer manhole and an underground buried structure in the vicinity thereof as an inspection target, and an image of the underground buried structure photographed with a wide-angle camera. [Figure 22] 1A and 1B are diagrams showing examples of an underground buried structure and an image of the underground buried structure photographed by a panoramic camera. DETAILED DESCRIPTION OF THE INVENTION

[0019] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

[0020] [Example 1] FIG. 1 is a diagram showing an example of the configuration of a structure inspection support device 1 according to a first embodiment. The structure inspection support device 1 is an information processing device that supports the inspection of defects, damage, etc. in structures having closed spaces such as water pipes and manholes (hereinafter referred to as target structures). In addition to manholes and manholes, for example, structures with pipe-like tubular sections such as tunnels or plant structures, or structures with narrow spaces, are also examples of such structures.

[0021] The structure inspection support device 1 uses a trained model (hereinafter referred to as a classifier) ​​that inputs a photographed image of a target structure and outputs estimated information on defective areas present in the target structure, and performs specified image processing on the output results of the trained model to present defects in the target structure so that the user does not overlook them.

[0022] The image of the target structure is, for example, a panoramic image taken by a photographing device such as a camera (such as a camera mounted on a drone) that has entered the target structure. In this embodiment, this image is taken by a 360-degree panoramic camera, and distortion occurs at the edges of the panoramic image due to a predetermined panoramic synthesis process. In this embodiment, the target structure is the underground buried structure 201 described in Figures 21 and 22. The camera that photographs the target structure may be a 360-degree camera, or may be, for example, a fisheye camera or a wide-angle camera.

[0023] The structure inspection support device 1 is an information processing device including a processing device 1930, a storage device 1940, an input device 1920, and an output device 1910. The processing device 1930 includes a CPU (Central Processing Unit) The input device 1940 is composed of a main memory (memory) such as a ROM (Read Only Memory) or a RAM (Random Access Memory), and an auxiliary memory such as a HDD (Hard Disk Drive) or an SSD (Solid State Drive). 20 is a keyboard, a mouse, or a disk drive to which an external storage medium is connected.

[0024] 1, the output device 1910, the input device 1920, the processing device 1930, and the storage device 1940 are shown as independent devices, but some or all of these may be configured as an integrated device. For example, the structure inspection support device 1 may be a notebook personal computer (PC) in which all of the devices are integrated, It may be a desktop PC that integrates only the processing device 1930 and the storage device 1940. Alternatively, it may be an independent device configuration in which the processing device 1930 is a microcomputer board, the storage device 1940 is an external HDD, the output device 1910 is an external display, and the input device 1920 is an external keyboard. Next, the structure inspection support device 1 stores the defect learning image data D100, the defect learning result data D110, the inspection image data D120, the defect probability map data D130, and the image distortion data D140.

[0025] The defect image learning data D100 is data in which images used to create a classifier (hereinafter referred to as learning images) and their correct labels (true values) are stored in association with each other.

[0026] The defect learning result data D110 is data that constitutes the classifier.

[0027] The inspection image data D120 is image data of the target structure. The inspection image data D120 may be, for example, image data of the target structure transmitted to the structure inspection support device 1 by an information processing device other than the structure inspection support device 1 or an imaging device such as a camera.

[0028] The defect probability map data D130 includes image data (hereinafter also referred to as a defect probability map) that indicates the probability (hereinafter referred to as defect probability) of a defect existing at each location (each pixel in the inspection image data D120) of the target structure, which is output by inputting the inspection image data D120 into the defect learning result data D110.

[0029] The image distortion data D140 is data that indicates the distribution of the locations and magnitudes of distortions in the image according to the characteristics of the lens of the camera that captured the image of the target structure. Specifically, the image distortion data D140 is a collection of data on the resolution set for each region of the defect probability map. The resolution differs depending on the location of the corresponding distortion in the image.

[0030] Next, the defect learning processing unit 111 creates a classifier. That is, the defect learning processing unit 111 receives an image of a structure and creates a classifier that outputs the probability that a defect exists in a location of the structure corresponding to each region of the image (defect probability).

[0031] The defect probability inference processing unit 112 inputs an image of the target structure (inspection image data D120) into a classifier, and outputs defect probability map data D130, which is data including the defect probability for each pixel in the image of the target structure.

[0032] The defect distribution display processing unit 113 identifies the location and magnitude of distortion of the subject (target structure) in the image of the target structure based on the image distortion data D140.

[0033] Then, based on the information on the identified distortion location, the defect distribution display processing unit 113 corrects the defect probability map data D130 by lowering the resolution of the area corresponding to the distortion location in the defect probability map data D130 output by the defect probability inference processing unit 112 and increasing the defect probability related to the area with the lowered resolution according to a predetermined algorithm.

[0034] In this embodiment, the defect distribution display processing unit 113 reduces the resolution by treating multiple pixels that make up an area corresponding to a distorted location in the defect probability map data D130 output by the defect probability inference processing unit 112 as a new pixel (grid), and increases the defect probability by setting the sum of the defect probabilities of the pixels in the grid as the defect probability of that grid.

[0035] Then, based on the defect probability map data D130, the defect distribution display processing unit 113 outputs information on defects at each location of the target structure to the output device 1910. Specifically, the defect distribution display processing unit 113 outputs to the output device 1910, at the reduced resolution, the resolution of the area in the defect probability map data D130 where the result probability has been increased, having pixel values ​​corresponding to the defect probability values.

[0036] The functions of each functional unit of the structure inspection support device 1 described above are realized by the processing device 1930 reading out a predetermined program that realizes each functional unit from the storage device 1940. Each program can be recorded on, for example, a portable or fixed recording medium and distributed. All or part of these programs may be realized using virtual information processing resources provided using virtualization technology, process space separation technology, or the like, such as a virtual server provided by a cloud system. All or part of these programs may be realized by a service provided by a cloud system via an API (Application Programming Interface), for example. Next, the processing performed by the structure inspection support device 1 will be described.

[0037] <Structure inspection support processing> 2 is a flow diagram illustrating an example of a structure inspection support process that is a process for supporting an inspection work for defects in a target structure according to Example 1. The structure inspection support process is started, for example, when a predetermined input is made to the structure inspection support device 1 by a user.

[0038] First, the defect learning processing unit 111 reads the learning images and their correct labels recorded in the defect learning image data D100, and performs machine learning to create a classifier. S100 is executed. The defect learning processing unit 111 stores the classifier output by the defect learning processing S100 in the defect learning result data D110.

[0039] The machine learning algorithm of the classifier is, for example, a known classification algorithm. In this embodiment, an image region division algorithm (segmentation algorithm) is used to create the classifier. Specifically, a deep learning (hereinafter referred to as DNN: Deep Neural Network) segmentation algorithm is adopted as the machine learning algorithm of the classifier, and the classifier learns, on a pixel-by-pixel basis within the image, whether each pixel belongs to a "defect class" or a "normal class" described below.

[0040] In this embodiment, the correct label for each pixel that is trained by the classifier is a label value indicating a "defect class" (for example, when the pixel value of the corresponding pixel is 255) or a label value indicating a "normal class" (for example, when the pixel value of the corresponding pixel is 0). In this embodiment, the classifier also outputs the defect probability of each pixel along with each label value of each pixel. Specifically, the classifier outputs the probability (0% to 100%) that the pixel falls into the defect class, obtained in the layer immediately before the final layer, in addition to the class label value output by the final output layer of the DNN.

[0041] Note that, as a machine learning algorithm other than the segmentation algorithm, an algorithm that learns and infers the defect probability for each pixel may be adopted. In this case, the true value (correct label) that the classifier learns is the defect probability value for each pixel. For example, a pixel value of 0 represents a defect probability of 0%, a pixel value of 127 represents a defect probability of 50%, and a pixel value of 255 represents a defect probability of 100%, and the defect probability image expressed in 256 8-bit gradations per pixel is learned as the true value.

[0042] As described above, in this embodiment, the number of classes of the classifier is set to two (two classes of normal or defect), the learning algorithm of the classifier is set to a defect probability learning algorithm, and the correct label is set to a pixel value (0 to 255 gradations) and a defect probability (0% to 100%).

[0043] (Defect learning image data) 3 is a diagram showing an example of the defect learning image data D100. The defect learning image data D100 is data formed by combining a feature amount image 410 and a true value image 420.

[0044] The feature image 410 is image data of a learning image captured by a panoramic camera of a target structure. This feature image 410 has, as defect areas, a side crack region 430 corresponding to a crack that has occurred on the side of the inspected room 210 of the underground buried structure 201, which is the target structure, and a ceiling surface crack region 440 corresponding to a crack that has occurred on the ceiling surface of the inspected room 210; the rest of the image is normal. The side crack region 430 is located in the center of the feature image 410 on the visual axis of the panoramic camera, and the ceiling surface crack region 440 is located in the upper part of the feature image 410, off the visual axis of the panoramic camera.

[0045] In this embodiment, it is assumed that the cracks corresponding to the side crack region 430 and the ceiling crack region 440 are actually both linear, elongated cracks of uniform thickness. However, even in this case, the shapes of the side crack region 430 and the ceiling crack region 440 on the feature image 410 are different from each other. That is, the side crack region 430 on the visual axis of the panoramic camera is a region of uniform thickness, while the ceiling crack region 440 located off the visual axis of the panoramic camera is a linear region with a gradation that becomes thicker and thinner as it approaches its edge. This is because the edge of the image is stretched and distorted during the process of combining images using the panoramic camera.

[0046] Next, the true value image 420 is image data that represents the true value (correct label) of the defect probability. Specifically, the true value image 420 is image data having the same image size and coordinate system as the feature image 410. The pixel value of each pixel in the true value image 420 is the true value of the defect probability of the part of the underground structure 201 indicated by the pixel in the feature image 410 corresponding to that pixel.

[0047] For example, if the pixel value of a pixel in the true value image 420 is large (i.e., the pixel value is close to 255 and close to white), there is a high probability that the part of the underground buried structure 201 corresponding to that pixel is a normal part, and if the pixel value of a pixel in the true value image 420 is small (i.e., the pixel value is close to 0 and close to black), there is a high probability that the part of the underground buried structure 201 corresponding to that pixel is a defective part.

[0048] Furthermore, areas with low pixel values ​​in the true image 420 (areas corresponding to defective areas) have a gradation pattern that approaches black from white from the center toward the periphery. This means that when the boundary between the defective area and the normal area is unclear, the probability of a defect decreases near the boundary. For example, considering the cross section of a crack in the wall of the inspected room 210, the depth of the crack from the wall is deep at the center in the axial direction and becomes shallower toward the periphery. Therefore, in the area corresponding to the crack in the true image 420, the color of the pixel corresponding to the axial center of the crack is white (high probability of being a crack), while the color of the pixel approaches black, the color of the normal wall surface, as it moves from the center in the axial direction toward the periphery (low probability of being a crack).

[0049] 3, the pixel values ​​(true values) of the side crack region 460 are high (white) in the region at the axis center, and gradually decrease (become black in a gradational fashion) as the pixel values ​​move away from the axis center. On the other hand, the pixel values ​​(true values) of the ceiling surface crack region 470 gradually decrease (become black in a gradational fashion) as the pixel values ​​move away from the axis center, and at the same time, become even lower as the pixel values ​​move closer to the edge, resulting in a complex gradational numerical distribution.

[0050] Next, as shown in FIG. 2, the defect probability inference processing unit 112 outputs a defect probability map by inputting the inspection image data D120 into a classifier of the defect result learning data D110, and stores the output defect probability map in defect probability map data D130 (S110).

[0051] (Inspection image data) FIG. 4 is a diagram showing an example of inspection image data D120 input to the classifier. The inspection image data D120 consists of one or more inspection target images 510, 520 (no true value, which is a correct label, is set). The data format of the inspection target images 510, 520 is the same as that of the feature image 410 in the defect learning image data D100. Unlike the feature image 410 in the defect learning image data D100, the inspection target images 510, 520 may not include a defective portion. Even if a defective portion is included in an image, the position and shape of the defective portion may not be the same as those in the feature image 410.

[0052] 4, the first inspection target image 510 has a side crack region 540 corresponding to a crack that has occurred on the side of the inspection target room 210, and a ceiling surface crack region 530 corresponding to a crack that has occurred on the ceiling surface of the inspection target room 210. In addition, the second inspection target image 520 has a ceiling surface crack region 550 corresponding to a crack that has occurred on the ceiling surface of the inspection target room 210.

[0053] (Defect probability map data) 5 is a diagram showing an example of defect probability map data D130. The defect probability map data D130 is image data that indicates the defect probability of the inspection target image for each pixel.

[0054] The defect probability map data D130 includes one or more defect probability maps 610, 620. 5, a first defect probability map 610 represents the defect probability of the first inspection target image 510, and a second defect probability map 620 represents the defect probability of the second inspection target image 520. In the example of FIG.

[0055] Specifically, the pixel value of each pixel in the defect probability maps 610, 620 is the defect probability of the part of the underground buried structure 201 indicated by the pixel in the inspection target images 510, 520 corresponding to that pixel, just like the true value image 420. When the pixel value is large (close to white), there is a high probability that the part of that pixel is a defective part, and when the pixel value is small (close to black), there is a high probability that the part of that pixel is a normal part.

[0056] In the example of FIG. 5 , the first defect probability map 610 has a first defect portion region 630 corresponding to the ceiling surface crack region 530 in the first inspection target image 510 and a second defect portion region 640 corresponding to the side surface crack region 540 in the first inspection target image 510. The second defect probability map 620 has a third defect portion region 670 corresponding to the ceiling surface crack region 550 in the second inspection target image 520. Note that some of the defect probabilities in the defect probability maps 610 and 620 may not be accurately detected as defect portions due to the classification performance of the classifier. In the example of FIG. 5 , the second defect portion region 640 in the first defect probability map 610 includes a missing portion 650, which is an overlooked defect portion, and a false detection portion 660, which is an erroneously recognized portion outside the second defect portion region 640. At the same time, a missing portion 680 and a false positive portion 690 also exist in a third defect portion region 670 in the second defect probability map 620.

[0057] Here, the pixel values ​​(defect probabilities) of the upper portion 631 of the first defect site region 630 of the first defect probability map 610 are small (close to black) due to distortion at the image edge in the first inspection target image 510.

[0058] FIG. 6 shows an example of an image showing the results of detecting defect sites in the underground buried structure 201 based on the defect probability maps 610 and 620 (without correction by the defect distribution display process S120, which will be described later). This image 700 is an image obtained by superimposing the first inspection target image 510 and the first defect probability map 610. The resolution of each region in this image 700 is the same as that of the first inspection target image 510 and the first defect probability map 610. Therefore, the region corresponding to the upper portion 631 of the first defect site region 630, which has a low defect probability (specifically, the upper end 711 of the ceiling surface crack 710), is not displayed sufficiently as a defect (for example, displayed in a very light color), resulting in poor visibility for the user. In other words, the user may miss part of the inspection of the cracks occurring in the ceiling surface of the inspected room 210.

[0059] Therefore, as shown in FIG. 2, the defect distribution display processing unit 113 reads the defect probability map data D130, creates image data with a resolution different from that of the image to be inspected, and executes the defect distribution display processing S120, which displays an image showing the defect detection results on the screen based on this image data.

[0060] <Defect distribution display processing> 7 is a flow diagram illustrating the details of the defect distribution display process S120. First, the defect distribution display processing unit 113 reads the image distortion data D140 and executes a resolution conversion process S910 to correct the resolution of the inspection target images 510 and 520 in the inspection image data D120.

[0061] (Image distortion data) 8 is a diagram showing an example of the image distortion data D140. The image distortion data D140 is a resolution multiplier that is information on the resolution set for each region of each of the inspection target images 510 and 520. The resolution magnification definition table 810 includes a resolution magnification definition table 810. The resolution magnification definition table 810 has the following data items: an area ID 811 in which an area identifier is set; an X-axis start point 812 in which the start point of the X coordinate of that area in the inspection target images 510 and 520 is set; an X-axis end point 813 in which the end point of the X coordinate of that area in the inspection target images 510 and 520 is set; a Y-axis start point 814 in which the start point of the Y coordinate of that area in the inspection target images 510 and 520 is set; a Y-axis end point 816 in which the end point of the Y coordinate of that area in the inspection target images 510 and 520 is set; and a magnification 817 in which a resolution magnification specifying the resolution of that area is set. Note that the image distortion data D140 is set in advance by a user, for example, depending on the characteristics of the camera used to capture the inspection target images.

[0062] The resolution magnification is a magnification relating to the number of X and Y axis grids (pixels) that make up an area, with an area of ​​magnification 1 as the base. For example, the number of pixels in the area with area ID=0 in the figure is twice the number of pixels in the area with area ID=1 (i.e., the area with area ID=1 is made up of (X axis: 2 pixels) x (Y axis: 2 pixels) = (2) x (2) pixels as one unit). The greater the resolution magnification, the greater the distortion in that area.

[0063] In this embodiment, the image distortion data D140 is in a data format that defines multiple rectangular regions and the resolution magnification for each rectangular region, but other data formats may be used. For example, a method may be used in which a default magnification for the entire screen is set to 1, a closed figure of any shape is identified, and a predetermined resolution magnification is specified within the closed figure. As yet another example, calibration may be performed by attaching a calibration marker (e.g., a chessboard pattern) in advance, and a correspondence coordinate map may be obtained between the coordinates of the distorted inspection target images 510 and 520 and the coordinates in the undistorted images, thereby defining the resolution of each region in the inspection target images 510 and 520. The image distortion data D140 may also include information on the resolution magnification for each pixel.

[0064] To explain an example of the resolution conversion process S910 based on the image distortion data D140 in Figure 8, the defect distribution display processing unit 113 identifies distortion points in the inspection target images 510 and 520, which have a resolution of 512 x 256, based on the image distortion data D140, and creates information about the resolution of the image (hereinafter referred to as grid information) in which a resolution corresponding to the identified distortion points is set.

[0065] The grid information is information that sets a new pixel area (grid) consisting of a predetermined number of pixels in an area corresponding to a distorted portion in an image. 8, the grid information defines the upper region of the image (x,y)=(0,0) to (511,19) (20 vertical pixels) with a resolution of 512×256 as a low-resolution region (distortion region) with 2×2 pixels as one unit (grid), the central region of the image (x,y)=(0,20) to (512,234) as a high-resolution region with 1×1 pixel as one unit (grid), and the lower region of the image (x,y)=(0,235) to (511,255) (20 vertical pixels) as a low-resolution region (distortion region) with 2×2 pixels as one unit (grid). That is, the defect distribution display processing unit 113 sets low-resolution regions at the upper and lower ends of the inspection target images 510 and 520 in accordance with distortion at the ends of the captured images due to the characteristics of the panoramic camera.

[0066] 9 is a diagram showing an example of a change in the resolution of the inspection target image when the resolution is changed using grid information. The grids of the inspection target image 1000 before the change are the same in all regions. In the changed image 1010, based on the image distortion data D140, the grids in the central portion 1020 have the same high resolution as the inspection target image 1000 before the change, and the grids in the upper portion 1030 and lower portion 1040 have a lower resolution than the inspection target image 1000 before the resolution change. Specifically, the size of this low-resolution grid is twice as large in both the vertical and horizontal directions as the high-resolution grid.

[0067] 7, the defect distribution display processing unit 113 identifies one of the low-resolution areas in the grid information (S920). The defect distribution display processing unit 113 creates a mask image for making the selected low-resolution area a target of the following processes S940 to S960 (S930).

[0068] Then, the defect distribution display processing unit 113 reads the defect probability map data D130, and calculates (aggregates) the defect probability of the low-resolution area in the defect probability map for each of the grids consisting of multiple pixels based on the created mask image (S940).

[0069] For example, for a low-resolution area with a resolution magnification of 2, since the grid (one unit) in that area is 2x2 pixels, the defect distribution display processing unit 113 determines the defect probability of that grid by adding up the defect probability values ​​of a total of four pixels within that grid.

[0070] Then, the defect distribution display processing unit 113 creates a new defect probability map with reduced resolution by setting the sum of the defect probabilities of the grids recalculated in s940 as the defect probability in the new pixel (i.e., a grid consisting of multiple pixels) (S950).

[0071] The defect distribution display processing unit 113 checks whether there is a low-resolution area that has not been identified so far (S960), and if there is a low-resolution area that has not been identified so far, repeats the processing of s920 to select that area.

[0072] By the above process, a new defect probability map with a lower resolution but an increased defect probability is set in all low-resolution regions.

[0073] Next, the defect distribution display processing unit 113 creates a mask image for processing the high-resolution area in the grid information, and based on the created mask image, sets the defect probability of the high-resolution area in the (original) defect probability map as the defect probability of the high-resolution area in the new defect probability map (S970).

[0074] Through the above processing, the defect distribution display processing unit 113 creates a new defect probability map by correcting the original defect probability map, which includes a high-resolution area based on the defect probability output by the classifier and a low-resolution area with an increased defect probability.

[0075] In this embodiment, the defect probability of a grid in a low-resolution area is calculated by simply adding up the defect probabilities of each pixel in the grid (2x2 pixels) in the low-resolution area. However, other calculation algorithms may be used to increase the defect probability. For example, the defect probability may be calculated by identifying the maximum defect probability among the defect probabilities in the grid and setting the defect probabilities of all pixels in the grid to that maximum defect probability. Furthermore, the position of each pixel in the grid may be weighted by normalization or the like (e.g., by setting a weight distribution such that the center of the grid has the maximum value), and the defect probability may be calculated using a value (e.g., a multiplication value) based on the weighted value at each position and each defect probability.

[0076] Then, the defect distribution display processing unit 113 superimposes the inspection object image and the new defect probability map created in the processing up to S970, and displays them on a defect detection result screen (S980), which will be described next.

[0077] At this time, the defect distribution display processing unit 113 may highlight a portion with a high defect probability. For example, the defect distribution display processing unit 113 identifies an area in the defect probability map that has a defect probability equal to or greater than a predetermined value, and displays a figure (circumscribing figure) that indicates the periphery of that area. Furthermore, the brightness of the pixels within the circumscribing figure is changed in accordance with the value of the defect probability.

[0078] (Defect detection result screen) 10 is a diagram showing an example of a defect detection result screen 1101 according to Example 1. This defect detection result screen 1101 is a screen in which the corrected first defect probability map is superimposed on an image 1100 of the target structure corresponding to the first inspection target image 510. The image 1100 of the target structure has a high-resolution region 1110, which is a region with the same resolution as the original defect probability map, and a low-resolution region 1120 set by the resolution conversion process S910.

[0079] The image 1100 of the target structure has a first defect area 1130 (corresponding to a side crack) corresponding to the second defect area area 640, and a second defect area 1140 and a third defect area 1150 (corresponding to a ceiling surface crack) corresponding to the first defect area area 630. The first defect area 1130 and the second defect area 1140 are displayed superimposed on the high-resolution area 1110 of the image 1100 of the target structure, and the third defect area 1150 is displayed superimposed on the low-resolution area 1120 of the image 1100 of the target structure.

[0080] A first defect region 1130 corresponding to a side crack is located within the high-resolution region 1110, and its resolution is the same as the resolution of the defect probability map detected by the classifier. Therefore, the resolution of the first defect region 1130 is the same as the resolution and shape of the second defect region 640 (including the missing region 650 and the false positive region 660) in the first defect probability map 610. The defect probability of the first defect region 1130 is also the same as the defect probability detected by the classifier.

[0081] On the other hand, the area corresponding to the ceiling surface crack is an area consisting of second defect area 1140 and third defect area 1150. Of these, second defect area 1140 is within high-resolution area 1110, and therefore its resolution is as high as the resolution of first defect site area 630 of first defect probability map 610, and second defect area 1140 is displayed in detail.

[0082] However, because the third defect area 1150 is in the low resolution area 1120, its resolution is lower than that of the first defect site area 630 of the first defect probability map 610 (resolution magnification is twice), and the shape of the third defect area 1150 is not displayed as precisely as the second defect area 1140.

[0083] However, as a result of the defect distribution display process S120, the defect probability of the third defect area 1150 has been corrected (defect probability with 2 x 2 pixels as one unit), so the defect probability of the third defect area 1150 is particularly high, and is displayed clearly in white on the defect detection result screen 1101, making it easy for the user to see.

[0084] In other words, defects at the edge of an image, such as the third defect area 1150, would normally be detected weaker than they actually are due to the characteristics of the camera, but by correcting the corresponding defect probability through the defect distribution display process S120, the user can more easily grasp defects at the edge of an image.

[0085] The device that displays the defect detection result screen as described above may be the structure inspection support device 1 or another information processing device that has transmitted image data of the target structure. For example, the structure inspection support device 1 creates data for displaying the defect detection result screen and transmits it to another information processing device. This allows the defect detection results to be provided to various users.

[0086] [Example 2] The structural inspection support device 1 of the second embodiment performs appropriate image processing for distortions in the image of the object to be inspected, not only static distortions such as lens distortion, but also distortions that occur dynamically depending on the conditions at the time of shooting, such as the distance between the camera and the subject.

[0087] 11 is a diagram illustrating an example of the hardware configuration and functions of a structure inspection support device 2 according to the second embodiment. The hardware configuration of the structure inspection support device 2 according to the second embodiment is the same as that of the first embodiment.

[0088] Here, the structure inspection support device 2 of the second embodiment stores distance data D150. The distance data D150 is image data that is a collection of data indicating the distance between each part of the inspection object and the camera. The distance data D150 is associated with the inspection image data D120.

[0089] The structure inspection support device 2 of the second embodiment also includes a mixed resolution conversion processing unit 114. As in the first embodiment, the mixed resolution conversion processing unit 114 changes the resolution of the defect probability map data D130 using the image distortion data D140, and then, based on the distance data D150, reduces the resolution of each region in the defect probability map data D130 in accordance with each distance indicated by the distance data D150, and increases the defect probability for the region with the reduced resolution, thereby creating further modified defect probability map data D130. Next, a structure inspection support process in the second embodiment will be described.

[0090] <Structure inspection support processing> 12 is a flow diagram illustrating an example of a structure inspection support process according to Example 2. The defect learning process S100 and the defect probability inference process S110 in this structure inspection support process are the same as those in Example 1. Thereafter, the defect distribution display processing unit 113 executes a defect distribution display processing process S141.

[0091] 13 is a diagram showing an example of inspection image data D120a read by the structure inspection support device 1 in the defect probability inference process S110 in Example 2. The inspection target image 1300 in this inspection image data D120a is an image captured showing cracks 1310 and 1320 occurring on the inner surfaces of the left and right sewer pipes 220 connected to the inspected room 210. Each of the cracks 1310 and 1320 extends from the connection between the inspected room 210 and the sewer pipe 220 in the extension direction of the sewer pipe 220 (i.e., from the front side to the back side).

[0092] 14 is a flow diagram illustrating an example of the defect distribution display process S141 of the embodiment 2. The mixed resolution conversion processing unit 114 executes the mixed resolution conversion process S1510. The subsequent processes of S920 to S980 are the same as those of the defect distribution display process S140 of the embodiment 1.

[0093] <Mixed resolution conversion processing> FIG. 15 is a flowchart illustrating the mixed resolution conversion process S1510 in detail.

[0094] First, the mixed resolution conversion processing unit 114 executes the same resolution conversion processing S910 as in the first embodiment. That is, the mixed resolution conversion processing unit 114 sets grid information using the image distortion data D140.

[0095] Next, based on the distance data D150, the mixed resolution conversion processing unit 114 performs posterization to modify the distance image 1410 in the distance image data D150 into an image with N gradations according to the distance between each part of the inspection object and the camera, resulting in an image that includes a bright area and one or more dark areas (S1610).

[0096] (Distance data) 16 is a diagram showing an example of distance data D150. The distance data D150 consists of one or more distance images 1410. The distance image 1410 shown in the figure is image data corresponding to the inspection target image 1300 of the inspection image data D120a.

[0097] Specifically, while the inspection target image 1300 is a visible light color image captured by a camera, the distance image 1410a is image data that represents the distance between the camera and each part of the object in grayscale. The pixel value of each pixel in the distance image 1410a reflects the distance between the camera and the part of the underground structure 201 indicated by the corresponding pixel in the inspection target image 1300. For example, white parts in the distance image 1410 are closer to the camera, and black parts are farther from the camera. For example, areas 1420 and 1430 in the distance image 1410 that correspond to the sewer pipe 220 of the underground structure 201 are black because the internal space of the sewer pipe 220 extends farther (from the front side to the back side).

[0098] In this embodiment, the distance image 1410 is acquired simultaneously with the image of the object to be inspected when photographing with a camera (for example, using a photographing device capable of acquiring distance data, such as a ToF (Time-of-Flight) camera or a stereo camera), but it may also be acquired by other means. For example, it may be three-dimensional point cloud data acquired by a three-dimensional laser scanner attached to a 360-degree panoramic camera.

[0099] Although the example shown here stores the data of the distance between the camera and the subject as an image, other data formats are also possible. For example, the distance data may be a two-dimensional array of data that stores the numerical value of the distance between the camera and the subject in real space for each pixel.

[0100] As a specific example of S1610, when N=3, the mixed resolution conversion processor 114 ternarizes each pixel value of the 256-level grayscale distance image 1410 (pixel value = either 255, 127, or 0). When N=2, the mixed resolution conversion processor 114 binarizes each pixel value of the distance image 1410 (pixel value = either 255 or 0). In this embodiment, the mixed resolution conversion processor 114 performs binarization (N=2), and the pixel values ​​of the black areas 1420 and 1430 of the sewer pipe in the distance image 1410 of FIG. 16 are set to 0, and the grayscale pixel value of the other areas is set to 255. Note that, hereinafter, the brightest areas (e.g., pixel value 255) are referred to as bright areas, and the other areas (e.g., pixel value 0) are referred to as dark areas.

[0101] The mixed resolution conversion processing unit 114 selects the brightest area from among the dark areas in the distance image 1410 that was output after posterization in S1610 (S1620).

[0102] The mixed resolution conversion processing unit 114 creates a mask image (S1630) for making the dark area of ​​the selected distance image 1410 the target of the following processing in S1640 and S1650.

[0103] Then, based on the created mask image, the mixed resolution conversion processing unit 114 reduces the resolution of the selected dark area in the distance image by a predetermined value (in this embodiment, the resolution magnification is reduced by 1), and adds information about the dark area with the reduced resolution to the grid information (S1640, S1650). Based on this grid information, the defect probability map is corrected (S950).

[0104] If there are no unselected dark areas, the mixed resolution conversion processing unit 114 ends the mixed resolution conversion processing S1510. If there are unselected dark areas, the mixed resolution conversion processing unit 114 selects the next brightest dark area after the currently selected area and repeats the processing from S1630 onwards.

[0105] Through the above processing, the mixed resolution conversion processor 114 can set lower resolution areas for darker areas (i.e., distant areas) in the image displayed on the defect detection result screen. For example, when N=2 as in this embodiment, the mixed resolution conversion processor 114 sets a resolution one level lower than that for bright areas (pixel value 255) for areas of the image corresponding to dark areas (pixel value 0) in the distance image 1410. When N=3, the mixed resolution conversion processor 114 sets a resolution one level lower than that for bright areas (pixel value 255) for areas of the image corresponding to areas of intermediate brightness (pixel value 127) in the distance image 1410, and sets a resolution two levels lower than that for bright areas (pixel value 255) for areas of the image corresponding to dark areas (pixel value 0).

[0106] FIG. 17 is a diagram illustrating an example of a change in image resolution by the resolution conversion process S910 according to the second embodiment.

[0107] The grid of the image 1700 without resolution conversion is the same high-resolution grid in all areas. In the image 1710 after the resolution change, the upper part 1030 and the lower part 1040 are low-resolution grids, as in the first embodiment.

[0108] On the other hand, the central portion 1020 of the image 1710 after resolution conversion is a high-resolution grid that is the same as the image 1700 without resolution conversion. However, in the image 1710 after resolution conversion, crack areas 1720 and 1730 corresponding to the cracks 1310 and 1320 on the inner surfaces of the left and right sewer pipes 220 of the underground buried structure 201 have become low-resolution areas due to the mixed resolution conversion process S1510.

[0109] Next, an example of a defect detection result screen displayed in the second embodiment will be described. First, FIG. 18 is a diagram illustrating an example of a defect probability map output by the defect probability inference process S110 according to the second embodiment.

[0110] This defect probability map 1810 has a first defect area 1820 corresponding to the crack 1310 on the inner surface of the left sewer pipe 220 in the inspection target image 1300, and a second defect area 1830 corresponding to the crack 1320 on the inner surface of the right sewer pipe 230 in the inspection target image 1300. The first defect area 1820 and the second defect area 1830 have a gradation that gradually becomes darker toward the top of the image (in real space, the pipe axis depth direction of the sewer pipes 220, 230, which is the direction of the tips of the cracks 1310, 1320). This is because the interior of the pipe appears smaller in the inspection target image 1300 and the defect probability decreases as it moves toward the pipe axis depth direction of the sewer pipes 220, 230. In this case, the first defect area 1820 and the second defect area 1830 have poor visibility, so the user may overlook them.

[0111] (Defect detection result screen) Next, FIG. 19 is a diagram showing an example of a defect detection result screen 1101 displayed by the defect distribution display processing S1401 according to the second embodiment.

[0112] An image 1840 corresponding to the image 1710 after the resolution conversion is displayed on the defect detection result screen 1101. That is, the resolution of crack detection areas 1850 and 1860 corresponding to the crack areas 1720 and 1730, which are the central part of this image 1840, is reduced.

[0113] Because the resolution is low, the defect probability of each pixel in the center of the image 1840 is corrected by the defect distribution display process S141. Therefore, the defect probability of the regions 1870 and 1880 in the crack detection regions 1850 and 1860 corresponding to the cracks 1310 and 1320 in the image 1710 after the resolution conversion is high, and the regions are clearly white. It will be displayed.

[0114] In this way, in the second embodiment, the distance data D150 reduces the image resolution of the portion of the subject that is far from the camera, and as a result, the defect probability of that portion is also increased and corrected. This makes it possible to clearly indicate to the user the presence of a defect (in white in this embodiment) even in a portion that is far from the camera and therefore would normally be weakly detected as a defect.

[0115] [Example 3] In the third embodiment, the device configuration is different from that in the first embodiment. That is, in the third embodiment, a structure inspection support system including a plurality of devices is constructed by dividing the learning function and the defect detection function of the classifier in the structure inspection support device 1 in the first embodiment. This makes it easier to create a classifier and detect defects independently, and for example, it is possible to provide the latest classifier to the user while improving user convenience.

[0116] 20 is a diagram illustrating an example of the configuration of a structure inspection support system 3 according to Example 3. This structure inspection support system 3 includes a learning computer 2015 (first information processing device) and an inspection computer 2055 (second information processing device).

[0117] The training computer 2015 is an information processing device managed by a trainee 2000 who is in charge of creating a classifier. The training computer 2015 includes a processing device 2020, a storage device 2025, and a communication device 2030 similar to those in the first embodiment. An output device 2005 and an input device 2010 used by the trainee 2000 are connected to the training computer 2015.

[0118] The inspection computer 2055 is an information processing device managed by an inspector 2040 who is in charge of inspection work. The inspection computer 2055 includes a processing device 2060, a storage device 2065, and a communication device 2070 similar to those in the first embodiment. An output device 2045 and an input device 2050 used by the inspector 2040 are connected to the inspection computer 2055.

[0119] The Learning Computer 2015 and the Inspection Computer 2055 are connected via the Internet, LAN (Local Area Network), WAN (Wide Area Network), or dedicated line. are communicatively connected by a wired or wireless network 2035.

[0120] The learning computer 2015 includes a defect learning processing unit 111 similar to that in the first embodiment. The learning computer 2015 also stores the defect learning image data D100 and the defect learning result data D110 similar to those in the first embodiment.

[0121] The inspection computer 2055 includes the defect probability inference processing unit 112 and the defect distribution display processing unit 113 (213) similar to those in the first or second embodiment. The inspection computer 2055 also stores the defect learning result data D110, the inspection image data D120, the defect probability map data D130, the image distortion data D140, and the distance data D150 similar to those in the first or second embodiment. In this embodiment, the image distortion data D150 is assumed to be distributed and stored in advance in the inspection computer as data calibrated to match the photographing camera. Next, the structure inspection support process performed by the structure inspection support system 3 will be described.

[0122] First, the defect learning processing unit 111 of the learning computer 2015 outputs the defect learning result data D110 (classifier) ​​to the storage device 2025 based on the defect learning image data D100. .

[0123] Next, based on input from the inspector 2040 via the input device 2050, the inspection computer 2055 receives the defect learning result data D110 from the learning computer 2015 via the network 2035, and stores the received defect learning result data D110 in the memory device 2065.

[0124] When distance data is required as in the second embodiment, the inspection computer 2055 acquires the distance data D150 from a camera or the like, and stores the acquired distance data D150 in the storage device 2065.

[0125] Then, in response to input from the inspector 2040 via the input device 2050, the inspection computer 2055 executes defect probability inference processing and defect distribution display processing in the same manner as in the first or second embodiment based on the inspection image data D120.

[0126] The inspection computer 2055 displays the inspection results (for example, a defect detection result screen) obtained through these processes on the output device 2045.

[0127] In this way, the structure inspection support system 3 according to the third embodiment is configured to execute the learning process of the classifier in advance on the learning computer 2015, which is a server computer (or computing cloud), and transfer the learning results (classifier) ​​to the inspection computer 2055, after which the defect detection process and the result display process are executed on the inspection computer 2055.

[0128] Although the third embodiment shows an example in which there is one inspector 2040 and one inspection computer 2055, there may be a plurality of these. For example, a plurality of inspection computers 2055 may be connected to a single training computer 2015 to inspect a large number of target structures.

[0129] Furthermore, in the third embodiment, a so-called pull-type system configuration was used in which the defect learning result data D110 was acquired by the inspection computer 2055 (inspector 2040), but a so-called push-type system configuration in which the defect learning result data D110 is distributed from the learning computer 2015 to the inspection computer 2055 may also be used.

[0130] As described above, the structural inspection support device of this embodiment inputs an image of a specified structure (target structure) into a classifier, outputs a defect probability map, which is data indicating the defect probability in each area of ​​the image of the target structure, and corrects the defect probability map by lowering the resolution of the area of ​​the distorted area in the defect probability map based on the distorted area in the image of the target structure and increasing the defect probability in the area where the resolution is lowered.

[0131] In other words, when a distorted portion is present in the target structure in the defect probability map of the target structure output by the classifier, the defect probability of the corresponding region of the defect probability map (for example, the edge of the image) is low. Therefore, the structure inspection support device reduces the resolution of the region of the distorted portion in the defect probability map and increases the defect probability in that region. This makes it possible to clearly indicate that the region is highly likely to be defective, even if the resolution is low.

[0132] In this way, the structure inspection support device of this embodiment can clearly show the results of detecting defects in a structure.

[0133] In addition, the structure inspection support device of this embodiment identifies the locations of distortion in the image of the target structure based on image distortion data D140 that indicates the locations of image distortion depending on the device, such as a camera, that captured the image of the target structure.

[0134] This makes it possible to accurately identify distortion in the image of the target structure depending on the characteristics of the imaging device such as a camera.

[0135] In addition, the structure inspection support device in Example 2 further corrects the defect probability map by lowering the resolution of each area identified by the defect probability map in accordance with each distance indicated by the distance data D150 based on distance data D150 relating to the distance between the camera or the like that captured the image of the target structure and each part of the target structure corresponding to each area of ​​the image, and increasing the defect probability for the area with the lowered resolution.

[0136] In this way, by lowering the resolution of the area of ​​the target structure and increasing the defect probability depending on the distance from the camera at the time of shooting (for example, when the distance from the camera is far), it is possible to clearly indicate that there is a high probability that there is a defect in that part even if the image resolution in the distorted part is low.

[0137] In addition, the structural inspection support device of this embodiment reduces the resolution by treating multiple pixels that make up the area corresponding to the distorted location in the defect probability map as new pixels (grids), and increases the defect probability by setting the sum of the defect probabilities within the grid as the defect probability of that grid.

[0138] This reliably increases the defect probability in the low-resolution image portion, making it possible to clearly indicate that there is a high probability that a defect exists in that portion.

[0139] Furthermore, the structure inspection support device of this embodiment outputs information on defects at each location of the target structure to an output device based on the modified defect probability map.

[0140] This makes it possible to provide the user with information indicating that there is a high possibility of a defect through a defect detection result screen, etc. This makes it possible to prevent the user from overlooking a defect.

[0141] For example, the structural inspection support device of this embodiment outputs to an output device the resolution of an area in a defect probability map having pixel values ​​corresponding to the defect probability value, where the defect probability has been increased, at a reduced resolution.

[0142] This allows the user to be provided with information indicating that there is a high probability of a defect even if the image of the target structure is distorted (for example, if the image of the target structure is partially stretched due to full-circle panoramic synthesis).

[0143] Furthermore, the structural inspection support device of this embodiment may output to the output device a graphic that indicates areas of the defect probability map with a high defect probability and that is highlighted according to the value of the defect probability.

[0144] This makes it possible to clearly indicate to the user that there is a high possibility of a defect.

[0145] The processing device is configured to detect at least one of a 360-degree camera, a fisheye camera, and a wide-angle camera. An image of a structure having a pipe portion, which is taken by a camera, including the pipe portion, is input to the classifier, and the defect probability map is output.

[0146] The present invention is not limited to the above-described embodiments, and can be implemented using any components within the scope of the present invention. The above-described embodiments and modifications are merely examples, and the present invention is not limited to these contents as long as the characteristics of the invention are not impaired. Furthermore, although various embodiments and modifications have been described above, the present invention is not limited to these contents. Other aspects conceivable within the scope of the technical idea of ​​the present invention are also included within the scope of the present invention.

[0147] For example, part of the hardware provided in each device of this embodiment may be provided in another device.

[0148] Furthermore, each program of each device may be provided in another device, a program may consist of multiple programs, or multiple programs may be integrated into one program.

[0149] In addition, in this embodiment, the number of classes output by the classifier is two, "defect" or "normal", but the classifier may output more than two classes. For example, if each pixel of an image is classified into three classes, namely, crack, corrosion, or normal, and a segmentation algorithm is used as the classifier, the true values ​​(correct labels) are normal = pixel value 0, Crack = pixel value 127, corrosion = pixel value 255 (in this case, the final output layer of the DNN) In the previous layer, one of three data sets is obtained: a normal probability image, a crack probability image, or a corrosion probability image.) In addition, when a defect probability learning algorithm is used as a classifier, the normal probability image, crack probability image, and corrosion probability image are trained as a set of three images as true values. [Explanation of symbols]

[0150] 1. Structural inspection support device 3. Structural inspection support system 111 Defect learning processing unit 112 Defect probability inference processing unit 113 Defect distribution display processing unit 114 Mixed resolution conversion processing unit

Claims

1. a storage device that stores a classifier that receives an image of a structure and outputs the probability that a defect exists in each location of the structure corresponding to each region of the image; a process of inputting an image of a specified structure into the classifier, and outputting a defect probability map, which is data indicating the probability in each region of the image of the specified structure; a processing device that executes a process of identifying distorted locations in an image of the specified structure, reducing the resolution of an area in the output defect probability map that corresponds to the distorted locations based on the identified distorted locations, and increasing the probability in the area with reduced resolution in accordance with a predetermined algorithm, thereby correcting the defect probability map.

2. The structure inspection support device according to claim 1, the storage device stores data indicating a distribution of distortion locations in the image corresponding to a device that captured the image of the specified structure; the processing device identifies a distortion location in the image of the specified structure based on the data; Structural inspection support device.

3. The structure inspection support device according to claim 1, the storage device stores data relating to the distance between a device that captured an image of the specified structure and each portion of the specified structure corresponding to each region of the image; the processing device further modifies the defect probability map based on the data by reducing the resolution of each region in the defect probability map in accordance with each distance indicated by the data and increasing the probability associated with the region with reduced resolution in accordance with a predetermined algorithm. Structural inspection support device.

4. The structure inspection support device according to claim 1, the processing device reduces the resolution of a region corresponding to the distortion location in the output defect probability map by setting a plurality of pixels constituting the region corresponding to the distortion location as a new pixel, and increases the probability by setting a sum of the probabilities of the plurality of pixels as the probability of the new pixel. Structural inspection support device.

5. The structure inspection support device according to claim 1, the processing device outputs information about defects at each location of the structure to an output device based on the corrected defect probability map. Structural inspection support device.

6. The structure inspection support device according to claim 5, the processing device outputs to the output device, at the reduced resolution, the region in which the probability has been increased, of the defect probability map having pixel values ​​according to the value of the probability; Structural inspection support device.

7. The structure inspection support device according to claim 5, the processing device identifies an area in the defect probability map having a probability equal to or greater than a predetermined value, and outputs to the output device a graphic that indicates the identified area and is emphasized according to the value of the probability. Structural inspection support device.

8. a first information processing device including: a processing device that receives an image of a structure as input and creates a classifier that outputs a probability that a defect exists in each location of the structure corresponding to each region of the image; and a communication device that transmits the created classifier; a storage device that receives the classifier and stores the received classifier; and a process of inputting an image of a specified structure into the classifier, and outputting a defect probability map, which is data indicating the probability in each region of the image of the specified structure; a process of identifying a distorted portion in the image of the specified structure, reducing the resolution of a region in the output defect probability map corresponding to the distorted portion based on the identified distorted portion, and increasing the probability in the region with reduced resolution according to a predetermined algorithm, thereby correcting the defect probability map; a second information processing device including a processing device that executes a process of outputting information about defects at each location of the structure to an output device based on the corrected defect probability map; A structure inspection support system comprising:

9. The structure inspection support device according to claim 1, The processing device includes: receiving an image of the specified structure from another information processing device via a communication device, and inputting the received image of the structure into the classifier, thereby outputting the defect probability map; creating information for the other information processing device to output information on defects at each location of the structure based on the corrected defect probability map, and transmitting the created information to the other information processing device via a communication device; Structural inspection support device.

10. The structure inspection support device according to claim 1, the processing device outputs the defect probability map by inputting an image of a structure having a pipe portion, the image being taken by at least one of a 360-degree camera, a fisheye camera, and a wide-angle camera, into the classifier; Structural inspection support device.

11. In the information processing device, an image of a structure is input, and a classifier is stored that outputs the probability that a defect exists in each location of the structure corresponding to each region of the image; inputting an image of a specified structure into the classifier, and outputting a defect probability map which is data indicating the probability in each region of the image of the specified structure; A distorted portion in the image of the specified structure is identified, and based on the identified distorted portion, a resolution of a region in the output defect probability map corresponding to the distorted portion is reduced, and the probability in the region with reduced resolution is increased in accordance with a predetermined algorithm, thereby correcting the defect probability map. Structural inspection support program.

Images