Image processing device, image processing method and program

Abstract

An image processing device (1) comprises: a structure range detection unit (12A) that detects structure ranges indicating representations of respective structures in an image; a specific range detection unit (12B) that detects, from each of the structure ranges, a specific range indicating a representation of a deteriorated portion in the structure; and an area correction unit (14) that corrects the pixel areas of the structure ranges and the specific ranges.

Description

[Technical Field]

[0001] The present disclosure relates to an image processing device, an image processing method, and a program. [Background technology]

[0002] Previously, when using image analysis techniques such as deep learning to detect the area of ​​a structure captured in a digital image, methods have been proposed to estimate the area of ​​the detected area based on the number of pixels.For example, when considering pipeline equipment, the corrosion area rate is estimated from the ratio of the pixels of each pipeline equipment to the corrosion pixels that have occurred above it.

[0003] Non-Patent Document 1 describes a method for detecting corrosion of pipes attached to a bridge from photographic images of the bridge using a two-stage deep learning algorithm. [Prior art documents] [Non-patent literature]

[0004] [Non-Patent Document 1] Yujin Hamano, 3 others, “Advanced Method for Detecting Deterioration of Bridge-Attached Equipment by Deep Learning Combining CNN and U-net”, SHMII-10(10th Structural Health Monitoring of Intelligent Infrastructure) Summary of the Invention [Problem to be solved by the invention]

[0005] However, digital images project structures that exist in three-dimensional space onto a two-dimensional plane, so in order to estimate the area of ​​a structure that exists in the original three-dimensional space, estimation by back projection is necessary.

[0006] Similarly, with conventional image analysis methods, the risk to a structure was evaluated equally regardless of the location of the detected area (corroded area, etc.) projected onto a plane. However, because forces such as gravity act on structures that exist in actual three-dimensional space, the risk to the structure must be evaluated by comprehensively considering the magnitude and direction of the acting force, as well as the location of the corrosion, etc. For this reason, just as with area evaluation, it is necessary to estimate the risk by back-projecting the image projected onto a two-dimensional plane into three-dimensional space, and then weight the impact of the detected area on the structure depending on the magnitude and direction of the force acting on the structure.

[0007] In view of the above circumstances, an object of the present invention is to provide an image processing device and an image processing method that correct the pixel area of ​​a region detected from an image projected onto a two-dimensional plane by image processing. [Means for solving the problem]

[0008] In order to solve the above problem, the image processing device of this embodiment is an image processing device that corrects the pixel area of ​​a detection area by image processing, and is equipped with a structure area detection unit that detects structure areas that show images of each structure in the image, a specific area detection unit that detects specific areas that show images of deteriorated parts of the structure from each of the structure areas, and an area correction unit that corrects the pixel area of ​​the structure areas and the specific areas.

[0009] In order to solve the above problem, the image processing method of this embodiment is an image processing method that corrects the pixel area of ​​a detection area by image processing, and executes the following steps using an image processing device: detecting structure areas that show images of each structure in an image; detecting specific areas from each of the structure areas that show images of deteriorated parts of the structure; and correcting the pixel area of ​​the structure areas and the specific areas.

[0010] In order to solve the above problem, the program according to this embodiment causes a computer to function as the image processing device described above. [Effects of the Invention]

[0011] The image processing device, image processing method, and program according to the present disclosure make it possible to accurately calculate the corrosion area on a structure. [Brief explanation of the drawings]

[0012] [Figure 1] 1 is a block diagram illustrating an example of the configuration of an image processing device according to an embodiment of the present invention. [Figure 2A] 2 is a diagram showing an example of an image input received by an image input unit shown in FIG. 1. FIG. [Figure 2B] FIG. 2B is a diagram showing an example of a structure region in the image shown in FIG. 2A. [Figure 2C] FIG. 2B is a diagram showing an example of a specific region in the image shown in FIG. 2A. [Figure 2D] FIG. 2C is a diagram showing a boundary line detected from a structure region in the image shown in FIG. 2B. [Figure 3] FIG. 10 is a diagram for explaining a Hough transform executed by a boundary detection unit. [Figure 4] 10A and 10B are diagrams illustrating an example of an estimation result of an interpolated boundary line executed by a boundary line interpolation unit. [Figure 5] 5 is a graph showing the number of pixels in each structure region in the image shown in FIG. 4. [Figure 6] 10A and 10B are diagrams illustrating a method for correcting pixel areas of a structure region and a specific region, which is executed by an area correction unit. [Figure 7] 10A and 10B are diagrams illustrating calculation of correction coefficients used to correct pixel areas. [Figure 8] FIG. 10 is a diagram illustrating an alternative correction coefficient that is applied to prevent the correction coefficient from diverging. [Figure 9] 10A and 10B are diagrams illustrating the depth correction of a structure executed by a viewpoint depth correction unit. [Figure 10] 10 is a flowchart illustrating an example of an image processing method executed by the image processing device according to the present embodiment. [Figure 11]FIG. 1 is a block diagram showing a schematic configuration of a computer that functions as an image processing apparatus. DETAILED DESCRIPTION OF THE INVENTION

[0013] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. The present invention is not limited to the following embodiments, and various modifications can be made within the scope of the present invention.

[0014] Fig. 1 is a diagram showing an example of the configuration of an image processing device 1 according to this embodiment. The image processing device 1 shown in Fig. 1 includes an image input unit 11, an area detection unit 12, a boundary processing unit 13, an area correction unit 14, and a data storage unit 15. The image processing device 1 corrects the pixel area of ​​the detected area through image analysis.

[0015] A control and arithmetic circuit 20 (controller 20) is configured by an image input unit 11, an area detection unit 12, a boundary line processing unit 13, and an area correction unit 14 of the image processing device 1. The control and arithmetic circuit 20 may be configured by dedicated hardware such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field-Programmable Gate Array), or may be configured by a processor, or may be configured by including both.

[0016] The image input unit 11 accepts input of image information representing an image generated by the imaging device 2. The imaging device 2 may be configured by a camera equipped with an optical element, an imaging element, and an output interface. The output interface is an interface for outputting the image information. The image input unit 11 may store the image information representing the image input from the imaging device 2 in an external memory.

[0017] The image may be an image of any format, for example, an RGB format image represented by red (R), green (G), and blue (B). An identifier for uniquely identifying the image information may be assigned to the image information. The identifier may be, for example, a number. The identifier may be a number obtained by incrementing a predetermined value in the order in which the image information is received by the image input unit 11. The predetermined value may be, for example, 1. In this way, when the image input unit 11 receives input of multiple pieces of image information, the image information can be associated with the results of processing, which will be described later and is executed based on the image information.

[0018] The area detection unit 12 detects areas in the image by analyzing the image received as input by the image input unit 11. The areas include a structure area ST in the image that shows an image of a structure (e.g., a pipeline) and a specific area that shows an image of a deteriorated portion of the structure. In this disclosure, a specific area, i.e., a deteriorated portion of a structure, refers to a corroded area, a paint peeling area, etc. that has occurred on the structure. The structure area detection unit 12A outputs the detection result of the structure area ST to the boundary processing unit 13. The specific area detection unit 12B outputs the detection result of the specific area to the area correction unit 14.

[0019] The region detection unit 12 can use any type of image analysis, including deep learning. The region detection unit 12 can use instance segmentation (e.g., Mask-R-CNN) to identify regions in an image. By using instance segmentation, it is possible to recognize individual structures in regions classified into the same class. Therefore, by using instance segmentation, the region detection unit 12 can detect multiple structure regions ST, each of which is part of the same structure, even if another object is located between the image capture device 2 and a portion of the structure to be detected in real space.

[0020] The area detection unit 12 includes a structure area detection unit 12A and a specific area detection unit 12B.

[0021] Fig. 2A is a diagram showing an example of an image input received by the image input unit shown in Fig. 1. Fig. 2B is a diagram showing an example of a structure region in the image shown in Fig. 2A. Fig. 2C is a diagram showing an example of a specific region in the image shown in Fig. 2A.

[0022] The structure area detection unit 12A detects a structure area (an area surrounded by a frame in FIG. 2B) that indicates an image of each structure in the image. In a configuration in which the structure is a cylindrical pipeline, the structure area detection unit 12A can detect the cylindrical structure area ST.

[0023] The specific area detection unit 12B detects a specific area (area surrounded by a frame in FIG. 2C) that indicates an image of a deteriorated part of the structure from each of the structure areas ST detected by the structure area detection unit 12A. The specific area may include a corroded area that indicates an image of a corroded part of the structure, a paint peeling area that indicates an image of a part of the structure where paint has peeled off, etc.

[0024] Boundary line processing section 13 has boundary line detection section 13 A and boundary line completion section 13 B. Fig. 2D is a diagram showing the boundary lines detected from the structure region in the image shown in Fig. 2B.

[0025] The boundary detection unit 13A detects a boundary line (first boundary line) BD1 (see FIG. 2D ), which is a line segment that defines each structure region ST. Specifically, the boundary detection unit 13A first detects one or more line segments from the structure region ST. The boundary detection unit 13A can detect line segments using known methods such as a probabilistic Hough transform, a Hough transform, or an LSD (Line Segment Detector). FIG. 3 is a diagram illustrating the Hough transform performed by the boundary detection unit 13A. The Hough transform is an image transformation method that detects objects having elements such as line segments from an image. The Hough transform extracts graphic elements by transforming from image space to a ρ-θ parameter space. For example, the boundary detection unit 13A may represent the boundary line BD1 using a line segment L, which is a set of points (x0, y0) that satisfy Equation (1) within the range of x0≧0, y0≧0 in a two-dimensional Cartesian coordinate system. In equation (1), ρ is the length of a perpendicular line drawn from the origin to a line passing through the coordinates (x, y), as shown in Figure 3, and θ is the angle between the x-axis and a perpendicular line drawn from the origin to a line passing through the coordinates (x, y). x0cosθ+y0sinθ=ρ (1)

[0026] The boundary detection unit 13A may detect line segments having a length equal to or greater than a predetermined value from the structure region ST, or may detect line segments of all lengths and then further detect line segments having a length equal to or greater than the predetermined value from the detected line segments. In this embodiment, the structure may be a pipeline extending in one direction. In this case, by excluding line segments having a length less than the predetermined value from the detection result and using line segments having a length equal to or greater than the predetermined value as the detection result, the accuracy of detection of boundary line BD1 by the boundary detection unit 13A, which will be described in detail later, can be improved.

[0027] FIG. 4 is a diagram illustrating an example of the estimation result of the interpolated boundary line performed by the boundary line interpolation unit 13B. The predetermined value may be appropriately set based on the length of the image of the pipeline, which is a structure expected to be included in the image. Furthermore, as shown in FIG. 4, multiple pipelines (structures) may be arranged adjacent to each other in a direction perpendicular to the extension direction. Furthermore, in this case, at least a portion of one pipeline may be located between at least a portion of another pipeline and the imaging device 2. In such a case, the image does not include an image of at least a portion of the pipeline. In the example shown in FIG. 4, portions of the structures represented by the structure regions STa and STb are located between the portion of the structure represented by the structure region STd and the imaging device 2. Therefore, the image generated by the imaging device 2 does not include an image of the portion of the structure represented by the structure region STd. Therefore, the predetermined value may be set in consideration of the length of the image of the portion of the pipeline expected to be included in the image, based on the positional relationship between the multiple pipelines indicated in the design information and the positional relationship between each of the multiple pipelines and the imaging device 2.

[0028] The boundary line detection unit 13A detects, as a boundary line BD1, a line segment corresponding to a line that demarcates the structure region ST from one or more line segments detected by the boundary line detection unit 13A.

[0029] The boundary line complementing unit 13B is a structure-equivalent region R, which is a region in the image that corresponds to a structure in real space. ST The boundary line (second boundary line) BD2 that defines the structure-corresponding region R and that was not detected by the boundary line detection unit 13A can be complemented. ST This is the area where an image would appear on the image if the structure was not hidden in real space, and includes the structure area.

[0030] Specifically, first, the boundary line completion unit 13B determines whether the number of pixels constituting each structure region ST, as shown in FIG. 5, is equal to or greater than a predetermined number of pixels pd1. The predetermined number of pixels pd1 is any number, but may be, for example, the average number of pixels constituting the structure region ST detected by the structure region detection unit 12A. In the example shown in FIG. 4, the boundary line completion unit 13B determines, for example, that the number of pixels in each of the structure regions STd and STe is less than the predetermined number of pixels pd1. Furthermore, the boundary line completion unit 13B determines, for example, that the number of pixels in the structure region STa, structure region STb, structure region STc, and structure region STf is equal to or greater than the predetermined number of pixels pd1.

[0031] When it is determined that the number of pixels constituting the structure region ST is equal to or greater than a predetermined pixel number pd1, the boundary line completion unit 13B determines whether the number of boundary lines BD1 in the structure region ST is equal to or greater than a predetermined number. For example, as shown in FIG. 4, in a configuration in which a plurality of pipelines (structures) are arranged adjacent to each other in a direction perpendicular to the extension direction, the predetermined number is 2. In the example shown in FIG. 4, the structure region STf of the structure f, whose number of pixels is determined to be equal to or greater than the predetermined pixel number pd1, is the hatched region. In this example, the boundary line completion unit 13B determines that the number of boundary lines BD1 in each of the structure region STa, structure region STb, and structure region STc is 2. Furthermore, the boundary line completion unit 13B determines that the number of boundary lines BD1 (indicated as "BDf1" in FIG. 4) in the structure region STf of the structure f is 1, i.e., less than 2.

[0032] Then, the boundary line complementing unit 13B extracts a structure-equivalent region R, which is a region in the image that corresponds to a structure in real space. ST 4, the structure region STf is the hatched portion, and the structure-corresponding region R ST is an area including both hatched and dotted portions. In this case, the boundary line complementing unit 13B extracts a structure-corresponding area R of a structure indicated by a structure area ST in which the number of boundary lines BD2 is less than a predetermined number (2 in the example shown in FIG. 4). STand the number of pixels of the structure region ST adjacent to the structure region ST that has the largest number of pixels, the structure-corresponding region R in which the number of boundary lines BD1 is less than a predetermined number (2 in the example shown in FIG. 4) is selected so that the difference between the number of pixels of the structure region ST and the number of pixels of the structure region ST that has the largest number of pixels among the structure regions ST adjacent to the structure region ST is minimized. ST In the example shown in FIG. 4, the boundary line BD2 of the structure corresponding to the structure indicated by the structure region STf is complemented. ST The boundary line BD2 (shown as "BDf2" in FIG. 4) is complemented so that the difference between the number of pixels of the structure area STf and the number of pixels of the structure area STc, which has the largest number of pixels among the structure areas ST adjacent to the structure area STf, is minimized (see FIG. 5).

[0033] The area correction unit 14 corrects the pixel areas of the structure area and specific areas (corroded areas, paint peeling areas, etc.) detected by the area detection unit 12. Images of structures that exist in three-dimensional space are originally projected onto a two-dimensional plane. This has led to the problem that it is difficult to accurately calculate the corroded area of ​​a structure. To solve this problem, the area correction unit 14 performs image analysis to estimate the projected areas of the structure area and specific areas by back projection correction, and then calculates correction coefficients by geometric processing to correct the pixel areas. A specific method for back projection correction of the pixel area of ​​the detected area performed by the area correction unit 14 is described below.

[0034] The area correction unit 14 defines, as the same correction target area, the structure area ST detected by the area detection unit 12 and the area surrounded by the two boundary lines detected by the boundary line processing unit 13 and the two line segments perpendicularly connecting the two boundary lines.The area correction unit 14 then multiplies the area of ​​the pixels included in the area surrounded by the two boundary lines and the two line segments perpendicularly connecting the two boundary lines by a correction coefficient determined by geometric processing, to correct the pixel area.

[0035] 6 is a diagram illustrating a method for correcting the pixel areas of the structure region ST and the specific region by the area correction unit 14. The area correction unit 14 performs correction by multiplying the pixel areas of the structure region ST and the specific region in the image by a correction coefficient α for pixels included in the region surrounded by two boundary lines (1) and (2) and two line segments perpendicularly connecting the boundary lines (1) and (2). An example of calculating the correction coefficient α using trigonometric functions will be described below.

[0036] In Figure 6, dx is the width of one pixel, dy is the height of one pixel, dS is the area of ​​one pixel before correction, and dS T is the area of ​​one pixel after correction, then dS T can be calculated using the following formula (2). dS T = dx·rdθ (2)

[0037] 7 is a diagram for explaining the calculation of the correction coefficient α used for correcting the pixel area. As shown in FIG. 7, the relationship between dy and rdθ can be expressed by the following equation (3). dy = rdθ·sinθ (3)

[0038] In FIG. 6, if the distance between the boundary line (1) and the boundary line (2) is l=2r and the number of pixels is n, the pixel area after correction is dS T can be calculated from the following equation (4): The correction coefficient α is as shown in equation (5).

number

[0039] (Prevention of divergence of correction coefficients) Fig. 8 is a diagram explaining the alternative correction coefficient ξ that is applied to prevent the correction coefficient α from diverging. Since the correction coefficient α is the inverse of the sine function as shown in equation (5), the closer the pixel bp is to the boundary of the pipe in the image shown in Fig. 8, the more likely it is to diverge to infinity as shown in equation (6) below.

number

[0040] To solve this problem, when the correction coefficient α is equal to or greater than the threshold ω, the area correction unit 14 replaces the correction coefficient α with a predetermined value and calculates the corrected pixel area dS T The boundary area correction unit 14A is provided to prevent the divergence of the pixel area dS after correction. The boundary area correction unit 14A determines a threshold value ω in advance as shown in the following equation (7), and when the correction coefficient α is equal to or greater than the threshold value ω, the correction coefficient α is T is replaced by an alternative correction factor ξ that prevents divergence of α≧ω ⇔ α=ξ (7)

[0041] (Depth correction of structures) Furthermore, there is a problem that the angle at which the image is captured by the imaging device 2 causes a sense of perspective in the appearance of the structure. To solve this problem, the area correction unit 14 sets a gradient coefficient β that indicates the gradient of the number of pixels based on the ratio of the number of pixels between the boundary lines of the pipeline of the structure (between the boundary lines (1) and (2) of the pipeline in Figures 6 and 8), and calculates the corrected pixel area dS T is multiplied by the gradient coefficient β to obtain the corrected pixel area dS T The viewpoint depth correction unit 14B performs depth correction on the structure. Figure 9 is a diagram illustrating the depth correction of structures performed by the viewpoint depth correction unit 14B. Long facilities such as pipelines in an image are mapped so that the front side of the image is larger and the back side is smaller due to the depth (in Figure 9, the number of pixels i>m). For this reason, the viewpoint depth correction unit 14B derives a gradient coefficient β and updates the pixel area based on the ratio of the number of pixels between boundary lines. For structures such as pipelines, for which standards are defined for each facility, the depth of the structure can be expressed by setting the gradient coefficient β based on the ratio of the number of pixels in the area surrounded by straight line components. The gradient coefficient β is introduced when there is a certain or greater difference between the maximum and minimum number of pixels between boundary lines. The criteria for introducing the gradient coefficient β can be defined arbitrarily, but it is necessary to determine in advance the percentage below which the minimum number of pixels must be introduced. Area after depth correction dS' TAs shown in Figure 9, when the maximum number of pixels is i and the number of pixels to be updated is m, it can be calculated by the following formula (8). If the number of pixels between the boundary lines of the pipeline is different, the pixel area dS after correction is calculated based on the maximum number of pixels between the boundary lines. T where the gradient coefficient β is the maximum number of pixels i on the front side of the image divided by the number of pixels m to be updated. dS' T =βdS T (8)

[0042] Furthermore, the area correction unit 14 calculates the corrosion area ratio from the area ratio of the pixels. The corrosion area ratio is the value obtained by dividing the pixel area of ​​a specific region (corroded region, paint peeling region, etc.) by the pixel area of ​​the structure region ST. By calculating the corrosion area ratio, it becomes possible to accurately grasp the proportion of the corrosion area on the structure.

[0043] The area correction unit 14 outputs the correction result of the pixel area of ​​the structure region, the correction result of the pixel area of ​​the specific region, and analysis information including the corrosion area rate to the data storage unit 15. The data storage unit 15 is a memory.

[0044] FIG. 10 is a flowchart showing an example of an image processing method executed by the image processing device 1 according to this embodiment.

[0045] In step S101, the image input unit 11 receives input of image information indicating an image captured by the imaging device 2.

[0046] In steps S102 and S103, the structure area detection unit 12A detects a structure area ST that indicates an image of a structure in the image.

[0047] Specifically, in step S102, the structure area detection unit 12A determines whether or not the image of a structure included in the image is a detection target.

[0048] If it is determined in step S102 that the images of structures included in the image are the detection targets, then in step S103, the structure area detection unit 12A detects structure areas ST showing the images of the respective structures.

[0049] If it is determined in step S102 that the image of the structure included in the image is not a detection target, the image processing device 1 ends the process.

[0050] In step S104, the specific area detection unit 12B detects a specific area showing an image of a deteriorated part in the structure from each of the structure areas ST.

[0051] In step S105, the boundary detection unit 13A detects one or more line segments from the structure region ST. Here, the boundary detection unit 13A may detect line segments having a length equal to or greater than a predetermined value from the structure region ST, or may detect line segments of all lengths and then further detect line segments having lengths equal to or greater than a predetermined value from the detected line segments.

[0052] In step S106, the boundary line detection unit 13A detects, from the one or more line segments detected in step S105, a line segment that corresponds to a line that demarcates the structure region ST as a boundary line BD1.

[0053] In step S107, the boundary line complementing unit 13B determines whether the number of pixels constituting the structural object region ST detected in step S102 is equal to or greater than a predetermined number of pixels pdl.

[0054] If it is determined in step S107 that the number of pixels constituting the structure region ST is less than the predetermined number of pixels pd1, the image processing device 1 terminates the processing of the structure region ST. Thereafter, the image processing device 1 may repeat the processing from step S102 onwards for images of other structures included in the image indicated by the image information acquired in step S101.

[0055] If it is determined in step S107 that the number of pixels constituting the structural object region ST is equal to or greater than the predetermined number of pixels pd1, then in step S108, the boundary line complementing unit 13B determines whether the number of boundary lines BD1 in the structural object region ST is two.

[0056] If it is determined in step S108 that the number of boundary lines BD1 is less than two, in step S109, the boundary line complementing unit 13B extracts a structure-corresponding region R, which is a region in the image that corresponds to a structure in real space. ST This complements the boundary line BD2 that defines the

[0057] In step S110, the area correction unit 14 corrects the pixel area of ​​the area detected as the structure area. The area correction unit 14 corrects the pixel area by multiplying the area of ​​the pixels included in the area surrounded by the two boundary lines and the two line segments perpendicularly connecting the two boundary lines by a correction coefficient determined by geometric processing.

[0058] In step S110, if the correction coefficient α is equal to or greater than the threshold value, the boundary area correction unit 14A replaces the correction coefficient α with a predetermined value (alternative correction coefficient ξ).

[0059] In step S111, the viewpoint depth correction unit 14B calculates the following value from the ratio of the number of pixels between the boundary lines of the pipelines of the structure: picture Set the gradient coefficient β, which indicates the gradient of the prime number, and calculate the corrected pixel area dS T is multiplied by a gradient coefficient β to perform depth correction of the pixel area after correction.

[0060] In step S112, the area correction unit 14 corrects the pixel area of ​​the region detected as the specific region.

[0061] In step S113, the area correction unit 14 calculates the corrosion area ratio from the area ratio of the pixels.

[0062] In step S114, the area correction unit 14 outputs to the data storage unit 15 the correction results of the pixel areas of the structure region and the specific region, and analysis information including the corrosion area rate.

[0063] Conventionally, digital images of structures existing in three-dimensional space are projected onto a two-dimensional plane, making it difficult to accurately calculate the corrosion area of ​​the structure. To solve this problem, the image processing device 1 according to the present disclosure performs image analysis to estimate the projected areas of the structure region ST and specific regions using back-projection correction, and then corrects the pixel area by calculating a correction coefficient using geometric processing. Furthermore, conventionally, there has been a problem in that the angle at which the image is captured by the imaging device 2 causes a sense of perspective in the appearance of the structure. The image processing device 1 sets a gradient coefficient β based on the ratio of the number of pixels, taking into account the depth of the structure, and corrects the pixel area using the gradient coefficient β. An image processing device 1 with such functions makes it possible to accurately calculate the corrosion area on a structure.

[0064] A computer capable of executing program instructions can also be used to function the image processing device 1. Fig. 11 is a block diagram showing a schematic configuration of a computer functioning as the image processing device 1. Here, the computer functioning as the image processing device 1 may be a general-purpose computer, a dedicated computer, a workstation, a PC (Personal Computer), an electronic notepad, etc. The program instructions may be program code, code segments, etc. for performing necessary tasks.

[0065] 11, the computer 100 includes a processor 110, a memory unit including a ROM (Read Only Memory) 120, a RAM (Random Access Memory) 130, and a storage 140, an input unit 150, an output unit 160, and a communication interface (I / F) 170. Each component is connected to each other via a bus 180 so as to be able to communicate with each other.

[0066] The ROM 120 stores various programs and various data. The RAM 130 temporarily stores programs or data as a working area. The storage 140 is configured with an HDD (Hard Disk Drive) or an SSD (Solid State Drive) and stores various programs including an operating system and various data. In the present disclosure, the programs related to the present disclosure are stored in the ROM 120 or the storage 140.

[0067] Specifically, the processor 110 is a CPU (Central Processing Unit), MPU (Micro Processing Unit), GPU (Graphics Processing Unit), DSP (Digital Signal Processor), SoC (System on a Chip), etc., and may be configured with multiple processors of the same or different types. The processor 110 reads a program from the ROM 120 or storage 140 and executes the program using the RAM 130 as a working area, thereby controlling the above components and performing various arithmetic processing. Note that at least a part of these processing contents may be realized by hardware.

[0068] The program may be recorded on a recording medium readable by the image processing device 1. If such a recording medium is used, the program can be installed in the image processing device 1. Here, the recording medium on which the program is recorded may be a non-transitory recording medium. The non-transitory recording medium is not particularly limited, and may be, for example, a CD-ROM, a DVD-ROM, a USB (Universal Serial Bus) memory, or the like. Furthermore, the program may be in a form that is downloaded from an external device via a network.

[0069] The following additional notes are provided regarding the above-described embodiments.

[0070] (Additional note 1) An image processing device that corrects the pixel area of ​​a detection area through image processing, comprising: an image processing device that detects structure areas that show images of each structure in an image; detects specific areas that show images of deteriorated parts of the structure from each of the structure areas; and corrects the pixel area of ​​the structure areas and the specific areas. (Additional note 2) The image processing device described in Appendix 1, wherein the controller detects boundary lines, which are line segments that define the structure area, and corrects the pixel area by multiplying the area of ​​pixels included in the area surrounded by two of the boundary lines and two line segments perpendicularly connecting the two boundary lines by a correction coefficient obtained by geometric processing. (Additional note 3) 3. The image processing device according to claim 2, wherein the controller includes a boundary area correction unit that replaces the correction coefficient with a predetermined value when the correction coefficient is equal to or greater than a threshold value. (Additional note 4) 4. The image processing device according to any one of appendix 1 to 3, wherein the controller sets a gradient coefficient indicating a gradient of the number of pixels based on a ratio of the number of pixels between boundary lines of the pipelines of the structure, and multiplies the corrected pixel area by the gradient coefficient to perform depth correction of the corrected pixel area. (Additional note 5) The image processing device according to any one of claims 1 to 4, wherein the controller complements any undetected boundary lines that define a structure-equivalent area, which is an area in the image that corresponds to the structure area in real space. (Additional note 6) An image processing method for correcting a pixel area of ​​a detection region by image processing, comprising: An image processing method that uses an image processing device to detect structure areas that show images of each structure in an image, detect specific areas from each of the structure areas that show images of deteriorated parts of the structure, and correct the pixel areas of the structure areas and the specific areas. (Additional note 7) A non-transitory storage medium storing a program executable by a computer, the non-transitory storage medium storing the program causing the computer to function as the image processing device described in any one of appendixes 1 to 5.

[0071] Although the above-described embodiments have been described as typical examples, it will be apparent to those skilled in the art that many modifications and substitutions can be made within the spirit and scope of the present disclosure. Therefore, the present invention should not be interpreted as being limited by the above-described embodiments, and various modifications or alterations are possible without departing from the scope of the claims. For example, multiple building blocks shown in the block diagrams of the embodiments can be combined into one, or one building block can be divided. [Explanation of symbols]

[0072] 1. Image processing device 2. Imaging device (camera) 11 Image input unit 12 Area detection unit 12A Structure area detection unit 12B Specific area detection section 13 Boundary processing section 13A Boundary line detection unit 13B Border complement part 14 Area correction section 14A Boundary area correction section 14B Viewpoint depth correction section 15 Data storage unit 20 Control operation circuit (controller) 100 computers 110 processors 120 ROM 130 RAM 140 Storage 150 Input section 160 Output section 170 Communication Interface (I / F) 180 Bus

Claims

1. An image processing device that corrects a pixel area of ​​a detection region by image processing, a structure area detection unit that detects a structure area showing an image of each structure in the image; a specific area detection unit that detects a specific area showing an image of a deteriorated portion of the structure from each of the structure areas; a boundary detection unit that detects boundary lines that are line segments that define the structure area; an area correction unit that corrects pixel areas of the structure region and the specific region; Equipped with the area correction unit corrects the pixel area by multiplying the area of ​​a pixel included in the area of ​​a pixel included in an area surrounded by the two boundary lines and two line segments perpendicularly connecting the two boundary lines by a correction coefficient obtained by geometric processing; the correction coefficient is a coefficient calculated using a trigonometric function based on the position of the pixel between the two boundary lines, and the closer the pixel is to the two boundary lines, the closer the value approaches infinity; The image processing device includes a boundary area correction unit that replaces the correction coefficient with a predetermined value when the correction coefficient is equal to or greater than a threshold value.

2. An image processing device that corrects a pixel area of ​​a detection region by image processing, a structure area detection unit that detects a structure area showing an image of each structure in the image; a specific area detection unit that detects a specific area showing an image of a deteriorated portion of the structure from each of the structure areas; a boundary detection unit that detects boundary lines that are line segments that define the structure area; a boundary line complementing unit that complements boundary lines that demarcate the structure-equivalent region, which is a region in the image that corresponds to the first structure in real space, and that complements boundary lines that were not detected by the boundary line detection unit so that a difference between the number of pixels of the structure-equivalent region, which is a region in the image that corresponds to the first structure in real space, and the number of pixels of a second structure region that has the largest number of pixels among structure regions adjacent to the first structure region, is minimized when the structure region in which the number of boundary lines is detected by the boundary line detection unit is defined as a first structure region that shows an image of a first structure; an area correction unit that corrects pixel areas of the structure region and the specific region; Equipped with The area correction unit corrects the pixel area by multiplying the area of ​​a pixel included in the area of ​​a pixel included in an area surrounded by the two boundary lines and two line segments perpendicularly connecting the two boundary lines by a correction coefficient obtained by geometric processing.

3. The correction coefficient is a coefficient calculated using a trigonometric function based on the position of the pixel between the two boundary lines, and the closer the pixel is to the two boundary lines, the closer it is to infinity, The image processing device according to claim 2 , wherein the area correction section includes a boundary area correction section that replaces the correction coefficient with a predetermined value when the correction coefficient is equal to or greater than a threshold value.

4. 4. The image processing device according to claim 1, wherein the area correction unit sets a gradient coefficient indicating a gradient of the number of pixels from a ratio of the number of pixels between boundary lines of the pipelines of the structure, and includes a viewpoint depth correction unit that multiplies the corrected pixel area by the gradient coefficient to perform depth correction of the corrected pixel area.

5. An image processing method executed by an image processing device that corrects a pixel area of ​​a detection region by image processing, comprising: detecting structure regions in the image that represent respective images of the structures; detecting a specific area showing an image of a deteriorated portion of the structure from each of the structure areas; detecting a boundary line that is a line segment that defines the structure area; an area correction step of correcting pixel areas of the structure region and the specific region; Including, the area correction step includes a step of correcting the pixel area by multiplying an area of ​​a pixel included in an area of ​​pixels included in an area surrounded by the two boundary lines and two line segments perpendicularly connecting the two boundary lines by a correction coefficient obtained by geometric processing; the correction coefficient is a coefficient calculated using a trigonometric function based on the position of the pixel between the two boundary lines, and the closer the pixel is to the two boundary lines, the closer the value approaches infinity; The image processing method, wherein the area correcting step includes a step of replacing the correction coefficient with a predetermined value when the correction coefficient is equal to or greater than a threshold value.

6. An image processing method executed by an image processing device that corrects a pixel area of ​​a detection region by image processing, comprising: detecting structure regions in the image that represent respective images of the structures; detecting a specific area showing an image of a deteriorated portion of the structure from each of the structure areas; detecting a boundary line that is a line segment that defines the structure area; a step of complementing undetected boundary lines that define the structure-equivalent region so that, when a structure region in which the number of detected boundary lines is less than a predetermined number is defined as a first structure region that shows an image of a first structure, the difference between the number of pixels of the structure-equivalent region, which is a region in the image that corresponds to the first structure in real space, and the number of pixels of a second structure region that has the largest number of pixels among structure regions adjacent to the first structure region, is minimized; an area correction step of correcting pixel areas of the structure region and the specific region; Including, The image processing method includes a step of correcting the pixel area by multiplying the area of ​​a pixel included in the area of ​​a pixel included in an area surrounded by the two boundary lines and two line segments perpendicularly connecting the two boundary lines by a correction coefficient obtained by geometric processing.

7. A program for causing a computer to function as the image processing device according to any one of claims 1 to 3.

Images