Image processing device, image processing method and program

Abstract

An image processing device (10) according to the present disclosure comprises: an image dividing unit (12) that divides an image to be processed including a predetermined subject into a plurality of divided images of a predetermined size; an area detecting unit (13) that, in each of the plurality of divided images, detects a subject area that is a pixel area of the subject in a divided image and a deteriorated area that is a pixel area of a deteriorated portion of the subject; an information integrating unit (14) that generates a subject detection result image in which the images of the subject areas detected for each of the plurality of divided images are joined together while maintaining the positional relationships of the plurality of divided images, and generates a deterioration detection result image in which the images of the deteriorated areas detected for each of the plurality of divided images are joined together while maintaining the positional relationships of the plurality of divided images; and a diagnostic unit (15) that diagnoses the deterioration of the subject on the basis of the subject detection result image and the deterioration detection result image.

Description

[Technical field]

[0001] The present disclosure relates to an image processing device, an image processing method, and a program. [Background technology]

[0002] In recent infrastructure inspections, a method is used to automatically determine whether or not the facility has deteriorated and the extent of the deterioration using photographed images of the facility and AI (Artificial Intelligence) created by deep learning methods. In order to not miss even the slightest deterioration, it is important to use images with as high a resolution as possible for the images used for the judgment. However, when applying deep learning methods to high-resolution images, there is an issue that calculations are impossible or require a long time even if calculations are possible using commercially available GPUs (Graphics Processing Units).

[0003] In order to address such issues, it is conceivable to use a high-performance computer. However, the photographic equipment (e.g., inexpensive compact digital cameras) used in the field of infrastructure facility inspections is mainly equipment that captures images with approximately 20 million pixels, and it is difficult to obtain a commercially available computer that can analyze images captured by such photographic equipment. For this reason, for example, Non-Patent Documents 1 and 2 describe a technique for compressing an image and performing analysis using the compressed image. By using the compressed image, it is possible to reduce the performance required for a computer that analyzes the image and to speed up image analysis. [Prior art documents] [Non-patent literature]

[0004] [Non-Patent Document 1] Yu Tabata et al., "Verification of automatic bridge damage detection using UAV photography and deep learning," Journal of the Japan Society of Civil Engineers, Vol. 74, No. 2, p. 62-74, 2018 [Non-Patent Document 2] Kengo Kawashiro et al., "An approach to tunnel damage extraction using semantic segmentation," Proceedings of the 82nd National Conference of Information Processing Society of Japan, No. 4, p.4.233-4.234, 2020 Summary of the Invention [Problem to be solved by the invention]

[0005] However, in the above-mentioned technology, the image resolution is reduced by image compression, so that the detection accuracy may be reduced. Also, in the above-mentioned technology, a minute area that was captured in the high-resolution image before compression may be lost due to compression. Therefore, when the technology described in Non-Patent Documents 1 and 2 is applied to the inspection of the above-mentioned infrastructure facilities, there is a problem that the accuracy of detection of the target (infrastructure facilities) and the deterioration of the target may be reduced.

[0006] The objective of the present disclosure, made in consideration of the above-mentioned problems, is to provide an image processing device, an image processing method, and a program that can increase the accuracy of detecting specified objects and deterioration of the objects contained in an image to be processed, while suppressing the need to increase the processing performance required for detection. [Means for solving the problem]

[0007] In order to solve the above problems, the image processing device of the present disclosure comprises an image division unit that divides an image to be processed including a specified object into multiple split images of a specified size, a region detection unit that detects, for each of the multiple split images, an object region, which is the pixel region of the object in the split image, and a degradation region, which is the pixel region of a degraded portion of the object, an information combination unit that generates an object detection result image by combining images of the object regions detected for each of the multiple split images while maintaining the positional relationship of the multiple split images, and a degradation detection result image by combining images of the degraded regions detected for each of the multiple split images while maintaining the positional relationship of the multiple split images, and a diagnosis unit that diagnoses degradation of the object based on the object detection result image and the degradation detection result image.

[0008] In addition, in order to solve the above problem, the image processing device of the present disclosure includes an image division unit that divides an image to be processed including a specified object into multiple divided images of a specified size, an image compression unit that compresses the image to be processed to a specified size, an object area detection unit that detects an object area which is a pixel area of ​​the object in the compressed image, a deteriorated area detection unit that detects a deteriorated area which is a pixel area of ​​a deteriorated portion of the object in each of the multiple divided images, an information combination unit that generates a deterioration detection result image by combining images of the deteriorated areas detected for each of the multiple divided images while maintaining the positional relationship of the multiple divided images, and a diagnosis unit that diagnoses deterioration of the object based on an object detection result image which is an image of the object area in the compressed image and the deterioration detection result image.

[0009] In addition, in order to solve the above-mentioned problems, the image processing method disclosed herein is an image processing method by an image processing device, and includes the steps of: dividing an image to be processed including a specified object into a plurality of split images of a specified size; detecting, for each of the plurality of split images, an object region which is the pixel region of the object in the split image, and a degradation region which is the pixel region of a degraded portion of the object; generating an object detection result image by combining images of the object region detected for each of the plurality of split images while maintaining the positional relationship of the plurality of split images, and generating a degradation detection result image by combining images of the degraded region detected for each of the plurality of split images while maintaining the positional relationship of the plurality of split images; and diagnosing deterioration of the object based on the object detection result image and the degradation detection result image.

[0010] In order to solve the above problem, a program according to the present disclosure causes a computer to operate as the image processing device described above. Effect of the Invention

[0011] The image processing device, image processing method, and program disclosed herein can improve the accuracy of detecting specified objects and deterioration of objects contained in an image to be processed, while suppressing the need to increase the processing performance required for detection. [Brief description of the drawings]

[0012] [Figure 1] 1 is a diagram illustrating an example of the configuration of an image processing device according to a first embodiment of the present disclosure. [Diagram 2] 2 is a diagram illustrating an example of the configuration of an image dividing unit illustrated in FIG. 1. [Diagram 3] 2 is a diagram illustrating an example of the configuration of a region detection unit illustrated in FIG. 1; [Figure 4] 3 is a flowchart showing an example of the operation of a division number determination unit shown in FIG. 2. [Diagram 5] 3 is a diagram for explaining the determination of a division number by a division number determination unit shown in FIG. 2. FIG. [Figure 6] 3 is a flowchart showing an example of the operation of a division execution unit shown in FIG. 2. [Figure 7] 3 is a diagram showing an example of division of an image to be processed by a division number determination unit shown in FIG. 2; [Figure 8] 3 is a diagram showing another example of division of the image to be processed by the division number determination unit shown in FIG. 2. FIG. [Figure 9] 3 is a diagram showing yet another example of division of the image to be processed by the division number determination unit shown in FIG. 2. FIG. [Figure 10] 2 is a flowchart showing an example of the operation of the image processing device shown in FIG. [Figure 11] FIG. 11 is a diagram illustrating an example of the configuration of an image processing device according to a second embodiment of the present disclosure. [Figure 12] 12 is a diagram illustrating an example of the configuration of an object region detection unit illustrated in FIG. 11. [Figure 13] 12 is a diagram illustrating an example of the configuration of a degraded area detection unit illustrated in FIG. 11. [Figure 14] 12 is a flowchart showing an example of the operation of the image processing device shown in FIG. [Figure 15]FIG. 13 is a diagram illustrating an example of the configuration of an image processing device according to a third embodiment of the present disclosure. [Figure 16] FIG. 13 is a diagram illustrating another exemplary configuration of the image processing device according to the third embodiment of the present disclosure. [Figure 17] FIG. 1 is a diagram illustrating an example of a hardware configuration of an image processing device according to the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings.

[0014] (First embodiment) 1 is a diagram showing an example of the configuration of an image processing device 10 according to a first embodiment of the present disclosure. The image processing device 10 according to this embodiment detects a predetermined object included in an image to be processed and deterioration of the object. The image to be processed is, for example, an image of infrastructure equipment (e.g., a utility pole, etc.). In this case, the image processing device 10 detects the infrastructure equipment and deterioration of the infrastructure equipment from the image to be processed.

[0015] As shown in FIG. 1, an image processing apparatus 10 according to this embodiment includes an image input unit 11, an image division unit 12, a region detection unit 13, an information combination unit 14, and a diagnosis unit 15.

[0016] An image to be processed is input to the image input unit 11. As described above, the image input to the image input unit 11 is, for example, an image of a predetermined infrastructure facility. The image input unit 11 outputs the input image to the image division unit 12.

[0017] The image dividing unit 12 divides the image to be processed, which is output from the image input unit 11, into a plurality of divided images of a predetermined size. Fig. 2 is a diagram showing an example of the configuration of the image dividing unit 12. As shown in Fig. 2, the image dividing unit 12 includes a division number determination unit 121 and a division execution unit 122.

[0018] The division number determination unit 121 determines the number of divisions in the width direction of the image to be processed and the number of divisions in the height direction of the image to be processed. The details of the determination of the division number by the division number determination unit 121 will be described later.

[0019] The division execution unit 122 divides the image to be processed by the division number determined by the division number determination unit 121, and outputs the divided images, which are images after division, to the area detection unit 123. When the image to be processed is divided sequentially, depending on the size and division number of the divided images, the divided images may extend beyond the image to be processed near the edge of the image to be processed. In this case, the division execution unit 122 standardizes the size of the divided images by adding a predetermined image or the like (standardizes the size of the divided images). Details of the division of the image to be processed by the division execution unit 122 will be described later.

[0020] Referring back to FIG. 1, the image dividing unit 12 divides the image to be processed and outputs a plurality of divided images to the area detection unit 13.

[0021] The area detection unit 13 detects an object area, which is a pixel area of ​​an object in each of the multiple divided images output from the image division unit 12, and a degraded area, which is a pixel area of ​​a degraded part of the object, in the divided image. Fig. 3 is a diagram showing an example of the configuration of the area detection unit 13. In Fig. 3, an example will be described in which the area detection unit 13 has a function of creating a model for detecting the object area and the degraded area from the divided image, and a function of detecting the object area and the degraded area from the divided image using the created model.

[0022] As shown in Fig. 3, the area detection unit 13 includes a model construction unit 131, an object detection unit 132, and a deterioration detection unit 133. As described above, the area detection unit 13 includes a function of creating a model for detecting an object area and a deteriorated area from a divided image, and a function of detecting an object area and a deteriorated area from the divided image using the created model. The divided image used for creating the model is input to the model construction unit 131. In addition, the divided image in which the object and deterioration of the object are detected is input to the object detection unit 132 and the deterioration detection unit 133.

[0023] The model construction unit 131 uses the input segmented image to create a model (detector) for detecting an object region in an image and a model for detecting a deteriorated region by a deep learning method. As shown in FIG. 3, the model construction unit 131 includes an object detection learning unit 1311 and a deterioration detection learning unit 1312.

[0024] The object detection learning unit 1311 uses the divided images and a mask image indicating an object region in the divided images to create an object detector that detects an object region in an image by a deep learning method. The object detection learning unit 1311 stores the created object detector in the object detection unit 132.

[0025] The degradation detection learning unit 1312 uses the divided image and a mask image indicating a degraded area in the divided image to create a degradation detector that detects a degraded area in the image by a deep learning method. The degradation detection learning unit 1312 stores the created degradation detector in the degradation detection unit 133.

[0026] The object detection unit 132 detects an object region in the input divided image (a divided image that is a target for detecting an object region) using the object detector created by the object detection learning unit 1311. The object detection unit 132 outputs the detection result of the object region to the information combination unit 14.

[0027] The degradation detection unit 133 detects a degraded area in the input divided image (a divided image in which a degraded area is to be detected) using the degradation detector created by the degradation detection learning unit 1312. The degradation detection unit 133 outputs the detection result of the degraded area to the information combination unit 14.

[0028] 3, an example in which the area detection unit 13 has a function of creating an object detector and a deterioration detector has been described, but the present invention is not limited to this. The object detector and the deterioration detector may be created outside the image processing device 10 and stored in the object detection unit 132 and the deterioration detection unit 133. In this case, the area detection unit 13 does not need to include the model construction unit 131.

[0029] 1 again, the information combination unit 14 generates an object detection result image by combining images of object regions detected for each of the multiple divided images by the region detection unit 13 (object detection unit 132). As described above, the object region is detected for each of the multiple divided images obtained by dividing the image to be processed. Therefore, the information combination unit 14 generates an object detection result image by combining images of the object regions detected for each of the multiple divided images while maintaining the positional relationship between the multiple divided images.

[0030] Furthermore, the information combination unit 14 generates a degradation detection result image by combining images of degraded regions detected for each of the multiple divided images by the area detection unit 13 (degradation detection unit 133). As described above, a degraded region is detected for each of the multiple divided images obtained by dividing the image to be processed. Therefore, the information combination unit 14 generates a degradation detection result image by combining images of degraded regions detected for each of the multiple divided images while maintaining the positional relationship between the multiple divided images.

[0031] The information combination unit 14 outputs the generated object detection result image and degradation detection result image to the diagnosis unit 15.

[0032] The diagnosis unit 15 diagnoses the deterioration of the object based on the object detection result image and the deterioration detection result image output from the information combination unit 14. For example, the diagnosis unit 15 superimposes the object detection result image and the deterioration detection result image, and calculates the deterioration rate of the object from the ratio of the deteriorated area to the object area, etc.

[0033] In the image processing device 10 according to the present embodiment, an image to be processed is divided into a plurality of divided images, and an object region and a degraded region are detected for each of the plurality of divided images. Therefore, since there is no need to compress the image (the number of pixels in the detection target region is not reduced), it is possible to improve the accuracy of detecting the object and degradation of the object contained in the image to be processed. In addition, by detecting the object region and the degraded region for the divided images obtained by dividing the image to be processed, it is possible to suppress the sophistication of the processing performance required for detection.

[0034] Next, the operation of the image processing device 10 according to this embodiment will be described.

[0035] First, the determination of the number of divisions of the image to be processed by the division number determination unit 121 will be described.

[0036] FIG. 4 is a flowchart showing an example of the operation of the division number determination unit 121.

[0037] The division number determination unit 121 acquires an image to be processed that is input via the image input unit 11 (step S11).

[0038] 5, the division number determination unit 121 sets the width direction (X direction) size of the acquired image to be processed as X, and sets the height direction (Y direction) size of the image to be processed as Y. The division number determination unit 121 also sets the width direction (X direction) size of the divided image as x, and sets the height direction (Y direction) size of the image to be processed as y (step S12). The division number determination unit 121 also sets a variable n=1, and a variable m=1 (step S13).

[0039] The division number determination unit 121 determines whether the product of the size x of the divided image in the width direction and the variable n is equal to or larger than the size X of the image to be processed in the width direction (n·x≧X) (step S14).

[0040] When it is determined that n·x < X (not n·x ≧ X) (step S14: No), the division number determination unit 121 adds 1 to the variable n (step S15) and returns to the process of step S14. That is, the division number determination unit 121 repeats the processes of step S14 and step S15 until the product of the size x in the width direction of the divided image and the variable n becomes greater than or equal to the size X in the width direction of the image to be processed.

[0041] When it is determined that n·x ≧ X (step S14: Yes), the division number determination unit 121 determines whether the product of the size y in the height direction of the divided image and the variable m is greater than or equal to the size Y in the height direction of the image to be processed (m·y ≧ Y) (step S16).

[0042] When it is determined that m·y < Y (not m·y ≧ Y) (step S16: No), the division number determination unit 121 adds 1 to the variable m (step S17) and returns to the process of step S16. That is, the division number determination unit 121 repeats the processes of step S16 and step S17 until the product of the size y in the height direction of the divided image and the variable m becomes greater than or equal to the size Y in the width direction of the image to be processed.

[0043] When it is determined that m·y ≧ Y (step S16: Yes), the division number determination unit 121 determines that the division number N = n in the width direction (X direction) of the image to be processed and determines that the division number M = m in the height direction (Y direction) of the image to be processed (step S18). In this way, the division number determination unit 121 (image division unit 12) determines the division number N in the width direction of the image to be processed so that the product of the size x in the width direction of the divided image and the division number N in the width direction of the image to be processed is greater than or equal to the size X in the width direction of the image to be processed. Also, the division number determination unit 121 determines the division number M in the height direction of the image to be processed so that the product of the size y in the height direction of the divided image and the division number M in the height direction of the image to be processed is greater than or equal to the size Y in the height direction of the image to be processed.

[0044] Next, the division of the image to be processed by the division execution unit 122 will be described.

[0045] FIG. 6 is a flowchart showing an example of the operation of the division execution unit 122.

[0046] For example, the division execution unit 122 divides the image to be processed, starting from the upper left corner of the image to be processed. In this case, the division execution unit 122 divides the image to be processed, starting from the upper left corner coordinate (x i ,y i )=(0,0) (step S21).

[0047] The division execution unit 122 i+1 =x i Set +x and y j+1 =y j +y (step S22). As described above, x is the size of the divided image in the width direction, and y is the size of the divided image in the height direction.

[0048] The division execution unit 122 divides the image to be processed into a width direction x i x i+1 y in the vertical direction j From y j+1 The area up to is cut out as a divided image (step S23).

[0049] Next, the division execution unit 122 determines whether i=N (step S24).

[0050] If it is determined that i=N is not true (step S24: No), the division execution unit 122 adds 1 to i (step S25) and returns to the process of step S22. This process is repeated until i=N, and thus the division of the image to be processed in the width direction is repeated N times.

[0051] If it is determined that i=N (step S25: Yes), the division execution unit 122 determines whether j=M (step S26).

[0052] If it is determined that j=M is ​​not true (step S26: No), the division execution unit 122 adds 1 to j (step S27), sets i=1, and returns to the process of step S22. This process is repeated until j=M is ​​true, whereby the division of the image to be processed in the height direction is repeated M times.

[0053] If it is determined that j=M (step S26: Yes), the division execution unit 122 ends the process since the image to be processed has been divided into N×M divided images. According to the process described with reference to Fig. 6, the image to be processed can be covered entirely with the minimum number of divided images, so that the calculation process in the area detection unit 13 can be speeded up.

[0054] 7, when N·x>X and M·y>Y, the divided images may exceed the size of the image to be processed at the right and bottom ends of the image to be processed. In this case, the division execution unit 122 adds, for example, an image made of uniform pixels (for example, a black image made of black pixels) to the areas of the divided images that exceed the size of the image to be processed. In this way, the division execution unit 122 can generate divided images of uniform size.

[0055] However, in the above-mentioned method, black images tend to be concentrated in the divided images near the right and bottom edges of the processing target, and if the proportion of black images in a divided image is large, the learning effect will be low even if the divided image is used to learn a model.

[0056] Therefore, the division execution unit 122 may divide the image to be processed into multiple divided images so that the center of the area formed by arranging the divided images in the height and width directions by the determined number of divisions (hereinafter referred to as the "sum area of ​​the divided images") coincides with the center of the image to be processed, as shown in Figure 8.

[0057] In the example shown in Fig. 5, black images are concentrated in the divided images near the right and bottom edges of the image to be processed. On the other hand, by dividing the image to be processed so that the center of the image to be processed coincides with the image at the center of the union region of the divided images, the divided images at the top, bottom, left, and right edges of the image to be processed protrude from the image to be processed, as shown in Fig. 8, and black images can be distributed among these divided images.

[0058] The division execution unit 122 may divide the image to be processed so that the divided image does not extend beyond the image to be processed. Specifically, for example, as shown in FIG. 9, it is assumed that the union area of ​​the divided image is larger than the image to be processed and the divided image extends beyond the right end and the bottom end of the image to be processed. In this case, the division execution unit 122 moves the divided image extending beyond the right end of the image to be processed to the left and overlaps it with the divided image adjacent to the left, so that the divided image does not extend beyond the image to be processed. The division execution unit 122 moves the divided image extending beyond the bottom end of the image to be processed to the upward direction and overlaps it with the divided image adjacent to the upward direction, so that the divided image does not extend beyond the image to be processed. The division execution unit 122 moves the divided image extending beyond the right end and the bottom end of the image to the left and upward direction and overlaps it with the divided image adjacent to the left and upward, so that the divided image does not extend beyond the image to be processed.

[0059] In this way, when the union region of the divided images is larger than the image to be processed, the division execution unit 122 (image division unit 12) may overlap adjacent divided images to make the size of the union region of the divided images equal to the size of the image to be processed. This prevents black images from being added to the divided images that extend beyond the image to be processed, thereby preventing a decrease in the learning effect using the divided images.

[0060] Next, the operation of the image processing device 10 according to this embodiment will be described.

[0061] FIG. 10 is a flowchart showing an example of the operation of the image processing device 10 according to the present embodiment, and is a diagram for explaining an image processing method by the image processing device 10 according to the present embodiment.

[0062] The image dividing section 12 divides an image to be processed, which is input via the image input section 11 and contains a predetermined object, into a plurality of divided images of a predetermined size (step S31).

[0063] The region detection unit 13 detects, for each of the plurality of divided images, an object region which is a pixel region of an object in that divided image, and a degraded region which is a pixel region of a degraded portion of the object (step S32).

[0064] The information combination unit 14 generates an object detection result image by combining images of the object regions detected in each of the multiple divided images while maintaining the positional relationship between the multiple divided images. The information combination unit 14 also generates a degradation detection result image by combining images of the deteriorated regions detected in each of the multiple divided images while maintaining the positional relationship between the multiple divided images (step S33).

[0065] The diagnosis unit 15 diagnoses the deterioration of the object based on the generated object detection result image and deterioration detection result image. For example, the diagnosis unit 15 superimposes the object detection result image and the deterioration detection result image, and calculates the deterioration rate of the object from the ratio of the deteriorated area to the object area, etc.

[0066] Thus, the image processing device 10 according to the present embodiment includes an image division unit 12, an area detection unit 13, an information combination unit 14, and a diagnosis unit 15. The image division unit 12 divides an image to be processed, including a predetermined object, into a plurality of divided images of a predetermined size. The area detection unit 13 detects an object area, which is a pixel area of ​​the object in the divided image, and a deteriorated area, which is a pixel area of ​​a deteriorated part of the object, for each of the plurality of divided images. The information combination unit 14 generates an object detection result image by combining images of the object areas detected for each of the plurality of divided images while maintaining the positional relationship between the plurality of divided images, and generates a deterioration detection result image by combining images of the deteriorated areas detected for each of the plurality of divided images while maintaining the positional relationship between the plurality of divided images. The diagnosis unit 15 diagnoses deterioration of the object based on the object detection result image and the deterioration detection result image.

[0067] In addition, the image processing method by the image processing device 10 of this embodiment includes a step (step S31) in which the image division unit 12 divides an image to be processed including a specified object into a plurality of divided images of a specified size, a step (step S32) in which the area detection unit 13 detects, for each of the plurality of divided images, an object area which is the pixel area of ​​the object in the divided image and a deteriorated area which is the pixel area of ​​the deteriorated portion of the object, a step (step S33) in which the information combination unit 14 generates an object detection result image by combining images of the object areas detected for each of the plurality of divided images while maintaining the positional relationship between the plurality of divided images, and a deterioration detection result image by combining images of the deteriorated areas detected for each of the plurality of divided images while maintaining the positional relationship between the plurality of divided images, and a step (step S34) in which the diagnosis unit 15 diagnoses deterioration of the object based on the object detection result image and the deterioration detection result image.

[0068] By detecting the object region and the deteriorated region for the divided images obtained by dividing the image to be processed, the detection can be performed without reducing the number of pixels in the detection target region, so that the detection of the predetermined object and deterioration of the object can be made with high accuracy. Also, by detecting the object region and the deteriorated region for the divided images obtained by dividing the image to be processed, the sophistication of the processing performance required for detection can be suppressed.

[0069] Second embodiment FIG. 11 is a diagram showing an example of the configuration of an image processing device 10A according to a second embodiment of the present invention. In the image processing device 10 according to the first embodiment, divided images are used for detecting the object and the deterioration of the object. In this case, compared with the case where an image resized (reduced in size) by compression is used for learning and detection, the number of images used for learning and detection increases, and the calculation time increases. In addition, since infrastructure facilities generally have uniform shapes, the detection of infrastructure facilities in an image depends heavily on shape features. Therefore, when an image to be processed is divided into a plurality of divided images, the shape features of the infrastructure facilities that are the object of detection may be lost, and the detection rate may decrease. Therefore, in the image processing device 10A according to this embodiment, an image obtained by compressing the image to be processed is used for detecting the object. The configuration of the image processing device 10 will be described in detail below. In FIG. 11, the same components as those in FIG. 1 are given the same reference numerals, and the description will be omitted.

[0070] As shown in FIG. 11, the image processing device 10A according to this embodiment includes an image input unit 11A, an image division unit 12, an image compression unit 16, an object region detection unit 17, a degraded region detection unit 18, an information combination unit 14A, and a diagnosis unit 15.

[0071] The image input unit 11A receives an image to be processed. As described above, the image input to the image input unit 11A is, for example, an image of a specific infrastructure facility. The image input unit 11A outputs the input image to the image division unit 12 and the image compression unit 16.

[0072] The image compression unit 16 compresses the image to be processed output from the image input unit 11A to a predetermined size (normalized size). When compressing the image to be processed, the image compression unit 16 may add a predetermined image (for example, a black image) to the image to be processed and compress the image to have the same aspect ratio as the normalized size. This makes it possible to prevent a decrease in the detection accuracy of the model created by deep learning caused by a change in aspect ratio due to compression. The image compression unit 16 outputs the compressed image to the object region detection unit 17.

[0073] The object region detection unit 17 detects an object region, which is a pixel region of an object, in an image (compressed image) compressed by the image compression unit 16. Fig. 12 is a diagram showing an example of the configuration of the object region detection unit 17. In Fig. 12, an example will be described in which the object region detection unit 17 has a function of creating a model for detecting an object region from a compressed image, and a function of detecting an object region from the compressed image using the created model.

[0074] 12, object region detection unit 17 includes object detection learning unit 171 and object detection unit 172. As described above, object region detection unit 17 includes a function of creating a model for detecting an object region from a compressed image, and a function of detecting an object region from the compressed image using the created model. A compressed image used for creating the model is input to object detection learning unit 171. In addition, a compressed image in which object detection is performed is input to object detection unit 172.

[0075] The object detection learning unit 171 uses the compressed image and a mask image indicating an object region in the compressed image to create an object detector that detects an object region in an image by a deep learning method. The object detection learning unit 171 stores the created object detector in the object detection unit 172.

[0076] 12, an example in which the object region detection unit 17 has a function of creating an object detector has been described, but the present invention is not limited to this. The object detector may be created outside the image processing device 10A and stored in the object detection unit 172. In this case, the object region detection unit 17 does not need to include the object detection learning unit 171.

[0077] Object detection unit 172 detects an object region in the input compressed image (compressed image that is the target of object region detection) using the object detector created by object detection learning unit 171. Object detection unit 172 outputs an object detection result image, which is an image of the object region in the compressed image, to diagnosis unit 15 as a detection result of the object region.

[0078] Referring again to Fig. 11, the degraded area detection unit 18 receives a plurality of divided images obtained by dividing the image to be processed by the image division unit 12. For each of the plurality of divided images, the degraded area detection unit 18 detects a degraded area, which is a pixel area of ​​a degraded part of the object in the divided image. Fig. 13 is a diagram showing an example of the configuration of the degraded area detection unit 18. In Fig. 13, an example will be described in which the degraded area detection unit 18 has a function of creating a model for detecting a degraded area from the divided images and a function of detecting a degraded area from the divided images using the created model.

[0079] 13, the degraded area detection unit 18 includes a degradation detection learning unit 181 and a degradation detection unit 182. As described above, the degraded area detection unit 18 includes a function of creating a model for detecting a degraded area from a divided image, and a function of detecting a degraded area from the divided image using the created model. The divided image used for creating the model is input to the degradation detection learning unit 181. In addition, the compressed image in which the degraded area is to be detected is input to the degradation detection unit 182.

[0080] The degradation detection learning unit 181 uses the divided image and a mask image indicating a degraded area in the divided image to create a degradation detector that detects a degraded area in an image by a deep learning method. The degradation detection learning unit 181 stores the created degradation detector in the degradation detection unit 182.

[0081] The degradation detection unit 182 detects a degraded area in the input divided image (a divided image that is a target for detecting a degraded area) using the degradation detector created by the degradation detection learning unit 181. The degradation detection unit 182 outputs the detection result of the degraded area to the information combination unit 14A.

[0082] 13, an example in which the degraded area detection unit 18 has a function of creating a degradation detector has been described, but the present invention is not limited to this. The degradation detector may be created outside the image processing device 10A and stored in the degradation detection unit 182. In this case, the degraded area detection unit 18 does not need to include the degradation detection learning unit 181.

[0083] 11 again, the information combination unit 14A generates a degradation detection result image by combining images of the deteriorated areas detected for each of the multiple divided images while maintaining the positional relationship between the multiple divided images. The information combination unit 14A outputs the generated degradation detection result image to the diagnosis unit 15.

[0084] Diagnosis section 15 detects deterioration of the object based on the object detection result image output from object region detection section 17 and the deterioration detection result image output from information combination section 14A.

[0085] In the image processing device 10A according to this embodiment, the image to be processed is compressed, and the object region is detected from the compressed image. Therefore, the object region can be detected while maintaining the shape characteristics of the object in the image, and the detection accuracy of the object region can be improved. In addition, to detect the deteriorated region, divided images obtained by dividing the image to be processed are used, as in the image processing device 10 according to the first embodiment. Therefore, the deteriorated region can be detected without reducing the number of pixels in the deteriorated region, and the accuracy of the detection of the deteriorated region can be improved. In addition, by detecting the object region and the deteriorated region using the compressed image obtained by compressing the processing target and the divided images obtained by dividing the image to be processed, the sophistication of the processing performance required for detection can be suppressed.

[0086] Next, the operation of the image processing device 10A according to this embodiment will be described.

[0087] FIG. 14 is a flowchart showing an example of the operation of the image processing device 10A according to the present embodiment, and is a diagram for explaining an image processing method by the image processing device 10A according to the present embodiment.

[0088] The image dividing unit 12 divides a processing target image including a predetermined object into a plurality of divided images of a predetermined size (step S41).

[0089] The image compression unit 16 compresses the image to be processed to a predetermined size (step S42).

[0090] The object region detection unit 17 detects an object region, which is a pixel region of an object, in the image compressed by the image compression unit 16 (step S43).

[0091] The degraded region detection unit 18 detects a degraded region, which is a pixel region of a degraded portion of the object, for each of the plurality of divided images (step S44).

[0092] The information combination unit 14A generates a degradation detection result image by combining images of the deteriorated regions detected for each of the plurality of divided images while maintaining the positional relationships between the plurality of divided images (step S45).

[0093] The diagnosing unit 15 diagnoses degradation of the object based on the object detection result image, which is an image of the object region in the compressed image, and the degradation detection result image (step S46).

[0094] Thus, in this embodiment, the image processing device 10A includes an image division unit 12, an image compression unit 16, an object region detection unit 17, a deteriorated region detection unit 18, an information combination unit 14A, and a diagnosis unit 15. The image division unit 12 divides an image to be processed, including a predetermined object, into a plurality of divided images of a predetermined size. The image compression unit 16 compresses the image to be processed to a predetermined size. The object region detection unit 17 detects an object region, which is a pixel region of the object in the compressed image. The deteriorated region detection unit 18 detects a deteriorated region, which is a pixel region of a deteriorated part of the object in each of the plurality of divided images, for each of the plurality of divided images. The information combination unit 14A generates a deterioration detection result image by combining images of the deteriorated regions detected for each of the plurality of divided images while maintaining the positional relationship of the plurality of divided images. The diagnosis unit 15 diagnoses deterioration of the object based on the object detection result image, which is an image of the object region in the compressed image, and the deterioration detection result image.

[0095] In addition, the image processing method by the image processing device 10A of this embodiment includes a step (step S41) in which the image division unit 12 divides an image to be processed including a specified object into a plurality of divided images of a specified size, a step (step S42) in which the image compression unit 16 compresses the image to be processed to a specified size, a step (step S43) in which the object area detection unit 17 detects an object area which is a pixel area of ​​the object in the compressed image, a step (step S43) in which the deteriorated area detection unit 18 detects a deteriorated area which is a pixel area of ​​a deteriorated part of the object in each of the plurality of divided images, a step (step S44) in which the information combination unit 14A generates a deterioration detection result image by combining images of the deteriorated areas detected for each of the plurality of divided images while maintaining the positional relationship between the plurality of divided images, and a step (step S46) in which the diagnosis unit 15 diagnoses deterioration of the object based on the object detection result image which is an image of the object area in the compressed image and the deterioration detection result image.

[0096] By compressing the image to be processed and detecting the object region from the compressed image, the object region can be detected while maintaining the shape characteristics of the object in the image, thereby improving the detection accuracy of the object region. In addition, by using divided images obtained by dividing the image to be processed to detect the degraded region, the degraded region can be detected without reducing the number of pixels in the degraded region, thereby improving the detection accuracy of the degraded region. In addition, by detecting the object region and the degraded region using a compressed image obtained by compressing the object to be processed and divided images obtained by dividing the image to be processed, the sophistication of the processing performance required for detection can be suppressed.

[0097] (Third embodiment) Fig. 15 is a diagram showing an example of the configuration of an image processing device 10B according to the third embodiment of the present disclosure. In Fig. 15, the same components as those in Fig. 1 are denoted by the same reference numerals, and the description thereof will be omitted.

[0098] 15, the image processing device 10B according to this embodiment includes an image input unit 11, an image dividing unit 12, a region detection unit 13, an information combination unit 14, a diagnosis unit 15, and an image clipping unit 19. The image processing device 10B according to this embodiment differs from the image processing device 10 according to the first embodiment in that the image clipping unit 19 is added.

[0099] The image cutout unit 19 cuts out an image of a rectangular area including a predetermined object from the input image input to the image input unit 11 as an image to be processed, and outputs it to the image division unit 12. This reduces the size of the image input to each block after the image cutout unit 19, thereby speeding up the calculation process.

[0100] 15, an example in which the image processing device 10B is configured by adding the image cropping unit 19 to the image processing device 10 according to the first embodiment has been described, but the present invention is not limited to this. As shown in FIG. 16, the image processing device 10C may be configured by adding the image cropping unit 19 to the image processing device 10A according to the second embodiment. In this case, the image input unit 11 outputs an input image to the image cropping unit 19, and the image cropping unit 19 crops out an image of a rectangular area including a predetermined object from the input image, and outputs the image to the image division unit 12 and the image compression unit 16.

[0101] Next, the hardware configuration of the image processing apparatuses 10, 10A, 10B, and 10C according to the above-mentioned embodiments will be described.

[0102] Fig. 17 is a diagram showing an example of a hardware configuration of the image processing device 10, 10A, 10B, 10C according to each of the above-mentioned embodiments. Fig. 17 shows an example of a hardware configuration of the image processing device 10 when the image processing device 10, 10A, 10B, 10C is configured by a computer capable of executing program instructions. Here, the computer may be a general-purpose computer, a dedicated computer, a workstation, a PC (Personal computer), an electronic notepad, etc. The program instructions may be program code, code segments, etc. for performing necessary tasks.

[0103] 17, the image processing devices 10, 10A, 10B, and 10C each have a processor 21, a ROM (Read Only Memory) 22, a RAM (Random Access Memory) 23, a storage 24, an input unit 25, a display unit 26, and a communication interface (I / F) 27. Each component is connected to each other so as to be able to communicate with each other via a bus 29. The processor 21 is specifically a CPU (Central Processing Unit), an MPU (Micro Processing Unit), a GPU (Graphics Processing Unit), a DSP (Digital Signal Processor), a SoC (System on a Chip), or the like, and may be composed of a plurality of processors of the same type or different types.

[0104] The processor 21 is a control unit that controls each component and executes various arithmetic processing. That is, the processor 21 reads a program from the ROM 22 or the storage 24, and executes the program using the RAM 23 as a working area. The processor 21 controls each component and executes various arithmetic processing according to the program stored in the ROM 22 or the storage 24. In this embodiment, the ROM 22 or the storage 24 stores a program for causing a computer to function as the image processing device 10, 10A, 10B, 10C according to the present disclosure. The program is read and executed by the processor 21, thereby realizing each component of the image processing device 10, 10A, 10B, 10C described above.

[0105] The program may be provided in a form stored in a non-transitory storage medium such as a CD-ROM (Compact Disk Read Only Memory), a DVD-ROM (Digital Versatile Disk Read Only Memory), a USB (Universal Serial Bus) memory, etc. The program may also be provided in a form downloaded from an external device via a network.

[0106] The ROM 22 stores various programs and various data. The RAM 23 temporarily stores programs or data as a working area. The storage 24 is configured with an HDD (Hard Disk Drive) or SSD (Solid State Drive) and stores various programs including an operating system and various data.

[0107] The input unit 25 includes a pointing device such as a mouse, and a keyboard, and is used to perform various input operations.

[0108] The display unit 26 is, for example, a liquid crystal display, and displays various information. The display unit 26 may function as the input unit 25 by adopting a touch panel system.

[0109] The communication interface 27 is an interface for communicating with other devices, for example, an interface for a LAN. For example, an image to be cut out as a cut-out image is input to the image input unit 11 via the communication interface 27. Also, for example, the cut-out image after standardization is output to the outside via the communication interface 27.

[0110] A computer can be suitably used to function as each of the image processing devices 10, 10A, 10B, and 10C described above. Such a computer can be realized by storing a program describing the processing contents for realizing the functions of each of the image processing devices 10, 10A, 10B, and 10C in a storage unit of the computer, and reading and executing the program by a processor of the computer. That is, the program can cause the computer to function as the image processing devices 10, 10A, 10B, and 10C described above. The program can also be recorded in a non-transitory storage medium. The program can also be provided via a network.

[0111] The following supplementary notes are further disclosed regarding the above embodiment.

[0112] [Additional note 1] Memory, A control unit connected to the memory; Equipped with The control unit is Dividing an image to be processed, including a predetermined object, into a plurality of divided images of a predetermined size; Detecting an object region, which is a pixel region of the object in each of the plurality of divided images, and a degraded region, which is a pixel region of a degraded portion of the object, in the divided image; generating an object detection result image by combining images of the object regions detected for each of the plurality of divided images while maintaining the positional relationship between the plurality of divided images, and generating a degradation detection result image by combining images of the deteriorated regions detected for each of the plurality of divided images while maintaining the positional relationship between the plurality of divided images; An image processing device that diagnoses deterioration of the object based on the object detection result image and the deterioration detection result image.

[0113] [Additional note 2] Memory, A control unit connected to the memory; Equipped with The control unit is Dividing an image to be processed, including a predetermined object, into a plurality of divided images of a predetermined size; compressing the image to be processed to a predetermined size; detecting an object region in the compressed image, the object region being a pixel region of the object; Detecting a deteriorated region, which is a pixel region of a deteriorated portion of the object in each of the plurality of divided images, generating a degradation detection result image by combining the images of the deteriorated regions detected for each of the plurality of divided images while maintaining the positional relationship between the plurality of divided images; An image processing device that diagnoses deterioration of the object based on an object detection result image, which is an image of the object region in the compressed image, and the deterioration detection result image.

[0114] [Additional note 3] In the image processing device according to claim 1, The control unit determines the number of width-wise divisions of the image to be processed so that the product of the width-wise size of the divided image and the number of width-wise divisions of the image to be processed is greater than or equal to the width-wise size of the image to be processed, and determines the number of height-wise divisions of the image to be processed so that the product of the height-wise size of the divided image and the number of height-wise divisions of the image to be processed is greater than or equal to the height-wise size of the image to be processed.

[0115] [Additional note 4] In the image processing device according to claim 3, The control unit of the image processing device divides the image to be processed into the multiple divided images so that the center of the union area of ​​the divided images coincides with the center of the image to be processed when the union area of ​​the divided images formed by arranging the divided images in the height and width directions by the determined number of divisions is larger than the image to be processed.

[0116] [Additional note 5] In the image processing device according to claim 4, The control unit, when the union area of ​​the divided images formed by arranging the divided images in the height and width directions by the determined number of divisions is larger than the image to be processed, overlaps adjacent divided images to make the size of the union area of ​​the divided images equal to the size of the image to be processed.

[0117] [Additional note 6] In the image processing device according to claim 1, The control unit cuts out an image of a rectangular area including the specified object from an input image as the image to be processed.

[0118] [Additional note 7] An image processing method by an image processing device, comprising: Dividing an image to be processed, including a predetermined object, into a plurality of divided images of a predetermined size; Detecting an object region, which is a pixel region of the object in each of the plurality of divided images, and a degraded region, which is a pixel region of a degraded portion of the object, in the divided image; generating an object detection result image by combining images of the object regions detected for each of the plurality of divided images while maintaining the positional relationship between the plurality of divided images, and generating a degradation detection result image by combining images of the deteriorated regions detected for each of the plurality of divided images while maintaining the positional relationship between the plurality of divided images; The image processing method further comprises diagnosing deterioration of the object based on the object detection result image and the deterioration detection result image.

[0119] [Additional note 8] A non-transitory storage medium storing a program executable by a computer, the non-transitory storage medium storing the program causing the computer to operate as the image processing device described in appended claim 1.

[0120] Although the above-mentioned embodiment has been described as a representative example, it is obvious to those skilled in the art that many modifications and substitutions can be made within the spirit and scope of the present disclosure. Therefore, the present invention should not be interpreted as being limited by the above-mentioned embodiment, and various modifications or changes can be made without departing from the scope of the claims. For example, it is possible to combine multiple building blocks shown in the configuration diagram of the embodiment into one, or to divide one building block. [Explanation of symbols]

[0121] 10, 10A, 10B, 10C Image processing device 11, 11A Image input section 12 Image division section 13 Area detection section 14,14A Information connection section 15 Diagnostics Department 16 Image Compression Section 17 Object region detection section 18 Degraded area detection section 19 Image Extraction Section 121 Division number determination section 122 Division Execution Unit 131 Model Construction Department 132,172 Object detection section 133,182 Deterioration detection unit 1311,171 Object detection learning unit 1312,181 Deterioration detection learning unit 21 Processors 22 ROM 23 RAM 24 Storage 25 Input section 26 Display section 27 Communication I / F 29 Bus

Claims

1. an image division unit that divides an image to be processed, which includes a predetermined object, into a plurality of divided images of a predetermined size; a region detection unit that detects, for each of the plurality of divided images, an object region that is a pixel region of the object in the divided image and a degraded region that is a pixel region of a degraded portion of the object; an information combination unit that generates an object detection result image by combining images of the object regions detected for each of the plurality of divided images while maintaining the positional relationship between the plurality of divided images, and a degradation detection result image by combining images of the deteriorated regions detected for each of the plurality of divided images while maintaining the positional relationship between the plurality of divided images; a diagnosis unit that diagnoses deterioration of the object based on the object detection result image and the deterioration detection result image.

2. an image division unit that divides an image to be processed, which includes a predetermined object, into a plurality of divided images of a predetermined size; an image compression unit that compresses the image to be processed to a predetermined size; an object region detection unit that detects an object region, which is a pixel region of the object in the compressed image; a deteriorated region detection unit that detects, for each of the plurality of divided images, a deteriorated region that is a pixel region of a deteriorated portion of the object in the corresponding divided image; an information combination unit that generates a degradation detection result image by combining images of the deteriorated areas detected for each of the plurality of divided images while maintaining a positional relationship between the plurality of divided images; an image processing device comprising: a diagnosis unit that diagnoses deterioration of the object based on an object detection result image, which is an image of the object region in the compressed image, and the deterioration detection result image.

3. 3. The image processing device according to claim 1, The image division unit determines the number of divisions in the width direction of the image to be processed so that the product of the width direction size of the divided image and the number of divisions in the width direction of the image to be processed is greater than or equal to the width direction size of the image to be processed, and determines the number of divisions in the height direction of the image to be processed so that the product of the height direction size of the divided image and the number of divisions in the height direction of the image to be processed is greater than or equal to the height direction size of the image to be processed.

4. 4. The image processing device according to claim 3, The image division unit divides the image to be processed into the multiple divided images so that the center of the union area of ​​the divided images coincides with the center of the image to be processed when the union area of ​​the divided images formed by arranging the divided images in the height and width directions by the determined number of divisions is larger than the image to be processed.

5. 4. The image processing device according to claim 3, The image processing device, wherein when the union area of ​​the divided images formed by arranging the divided images in the height and width directions by the determined number of divisions is larger than the image to be processed, the image division unit overlaps adjacent divided images to make the size of the union area of ​​the divided images equal to the size of the image to be processed.

6. 6. The image processing device according to claim 1, The image processing device further comprises an image cutout unit that cuts out an image of a rectangular area including the predetermined object from an input image as the image to be processed.

7. An image processing method by an image processing device, comprising: Dividing an image to be processed, including a predetermined object, into a plurality of divided images of a predetermined size; detecting, for each of the plurality of divided images, an object region that is a pixel region of the object in the divided image, and a degraded region that is a pixel region of a degraded portion of the object; generating an object detection result image by combining images of the object regions detected for each of the plurality of divided images while maintaining the positional relationship between the plurality of divided images, and generating a degradation detection result image by combining images of the deteriorated regions detected for each of the plurality of divided images while maintaining the positional relationship between the plurality of divided images; diagnosing deterioration of the object based on the object detection result image and the deterioration detection result image.

8. A program for causing a computer to operate as the image processing device according to any one of claims 1 to 6.

Images