Area detection device, area detection method, and program

Abstract

A region detection device (1) according to the present disclosure comprises: a line segment detection unit (32) that detects a line segment from a captured image which contains an image of an object having a linear band shape; a main line segment extraction unit (33) that extracts a main line segment (SL) which is a line segment having a length of not less than a predetermined value; a correction unit (34) that generates, as a corrected image, a rotated image obtained by rotating the captured image so that a main line segment (SL) becomes perpendicular or parallel to the arrangement direction of pixels in the captured image; and a region detection unit (35) that detects an object region which shows the image of the object from the corrected image.

Description

[Technical field]

[0001] The present disclosure relates to an area detection device, an area detection method, and a program. [Background technology]

[0002] It is known that an object region showing an image of an object included in a captured image is detected by using an image analysis method such as deep learning. When detecting such an object region, there may be cases where a region showing the image of the object is overlooked, resulting in a non-detection, or where a region not showing the image of the object is erroneously detected, including an overdetection.

[0003] As an example, an example of detecting an area showing an image of a linear band-shaped object from a captured image described with reference to FIG. 13A will be described. In this example, the objects are pipes PL1 and PL2. In the captured image described with reference to FIG. 13A, the images of the pipes PL1 and PL2 extend obliquely with respect to the pixel arrangement direction (x-axis direction). The object areas (areas showing the images of the pipes PL1 and PL2) in the captured image described with reference to FIG. 13A are areas R1_true and R2_true shown in FIG. 13B, respectively. However, by using a conventional image analysis method, an object area R90 is detected in FIG. 13C as an area showing the images of the pipes PL1 and PL2, and the object area R90 within the range of the rectangle N90 is recognized as showing one object. As shown in FIG. 13C, the object area R90 does not include a part of the area showing the images of the pipes PL1 and PL2, and non-detection occurs.

[0004] As another example, an example of detecting an area showing an image of a linear band-shaped object from a captured image described with reference to FIG. 14A will be described. In this example, the objects are pipes PL1 and PL2. In the captured image described with reference to FIG. 14A, the images of the pipes PL1 and PL2 extend in a direction substantially perpendicular to the pixel arrangement direction (x-axis direction). In the captured image described with reference to FIG. 14A, the areas showing the images of the pipes PL1 and PL2 are the areas R1_true and R2_true shown in FIG. 14B. In this example, by using a conventional image analysis method, the object area R91 and the object area R92 in FIG. 14C are detected as areas showing the images of the pipes PL1 and PL2. Then, the object area R91 and the object area R92 in the ranges of the rectangles N91 and N92 are recognized as showing one object each. As shown in FIG. 14C, the entire regions in which the images of the pipelines PL1 and PL2 are displayed are included in the object regions R91 and R92, respectively, and no undetected regions occur.

[0005] In this way, non-detection may or may not occur depending on the direction in which the image of the object extends in the captured image. In order to solve this problem, Non-Patent Document 1 describes performing tilt correction so that the inspection surface is oriented squarely relative to the imaging surface in a captured image generated by capturing an image of the object in a state in which the inspection surface is tilted relative to the imaging surface. In addition, it is known that tilt correction uses an affine transformation that performs projective transformation by setting four points in the area surrounding the image of the object. [Prior art documents] [Non-patent literature]

[0006] [Non-Patent Document 1] Image processing service Image-based infrastructure inspection service Inspection EYE for Infrastructure: Canon, [online], [Searched on October 14, 2021], Internet<URL:https: / / cweb.canon.jp / imaging-solutions / lineup / inspection-eye / image-processing / > Summary of the Invention [Problem to be solved by the invention]

[0007] However, in tilt correction using affine transformation, it is necessary to manually select the four points. Depending on how the four points are selected, some of the areas showing the images of the pipes may not be included in the detected object area, resulting in failure to detect the object area with high accuracy.

[0008] In view of the above circumstances, an object of the present disclosure is to provide an area detection device, an area detection method, and a program that are capable of detecting an object area with high accuracy. [Means for solving the problem]

[0009] In order to solve the above problems, the area detection device of the present disclosure includes a line segment detection unit that detects lines from an captured image including an image of an object having a linear band shape; a main line segment extraction unit that extracts a main line segment, which is the line segment having a length equal to or greater than a predetermined value; a correction unit that generates a rotated image as a corrected image by rotating the captured image so that the main line segment is perpendicular or parallel to the arrangement direction of pixels in the captured image; and an area detection unit that detects an object area showing the image of the object from the corrected image.

[0010] In addition, in order to solve the above problem, the area detection method disclosed herein includes the steps of detecting lines from an captured image including an image of an object having a linear band shape, extracting a main line segment, which is the line segment having a length equal to or greater than a predetermined value, generating a rotated image as a corrected image by rotating the captured image so that the main line segment is perpendicular or parallel to the arrangement direction of pixels in the captured image, and detecting an object area showing the image of the object from the corrected image.

[0011] In order to solve the above problem, a program according to the present disclosure causes a computer to function as the above-mentioned area detection device. Effect of the Invention

[0012] According to the area detection device, area detection method, and program of the present disclosure, an object area can be detected with high accuracy. [Brief description of the drawings]

[0013] [Figure 1] 1 is a schematic diagram of a region detection system according to a first embodiment. [Figure 2A] 2 is a diagram showing an example of a captured image acquired by the image acquisition device shown in FIG. 1; [Figure 2B] FIG. 2B is a diagram showing line segments extracted from the captured image shown in FIG. 2A. [Figure 2C] 2B is a diagram in which a main line segment detected from the line segment shown in FIG. 2B is emphasized in the captured image shown in FIG. 2A. [Figure 3A] 2D is a diagram showing the angle of the direction in which the main line segment extends with respect to the pixel array direction in the captured image shown in FIG. 2C. FIG. [Figure 3B] 3B is a diagram showing a rotated image obtained by rotating the captured image shown in FIG. 3A. FIG. [Figure 3C] 3C is a diagram showing an example of a corrected image obtained by reducing the captured image shown in FIG. 3B. FIG. [Figure 3D] 3C is a diagram showing another example of a corrected image obtained by reducing the captured image shown in FIG. 3B. FIG. [Figure 4]FIG. 3C is a diagram showing an image having the same size as the captured image, including a portion of the rotated image shown in FIG. 3B. [Diagram 5] FIG. 3D is a diagram showing object region information generated from the corrected image shown in FIG. 3C. [Figure 6] 2 is a flowchart showing an example of an operation of the area detection device shown in FIG. [Figure 7] FIG. 11 is a schematic diagram of a region detection system according to a second embodiment. [Figure 8] 8 is a flowchart showing an example of the operation of the area detection device shown in FIG. [Figure 9] FIG. 11 is a schematic diagram of a region detection system according to a third embodiment. [Figure 10] 10 is a flowchart showing an example of the operation of the area detection device shown in FIG. [Figure 11] FIG. 6 is a diagram showing the object region information shown in FIG. 5 rotated in a reverse direction. [Figure 12] FIG. 2 is a hardware block diagram of the area detection device. [Figure 13A] FIG. 4 is a diagram showing an example of a captured image. [Figure 13B] 13B is a diagram showing an object region in the captured image shown in FIG. 13A. FIG. [Figure 13C] 13B is a diagram showing an object region detected from the captured image shown in FIG. 13A. FIG. [Figure 14A] FIG. 11 is a diagram showing another example of a captured image. [Figure 14B] 14B is a diagram showing an object region in the captured image shown in FIG. 14A. FIG. [Figure 14C] 14B is a diagram showing an object region detected from the captured image shown in FIG. 14A. FIG. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0014] <<First embodiment>> The overall configuration of the first embodiment will be described with reference to FIG.

[0015] As shown in FIG. 1, the area detection system 100 according to the first embodiment includes an image acquisition device 1, an image saving device 2, an area detection device 3, and a data storage device 4.

[0016] <Configuration of image acquisition device> The image capture device 1 may be configured by a camera including an optical element, an imaging element, and an output interface. The output interface is an interface for outputting information.

[0017] The image acquisition device 1 acquires a captured image of an object having a linear band shape. As shown in Fig. 2A, the captured image is composed of pixels arranged two-dimensionally in a predetermined direction (x-axis direction) and a direction perpendicular to the predetermined direction (y-axis direction). The captured image is preferably an RGB image.

[0018] The object may be a component to be inspected. The object having a linear band shape may be a component that constitutes a structure and extends in one direction. In the example shown in FIG. 2A, the objects having a linear band shape in the captured image are pipes PL1 and PL2. In the following description, pipes PL1 and PL2 may be simply referred to as "pipes PL."

[0019] In the example shown in Fig. 2A, in the captured image, the image of the pipeline PL extends obliquely with respect to the pixel arrangement direction (x-axis direction). In this example, the captured image includes images of structures such as the walls, beams, and ceiling of the building in which the pipeline PL is arranged, in addition to the pipeline PL, and Fig. 2A shows the contour Ct of the image of the structure. In this example, the captured image also includes images of a damaged portion Cr of the structure and a damaged portion Rs of the pipeline PL including rust, dirt, defects, etc. In Fig. 2A, the symbols Ct, Cr, and Rs are respectively attached only to the contour of the structure, the damaged portion of the structure, and a part of the damaged portion of the pipeline PL.

[0020] Moreover, the image acquisition device 1 outputs the captured image to the image storage device 2.

[0021] <Image storage device configuration> The image storage device 2 shown in FIG. 1 may be a PC (Personal Computer), a tablet terminal, or the like. The image storage device 2 may be configured by a computer including a memory, a controller, an input interface, and an output interface. The memory may be configured by a HDD (Hard Disk Drive), an SSD (Solid State Drive), an EEPROM (Electrically Erasable Programmable Read-Only Memory), a ROM (Read-Only Memory), a RAM (Random Access Memory), or the like. The controller may be configured by dedicated hardware such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field-Programmable Gate Array), or may be configured by a processor, or may be configured including both. The input interface may be a pointing device, a keyboard, a mouse, or the like. The input interface may also be an interface that accepts input of information received by a communication interface. For the communication interface, a standard such as Ethernet (registered trademark), FDDI (Fiber Distributed Data Interface), or Wi-Fi (registered trademark), may be used.

[0022] The image storage device 2 receives an input of a captured image acquired by the image acquisition device 1, and stores the captured image. The image storage device 2 also outputs the captured image to the area detection device 3.

[0023] <Configuration of area detection device> The area detection device 3 includes an input unit 31, a line segment detection unit 32, a main line segment extraction unit 33, a correction unit 34, an area detection unit 35, and an output unit 36. The input unit 31 is configured by an input interface. The line segment detection unit 32, the main line segment extraction unit 33, the correction unit 34, and the area detection unit 35 are configured by a controller. The output unit 36 ​​is configured by an output interface.

[0024] The input unit 31 accepts input of image data indicating a captured image such as the example shown in Fig. 2A, which is stored in the image storage device 2. The input unit 31 may accept input of image data from the image acquisition device 1 without going through the image storage device 2.

[0025] Furthermore, the input unit 31 may be configured by an input interface and may further be configured by a controller. In such a configuration, the input unit 31 may assign an identifier to the image data that has been received as input, for uniquely identifying the image data. The identifier may be, for example, a number. The identifier may also be a number that is incremented by a predetermined value in the order in which the image data is input. The predetermined value may be 1. This allows the results of processing by each functional unit, which will be described later, to be associated with the image data, even when multiple pieces of image data are received as input.

[0026] The line segment detection unit 32 performs pre-processing for the captured image indicated by the image data received by the input unit 31, in order for the area detection unit 35, which will be described later in detail, to perform processing. Specifically, the line segment detection unit 32 detects lines from the captured image including an image of an object having a linear band shape. The line segment detection unit 32 can detect lines from the captured image using a general image processing method such as a Hough transform, a probabilistic Hough transform, or an LSD (Line Segment Detector). For example, the line segment detection unit 32 detects lines as shown in white in FIG. 2B using a Hough transform from the captured image shown in FIG. 2A as an example.

[0027] 2C from the lines detected by the line segment detection unit 32. The predetermined value is a value greater than the detection threshold described above, and can be set appropriately depending on the size of the object, the settings of the image acquisition device 1, the distance between the image acquisition device 1 and the object, etc. Furthermore, the main line segment extraction unit 33 may extract the main line segment SL, which is the line segment with the longest length, from the lines detected by the line segment detection unit 32.

[0028] The line segment detection unit 32 detects multiple lines of various lengths due to images of various subjects (for example, structures other than the object (beams, columns, ceilings, etc.), damaged parts Cr of the structures, damaged parts Rs of the object, etc.) contained in the captured image. In contrast, the image of the pipeline PL, which is an object having a linear band shape, extends at a constant length, so the main line segment SL, which is a line segment having a length equal to or greater than a predetermined value, is expected to be a line segment extending in the extension direction of the image of the pipeline PL. In other words, the main line segment extraction unit 33 can extract a line segment extending in the extension direction of the object as the main line segment SL.

[0029] The line segment detection unit 32 described above may detect line segments having a length equal to or greater than a preset detection threshold, or may detect all line segments. In a configuration in which the line segment detection unit 32 detects all line segments, it can detect more line segments than when it detects line segments having a length equal to or greater than a preset detection threshold. This allows the main line segment extraction unit 33 to extract the main line segment SL from more line segments, thereby enabling the main line segment SL to be extracted with higher accuracy.

[0030] The correction unit 34 generates, as a corrected image, a rotated image by rotating the captured image so that the main line segment SL is perpendicular or parallel to the arrangement direction of pixels in the captured image.

[0031] Specifically, first, the correction unit 34 calculates the angle θ (see FIG. 3A) that the extension direction of the main line segment SL extracted by the main line segment extraction unit 33 makes with respect to the arrangement direction (x-axis direction) of pixels in the captured image.

[0032] Then, the correction unit 34 generates a rotated image by rotating the captured image so that the angle θ becomes 0° or 90°, as shown in Fig. 3B. For example, the correction unit 34 may rotate the captured image by (90°-θ) using the center of the captured image as the center point of rotation so that the angle θ becomes 90°. The correction unit 34 may generate an image by rotating the captured image around an arbitrary point of the captured image, and translate the image so that the center of the image becomes the center of the captured image, thereby generating a rotated image.

[0033] The correction unit 34 may generate a rotated image by rotating the captured image using, for example, an image processing library such as Opencv or pillow, or image processing software. In this case, it is preferable that the correction unit 34 uses an affine transformation to rotate the captured image based on an arbitrary angle θ.

[0034] The correction unit 34 may also generate a corrected image by reducing the rotated image so that the entire rotated image is included in the entire area of ​​the captured image. Specifically, the correction unit 34 may generate a corrected image by reducing the rotated image so that the entire rotated image is included in the entire area of ​​the captured image. As shown in FIG. 3C, the correction unit 34 may generate a corrected image by reducing the rotated image so that the area is maximized while the entire rotated image is included in the entire area of ​​the captured image. As shown in FIG. 3D, the correction unit 34 may generate a corrected image by further reducing the rotated image compared to the case shown in FIG. 3C.

[0035] If the image obtained by rotating the captured image is not reduced, a portion (missing portion) that is not included in the entire area of ​​the captured image will be generated in the corrected image. In the example shown in FIG. 4, a part of the upper end of the image of the pipeline PL2 shown in FIG. 3A and FIG. 3B is not shown, and the part is a missing portion. In addition, in the example shown in FIG. 4, a part of the lower end of the images of the pipelines PL1 and PL2 shown in FIG. 3A and FIG. 3B is not shown, and the part is a missing portion. In response to this, as shown in FIG. 3B and FIG. 3C, the correction unit 34 generates a corrected image by reducing the rotated image, thereby preventing the occurrence of missing portions.

[0036] In this way, the correction unit 34 generates a corrected image, so that the image of the pipeline PL extends in a direction horizontal or perpendicular to the pixel array direction in the corrected image, and therefore the aspect ratio of a rectangle formed by line segments along the pixel array direction surrounding the image of the pipeline PL becomes large.

[0037] The area detection unit 35 detects an object area showing an image of an object from the corrected image. The area detection unit 35 can detect the object area by any method. For example, the area detection unit 35 can detect the object area by a method using deep learning. Specifically, the area detection unit 35 may detect the image of the inspection object by using a bounding box (for example, YOLO (You Only Look Once)). The area detection unit 35 may also detect the image of the inspection object by using class-specific segmentation by instance segmentation (for example, Mask-R-CNN) or the like.

[0038] The output unit 36 ​​outputs object region information indicating the object region detected by the region detection unit 35. The object region information can be, for example, information indicating an image in which a predetermined color or pattern is applied to the object region of the corrected image. The object region information can also be information indicating an image in which a rectangle surrounding the object region is superimposed on the corrected image. In the example shown in FIG. 5, the object region information is information indicating an image in which a predetermined pattern (halftone dot pattern) is applied to the object regions R1 and R2 of the corrected image, and rectangles N1 and N2 surrounding the object regions R1 and R2, respectively, are superimposed. Note that in FIG. 5, hatching indicating an image of the damaged portion Rs of the pipeline PL shown in FIGS. 2A, 2C, and 3A to 3D is omitted.

[0039] The output unit 36 ​​may output the object region information to the data storage device 4 via a communication network. The output unit 36 ​​may output the object region information to a display device configured with an organic EL (Electro Luminescence) panel, a liquid crystal panel, or the like.

[0040] <Data storage device configuration> 1 is configured by a computer equipped with a memory, a controller, and an input interface. The data storage device 4 stores the object region information output from the region detection device 3.

[0041] <Operation of area detection device> Here, the operation of the area detection device 3 according to the first embodiment will be described with reference to Fig. 6. Fig. 6 is a flowchart showing an example of the operation of the area detection device 3 according to the first embodiment. The operation of the area detection device 3 described with reference to Fig. 6 corresponds to an example of an area detection method of the area detection device 3 according to the first embodiment.

[0042] In step S11, the input unit 31 accepts input of image data indicating a captured image stored in the image storage device 2. The input unit 31 may accept input of image data from the image acquisition device 1 without going through the image storage device 2.

[0043] In step S12, the line segment detection unit 32 detects line segments from a captured image including an image of an object having a linear band shape. In this embodiment, the captured image is the captured image represented by the image data received in step S11.

[0044] In step S13, the main line segment extraction unit 33 extracts a main line segment SL, which is a line segment having a length equal to or greater than a predetermined value.

[0045] In step S14, the correction unit 34 generates a rotated image as a corrected image by rotating the captured image so that the main line segment SL is perpendicular or parallel to the arrangement direction of pixels in the captured image. Here, the correction unit 34 may generate, as the corrected image, an image by reducing the rotated image so that the entire rotated image is included in the entire area of ​​the captured image.

[0046] In step S15, the area detection unit 35 detects an object area showing an image of an object from the corrected image.

[0047] In step S16, the output unit 36 ​​outputs object region information indicating the object region.

[0048] It should be noted that the area detection device 3 does not need to execute step S1. In such a configuration, the line segment detection unit 32 may detect lines from a captured image that is pre-stored in the area detection device 3 or generated by the area detection device 3. Also, the area detection device 3 does not need to execute step S16. In such a configuration, the area detection device 3 may include a storage unit configured by a memory, and the storage unit may store the object area information.

[0049] As described above, according to the first embodiment, the area detection device 3 includes a line detection unit 32 that detects lines from a captured image including an image of an object having a linear band shape, a main line extraction unit 33 that extracts a main line segment SL, which is a line segment having a length equal to or greater than a predetermined value, a correction unit 34 that generates, as a corrected image, a rotated image by rotating the captured image so that the main line segment SL is perpendicular or parallel to the arrangement direction of pixels in the captured image, and an area detection unit 35 that detects an object area showing the image of the object from the corrected image.

[0050] As described with reference to FIG. 14A to FIG. 14C, when an image of an object such as a pipeline PL extends obliquely with respect to the pixel arrangement direction, it is known that the object region cannot be detected with high accuracy due to the small aspect ratio of the rectangle formed by the line segments along the pixel arrangement direction surrounding the image of the object. In contrast, the region detection device 3 of the present embodiment generates a corrected image by rotating and reducing the captured image so that the main line segment SL is perpendicular or parallel to the pixel arrangement direction, so that the image of the object extends in the pixel arrangement direction in the corrected image. Therefore, the aspect ratio of the rectangle formed by the line segments along the pixel arrangement direction surrounding the image of the object becomes large, so that the region detection device 3 can detect the object region with high accuracy. Accordingly, the region detection device 3 can inspect the object with higher accuracy by further extracting the region in which the object is damaged in the object region, and can appropriately maintain the safety of buildings, facilities, etc. formed using the object.

[0051] <<Second embodiment>> The overall configuration of the second embodiment will be described with reference to Fig. 7. In the second embodiment, the same functional units as those in the first embodiment are denoted by the same reference numerals, and the description thereof will be omitted.

[0052] As shown in FIG. 7, the area detection system 100-1 according to the second embodiment includes an image acquisition device 1, an image saving device 2, an area detection device 3-1, and a data storage device 4.

[0053] <Configuration of area detection device> The area detection device 3-1 includes an input unit 31, a line segment detection unit 32, a main line segment extraction unit 33, a correction unit 34, an area detection unit 35, an output unit 36, and an edge detection unit 37. The edge detection unit 37 is constituted by a controller.

[0054] The edge detection unit 37 detects edges from the captured image. Specifically, the edge detection unit 37 detects edges based on discontinuity in luminance in the captured image. For example, the edge detection unit 37 calculates the difference in luminance between a pixel in the captured image and a pixel adjacent to the pixel, and can detect a pixel whose absolute value of the difference is greater than a predetermined edge threshold as an edge.

[0055] The line segment detection unit 32 detects line segments from the edges detected by the edge detection unit 37 .

[0056] <Operation of area detection device> Here, the operation of the area detection device 3-1 according to the second embodiment will be described with reference to Fig. 8. Fig. 8 is a flowchart showing an example of the operation of the area detection device 3-1 according to the second embodiment. The operation of the area detection device 3-1 described with reference to Fig. 8 corresponds to an example of an area detection method of the area detection device 3-1 according to the second embodiment.

[0057] The area detection device 3-1 executes the process of step S21. The process of step S21 is the same as the process of step S11 in the first embodiment.

[0058] In step S22, the edge detection unit 37 detects edges from the captured image.

[0059] In step S23, the line segment detection unit 32 detects line segments from the edges detected in step S22.

[0060] Next, the area detection device 3-1 executes the processes from step S24 to step S27. The processes from step S24 to step S27 are the same as the processes from step S13 to step S16 in the first embodiment.

[0061] As described above, according to the second embodiment, the area detection device 3-1 further includes an edge detection unit 37 that detects edges from the captured image, and the line detection unit 32 detects lines from the edges. This allows the area detection device 3-1 to detect the boundaries of the image of the object, thereby more appropriately generating a corrected image such that the image of the object extends in the pixel array direction. Therefore, the area detection device 3-1 can detect the object area with higher accuracy compared to a configuration that directly detects lines from the captured image.

[0062] <<Third embodiment>> The overall configuration of the third embodiment will be described with reference to Fig. 9. In the third embodiment, the same functional units as those in the second embodiment are denoted by the same reference numerals, and the description thereof will be omitted.

[0063] As shown in FIG. 9, the area detection system 100-2 according to the third embodiment includes an image acquisition device 1, an image saving device 2, an area detection device 3-2, and a data storage device 4.

[0064] <Configuration of area detection device> The area detection device 3-2 includes an input unit 31, a line segment detection unit 32, a main line segment extraction unit 33, a correction unit 34, an area detection unit 35, an output unit 36, an edge detection unit 37, and a noise removal unit 38. The noise removal unit 38 is constituted by a controller.

[0065] The noise removal unit 38 removes noise, which is an edge that is shorter than a predetermined edge threshold, from among the edges detected by the edge detection unit 37 .

[0066] The line segment detection unit 32 detects line segments from the edges that have been detected by the edge detection unit 37 and not removed by the noise removal unit .

[0067] <Operation of area detection device> Here, the operation of the area detection device 3-2 according to the third embodiment will be described with reference to Fig. 10. Fig. 10 is a flowchart showing an example of the operation of the area detection device 3-2 according to the third embodiment. The operation of the area detection device 3-2 described with reference to Fig. 10 corresponds to an example of an area detection method of the area detection device 3-2 according to the third embodiment.

[0068] The area detection device 3-2 executes the processes of steps S31 and S32. The processes of steps S31 and S32 are the same as the processes of steps S21 and S22 in the second embodiment.

[0069] In step S33, the noise removal unit 38 removes noise from the edges detected by the edge detection unit 37, the edges being shorter than a predetermined edge threshold value.

[0070] In step S , the line segment detection unit 32 detects line segments from the edges that have been detected by the edge detection unit 37 and not removed by the noise removal unit .

[0071] Next, the area detection device 3-2 executes the processes from step S35 to step S38. The processes from step S35 to step S38 are the same as the processes from step S24 to step S27 in the second embodiment.

[0072] As described above, according to the third embodiment, the area detection device 3-2 further includes a noise removal unit 38 that removes noise from edges that are shorter than a predetermined edge threshold, and the line detection unit 32 detects lines from edges that have been detected by the edge detection unit 37 but not removed by the noise removal unit 38. This allows the area detection device 3-2 to suppress erroneous detection of lines, and therefore allows the area detection device 3-2 to detect the object area with high accuracy.

[0073] (Modification) In the first embodiment described above, the area detection device 3 may further include a reverse rotation correction unit. The reverse rotation correction unit rotates an image indicated by the object area information as shown in FIG. 5 by an angle (θ-90°) to generate a reverse rotation image as shown in FIG. 11. In other words, the reverse rotation correction unit rotates an image indicated by the object area information by the same angle in a direction opposite to the direction in which the correction unit 34 rotated the captured image to generate a reverse rotation image. In such a configuration, the output unit 36 ​​may output information indicating the reverse rotation image as the object area information to the data storage device 4, a display device, etc. Also, the area detection device 3-1 of the second embodiment may further include a reverse rotation correction unit, and the area detection device 3-2 of the third embodiment may further include a reverse rotation correction unit.

[0074] <Program> The above-mentioned area detection devices 3, 3-1, and 3-2 can be realized by a computer 101. A program for causing the area detection devices 3, 3-1, and 3-2 to function may be provided. The program may be stored in a storage medium or provided through a network. FIG. 12 is a block diagram showing a schematic configuration of a computer 101 functioning as the area detection devices 3, 3-1, and 3-2, respectively. Here, the computer 101 may be a general-purpose computer, a dedicated computer, a workstation, a PC (Personal Computer), an electronic notepad, or the like. The program instructions may be program code, code segments, or the like for performing necessary tasks.

[0075] 12, the computer 101 includes a processor 110, a read only memory (ROM) 120, a random access memory (RAM) 130, a storage 140, an input unit 150, an output unit 160, and a communication interface (I / F) 170. Each component is connected to each other so as to be able to communicate with each other via a bus 180. The processor 110 is specifically a central processing unit (CPU), a micro processing unit (MPU), a graphics processing unit (GPU), a digital signal processor (DSP), a system on a chip (SoC), or the like, and may be composed of multiple processors of the same type or different types.

[0076] The processor 110 controls each component and executes various arithmetic processing. That is, the processor 110 reads a program from the ROM 120 or the storage 140, and executes the program using the RAM 130 as a working area. The processor 110 controls each component and executes various arithmetic processing according to the program stored in the ROM 120 or the storage 140. In the above-described embodiment, the program according to the present disclosure is stored in the ROM 120 or the storage 140.

[0077] The program may be stored in a storage medium readable by the computer 101. By using such a storage medium, the program can be installed in the computer 101. Here, the storage medium in which the program is stored may be a non-transitory storage medium. The non-transitory storage medium is not particularly limited, and may be, for example, a CD-ROM, a DVD-ROM, or a USB (Universal Serial Bus) memory. In addition, the program may be in a form that is downloaded from an external device via a network.

[0078] The ROM 120 stores various programs and various data. The RAM 130 temporarily stores programs or data as a working area. The storage 140 is configured with a hard disk drive (HDD) or a solid state drive (SSD), and stores various programs including an operating system and various data.

[0079] The input unit 150 includes one or more input interfaces that receive an input operation from a user and acquire information based on the user's operation. For example, the input unit 150 is a pointing device, a keyboard, a mouse, etc., but is not limited to these.

[0080] The output unit 160 includes one or more output interfaces that output information. For example, the output unit 160 is a display that outputs information as a video or a speaker that outputs information as a sound, but is not limited to these. Note that, if the output unit 160 is a touch panel type display, it also functions as the input unit 150.

[0081] The communication interface (I / F) 170 is an interface for communicating with an external device.

[0082] The following supplementary notes are further disclosed regarding the above embodiment.

[0083] (Additional note 1) Memory, at least one controller coupled to the memory; The controller: Detecting line segments from a captured image including an image of an object having a linear band shape; Extracting a main line segment, which is the line segment having a length equal to or greater than a predetermined value; generating, as a corrected image, a rotated image obtained by rotating the captured image so that the main line segment is perpendicular or parallel to an array direction of pixels in the captured image; detecting an object region showing an image of the object from the corrected image; Area detection device. (Additional note 2) The area detection device according to claim 1, wherein the controller generates, as a corrected image, an image obtained by reducing the rotated image so that the entire rotated image is included in the entire area of ​​the captured image. (Additional note 3) The controller: Detecting edges from the captured image; 3. The area detection device according to claim 1, wherein the line segment detection unit detects the line segments from the edges. (Additional note 4) The controller: Among the edges, noise that is an edge shorter than a predetermined edge threshold is removed; The area detection device according to claim 3, further comprising: detecting the line segment from the edge that was detected from the corrected image and that was not removed. (Additional note 5) 5. The area detection device according to claim 1, wherein the controller extracts a main line segment, which is the line segment having the longest length. (Additional note 6) Detecting line segments from a captured image including an image of an object having a linear band shape; extracting a main line segment, which is the line segment having a length equal to or greater than a predetermined value; generating, as a corrected image, a rotated image obtained by rotating the captured image so that the main line segment is perpendicular or parallel to an array direction of pixels in the captured image; detecting an object region from the corrected image that represents an image of the object; The region detection method includes: (Additional note 7) A non-transitory storage medium storing a program executable by a computer, the non-transitory storage medium storing the program causing the computer to function as the area detection device described in any one of appendix 1 to 5.

[0084] All publications, patent applications, and standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, and standard was specifically and individually indicated to be incorporated by reference.

[0085] Although the above-mentioned embodiment has been described as a representative example, it is obvious to those skilled in the art that many modifications and substitutions can be made within the spirit and scope of the present disclosure. Therefore, the present invention should not be interpreted as being limited by the above-mentioned embodiment, and various modifications or changes can be made without departing from the scope of the claims. For example, it is possible to combine multiple building blocks shown in the configuration diagram of the embodiment into one, or to divide one building block. [Explanation of symbols]

[0086] 1. Image acquisition device 2. Image storage device 3, 3-1, 3-2 Area detection device 4 Data storage device 31 Input section 32 Line detector 33 Main line segment extraction part 34 Correction section 35 Area detection section 36 Output section 37 Edge detection section 38 Noise Removal Section 100, 100-1, 100-2 Area Detection System 101 Computer 110 Processor 120 ROM 130 RAM 140 Storage 150 Input section 160 Output section 170 Communication Interface 180 Bus

Claims

1. a line segment detection unit that detects line segments from a captured image including an image of an object having a linear band shape; a main line segment extraction unit that extracts a main line segment, which is the line segment having a length equal to or greater than a predetermined value; a correction unit that generates, as a corrected image, a rotated image by rotating the captured image so that the main line segment is perpendicular or parallel to an array direction of pixels in the captured image; a region detection unit that detects an object region showing an image of the object from the corrected image by a bounding box or a deep learning method including class-based segmentation by instance segmentation; An area detection device comprising:

2. The area detection device according to claim 1 , wherein the correction section generates, as the corrected image, an image obtained by reducing the rotated image so that the entire rotated image is included in an entire area of ​​the captured image.

3. An edge detection unit that detects edges from the captured image, The area detection device according to claim 1 , wherein the line segment detection unit detects the line segments from the edges.

4. a noise removal unit that removes noise from the edges that are shorter than a predetermined edge threshold value, The area detection device according to claim 3 , wherein the line segment detection section detects the line segment from an edge that has been detected by the edge detection section and not removed by the noise removal section.

5. The area detection device according to claim 1 , wherein the main line segment extraction unit extracts a main line segment that is the line segment having the longest length.

6. Detecting line segments from a captured image including an image of an object having a linear band shape; extracting a main line segment, which is the line segment having a length equal to or greater than a predetermined value; generating, as a corrected image, a rotated image obtained by rotating the captured image so that the main line segment is perpendicular or parallel to an array direction of pixels in the captured image; Detecting an object region showing an image of the object from the corrected image by a deep learning technique including a bounding box or class-specific segmentation by instance segmentation; The region detection method includes:

7. A program for causing a computer to function as the area detection device according to any one of claims 1 to 5.

Images