Plate contour estimation device

Abstract

To provide an estimation device capable of accurately estimating the contours of a plurality of rectangular plates fixed by a plurality of fixtures.SOLUTION: An estimation device 10 comprises: an extracted image generation unit 12 which generates an extracted image GA where an image ga including a contour of a plaster board has been extracted for a captured image GT by a learning model; a binarization processing image generation unit 13 which generates a binarization processing image GB; a masking processing image generation unit 14 which contrasts the extracted image GA with the binarization processing image GB, leaves an overlapping image gc of an overlapping portion located in a position where it overlaps an image gb of an identification portion identified by adaptive binarization processing among images including the contour detected in the extracted image GA, and generates a masking processing image GC having being subjected to mask processing such as to delete other images from the extracted image GA; and a counter estimation unit 15 which estimates the contour of the plate on the basis of the overlapping image gc left in the masking processing image GC.SELECTED DRAWING: Figure 3

Description

[Technical field]

[0001] The present invention relates to a plate contour estimation device that estimates the contour of a rectangular plate from a captured image that includes a plurality of fasteners driven into the plate. [Background technology]

[0002] Conventionally, as shown in Patent Document 1, board materials such as gypsum boards used for interior wall or ceiling materials are attached to a base via a plurality of fasteners such as screws. The fasteners are driven into the board within a prescribed interval along at least the periphery of the board. After the board is installed, the installation state of the board is inspected to see whether the intervals between fasteners in the same fastener row are within the prescribed interval.

[0003] Incidentally, such an inspection can be performed visually, but sometimes multiple plate materials attached with multiple fasteners are photographed and the mounting state of the plate materials is checked from the captured image. In this case, it is important to identify the contour line (i.e., the peripheral portion) of each plate material. However, since multiple fasteners are present in the vicinity along this contour line, it is expected that the contour line cannot be accurately identified from the images of these fasteners when attempting to identify the contour line of the plate material by image processing. In view of this, for example, Patent Document 2 proposes an estimation device that estimates the contour line of a plate material by performing edge detection processing from a captured image including the plate material. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2018-031241 A [Patent Document 2] JP 2022-157018 A Summary of the Invention [Problem to be solved by the invention]

[0005] However, even if edge detection processing is performed on a captured image that includes a plate material, it is difficult to accurately detect the contour line of the plate material.

[0006] The present invention has been made in consideration of the above points, and an object of the present invention is to provide an estimation device that can accurately estimate the contour line of a rectangular plate material that is attached in multiple pieces by multiple fixing devices. [Means for solving the problem]

[0007] In view of the above problem, the device for estimating the contour line of a rectangular plate material according to the present invention is a device for estimating the contour line of the plate material from an image including a plurality of fasteners driven into the plate material, and is characterized in that it comprises: an extracted image generation unit that generates an extracted image by extracting an image including the contour line of the plate material from the image using a learning model that has been machine-learned using the contour line of the plate material as a feature; a binary processed image generation unit that generates a binary processed image by performing adaptive binary processing on the image; a masking processed image generation unit that compares the extracted image with the binary processed image, and generates a masking processed image from the extracted image that has been subjected to masking processing such that, among images including the contour line detected in the extracted image, an overlapping image of an overlapping portion that is located at a position that overlaps with an image of an identification portion identified by adaptive binary processing is left, and other images are deleted; and a contour line estimation unit that estimates the contour line of the plate material based on the overlapping image left in the masking processed image.

[0008] According to the present invention, the extracted image generating unit utilizes a learning model that has been machine-learned for an image captured by an imaging device to generate an extracted image that extracts an image that includes the contour line of the plate material. The extracted image includes not only an image of the contour line of the plate material, but also an image of a linear pattern in the background other than the plate material, an image of lines due to shading at the edge of the plate material, and the like that includes the contour line of the plate material.

[0009] The binarization image generating unit generates a binarization image by performing adaptive binarization on the captured image. The adaptive binarization does not fix a threshold value for the density of the image, but changes the threshold value for each pixel depending on the surrounding images, so that an image of an identified part that is identified depending on the difference in density in the captured image can be specified. Therefore, the image of the identified part obtained in this way includes not only an image of the outline of the plate material, but also images of the fixtures and the scribe lines of the plate material.

[0010] Here, the inventors considered that, when comparing the extracted image with the binarized image, both would contain images of the contour lines of the plate material. Thus, by comparing the image including the contour lines detected in the extracted image with the image of the identification part identified by the adaptive binarization process, the inventors obtained a new finding that an image existing at a common position in the entire image area between the image including the contour lines and the image of the identification part is highly likely to be an image of the contour lines of the plate material.

[0011] Based on this knowledge, the masking processed image generation unit compares the extracted image with the binarized processed image, and generates a masking processed image from the extracted image by retaining the overlapping image of the overlapping portion that overlaps with the image of the identified portion identified by the adaptive binarization process, among the images including the contour lines detected in the extracted image, and masking the remaining images.

[0012] As a result, the masked image is an image in which images other than the contour lines (specifically, images of linear patterns in the background other than the board material, images of short scribe lines, etc.) have been deleted from the image including the contour lines. This masked image does not reflect images of fixtures, images of short scribe lines on the board material, images of elongated stains, and images of small dot-like patterns, among the images of the identified parts identified by the adaptive binarization process. In this way, the overlapping images of multiple overlapping parts included in the masked image are highly likely to be images of contour lines.

[0013] Therefore, the contour estimating unit identifies an image of the contour of the plate based on the multiple overlapping parts remaining in the masked image, thereby making it possible to accurately estimate the image of the contour of the plate using the captured image.

[0014] In a more preferred embodiment, the binarized image generating unit generates the binarized image from an edge-detection-processed image obtained by performing edge detection processing on the captured image. In this way, when a binarized image is generated from an edge-detection-processed image, an image of the contour line of the plate can be accurately detected in the binarized image. In particular, the contour lines corresponding to the upper and lower edges of the plate arranged on the wall along the horizontal direction are difficult to detect as an image due to the influence of lighting, etc., but by performing edge detection processing at least along the horizontal direction, the detection accuracy of the image of the contour line along the horizontal direction can be improved.

[0015] In a more preferred embodiment, the contour line estimation unit divides the masked image into regions each including each contour line of the plate material, and estimates the contour line of the plate material based on the overlapping images included in the divided images. According to this embodiment, the contour line of the plate material is estimated based on the overlapping images included in the divided images, so that the contour line of the plate material can be estimated simply and accurately.

[0016] In a more preferred embodiment, the partitioned image contains multiple overlapping images, and the contour estimation unit selects two pixels from among the pixels constituting the multiple overlapping images, calculates a straight line connecting the two selected pixels, and counts the number of pixels of the overlapping images that are within a certain distance from the calculated straight line for all of the overlapping images, repeatedly performing this process for all combinations of pixels constituting the multiple overlapping images, extracts the pixel within the certain distance that gives the maximum count, and estimates the contour of the board based on the extracted pixel.

[0017] According to this aspect, by extracting pixels within a certain distance where the count number reaches a maximum value, it becomes easier to eliminate overlapping pixels that are off the contour of the board, and as a result, the contour of the board can be more accurately estimated based on the extracted pixels.

[0018] In a further preferred embodiment, the partitioned image contains a plurality of overlapping images, and the contour estimation unit calculates an approximate straight line from the coordinates of all pixels contained in two of the overlapping images selected from the plurality of overlapping images contained in the partitioned image, and counts the number of pixels in the overlapping images that are within a certain distance from the calculated approximate straight line for all of the overlapping images, repeatedly performing this process for a specific number or all combinations of the two overlapping images, extracting pixels within the certain distance that produce a maximum count number, and estimating the contour of the board based on the extracted pixels.

[0019] According to this aspect, an approximate straight line is calculated from the coordinates of all pixels included in the two selected overlapping images, and the number of pixels of the overlapping images that are within a certain distance from the calculated approximate straight line is counted for all overlapping images. By extracting pixels within a certain distance that maximizes the count number, it becomes easier to exclude overlapping pixels that are outside the contour line of the board. As a result, the contour line of the board can be estimated more accurately based on the extracted pixels. The calculation load can be reduced compared to the case where a straight line is calculated and pixels are counted for all combinations of two overlapping pixels from the pixels of multiple overlapping images.

[0020] In a further preferred embodiment, the partitioned image contains a plurality of overlapping images, and the contour estimation unit extracts a maximum overlapping image with the largest number of pixels from the plurality of overlapping images contained in the partitioned image, selects two pixels from the pixels contained in the maximum overlapping image, calculates a straight line connecting the two selected pixels, and counts the number of pixels of the overlapping image that are within a certain distance from the calculated straight line for all pixels of the overlapping images, repeatedly performing this process for a specific number or all combinations of pixels constituting the maximum overlapping image, extracts pixels within the certain distance that give a maximum count number, and estimates the contour of the board based on the extracted pixels.

[0021] According to this aspect, the maximum overlapping image having the maximum number of pixels from among the multiple overlapping images included in the partitioned image is highly likely to be an image that forms a straight line that estimates the contour of the board. Therefore, two pixels are selected from the pixels included in the maximum overlapping image, a straight line connecting the two selected pixels is calculated, and the number of pixels of the overlapping image that are within a certain distance from the calculated straight line is counted for all pixels of the overlapping image. By extracting pixels within a certain distance that have a maximum count number, it becomes easier to exclude pixels of the overlapping pixels that are out of the contour of the board. As a result, the contour of the board can be estimated more accurately based on the extracted pixels. Compared to the case where a straight line is calculated and pixels are counted for all combinations of two pixels from the pixels of the multiple overlapping images, the calculation load can be reduced.

[0022] In a more preferred aspect, the contour line estimation unit identifies a position of an image of each of the fixing devices from the captured image, and if the number of images of the fixing devices that exist within a certain distance from the estimated contour line is equal to or greater than a predetermined number, it determines that the estimated contour line is an appropriate contour line.

[0023] According to this aspect, since a plurality of fasteners are driven into the board within a certain distance range from the outline of the board to the inside of the board, if the number of images of the fasteners is equal to or greater than a predetermined number, it is possible to accurately determine that the estimated outline is appropriate. Effect of the Invention

[0024] According to the present invention, the contour lines of a plurality of rectangular plate materials attached by a plurality of fasteners can be estimated with high accuracy. [Brief description of the drawings]

[0025] [Figure 1] FIG. 2 is a schematic diagram of a gypsum board into which a screw that is the subject of estimation in this embodiment is driven. [Diagram 2] 1 is a schematic diagram of a contour line estimation device according to an embodiment of the present invention. [Diagram 3] FIG. 2 is a block diagram of the estimation device shown in FIG. [Figure 4] FIG. 2A is a conceptual diagram showing the entire captured image, and FIG. 2B is a conceptual diagram showing the entire extracted image. [Diagram 5] FIG. 2 is a conceptual diagram showing an entire binarized image. [Figure 6] FIG. 13 is a conceptual diagram of an edge-detected image obtained by performing edge detection processing before adaptive binarization processing. [Figure 7] 1A and 1B are conceptual diagrams for explaining masking processing, in which (a) is a partial image of an extracted image, (b) is a partial image of a binarized image, and (c) is a partial image of a masking processed image. [Figure 8] FIG. 2 is a conceptual diagram showing an entire masking processed image. [Figure 9] 13(a) and (b) are schematic diagrams for explaining an estimation method performed by a contour line estimation unit. [Figure 10] 10 is a schematic diagram of an image for explaining a contour line estimation section according to the modified example of FIG. 9. FIG. [Figure 11] 10 is a schematic diagram of an image for explaining a contour line estimation section according to another modified example of FIG. 9. [Figure 12] FIG. 1A is a conceptual diagram of an image in which a contour line has been estimated, and FIG. 1B is a conceptual diagram for explaining a process for determining whether or not the estimated contour line is appropriate. [Figure 13] FIG. 4 is a calculation flow diagram using the estimation device shown in FIG. [Figure 14] 11 is a flow diagram for explaining an estimation method by a contour line estimation section according to a modified example of FIG. 10. FIG. [Figure 15] 12 is a flowchart for explaining an estimation method performed by a contour line estimation section according to another modified example of FIG. 11. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0026] The plate material contour line estimation device according to this embodiment will be described below with reference to FIGS.

[0027] 1. About the boards and fixtures In this embodiment, a board outline estimation device 10 (hereinafter referred to as "estimation device 10") is a device that identifies a plurality of rectangular boards into which fasteners have been driven. Here, fasteners are driven at predetermined intervals at least along the periphery of the boards. Examples of combinations of boards and fasteners include eaves soffit material (decorative material under the eaves) and nails that secure it, structural plywood and nails that secure it in wooden houses, and floor underlayment and nails / screws that secure it.

[0028] In the following embodiment, as shown in Fig. 1, a gypsum board 5A is exemplified as a plate material, and a screw 6A is exemplified as a fastener. Therefore, the gypsum board 5A corresponds to the "plate material" in the present invention, and the screw 6A corresponds to the "fastener" in the present invention. The contour lines 7H and 7V of the gypsum board 5A correspond to the "contour lines of the plate material."

[0029] The fasteners for fixing the gypsum board 5A may be nails or staples. The gypsum board 5A is used as an interior wall material or ceiling material. The gypsum board 5A is fixed with screws 6A along the long and short sides of the gypsum board 5A at a pitch equal to or less than that specified by laws and regulations. The screw pitch here refers to the distance between adjacent screws 6A, 6A along the long and short sides of the gypsum board 5A.

[0030] Here, the multiple screws 6A, 6A, ... are driven into the gypsum board 5A at a specified pitch along the periphery of the gypsum board 5A, inside the periphery of the gypsum board 5A. Furthermore, in this embodiment, the multiple screws 6A, 6A, ... are driven into the gypsum board 5A at a specified pitch P along the long side direction of the gypsum board 5A, at the center in the width direction of the gypsum board 5A. After the gypsum board 5A is installed, the screw pitch is checked to see if it is within the specified pitch P. The screw pitch is the center-to-center distance between adjacent screws 6A along the long side direction and the short side direction when the gypsum board 5A is viewed from the front.

[0031] The contour line estimation device 10 according to this embodiment is a device that estimates contour lines 7H, 7V of a plaster board 5A into which screws 6A have been driven after the plaster board 5A has been installed. The estimation device 10 may perform projective transformation of an image 5 of the plaster board 5A together with an image of the screws 6A from the contour lines 7H, 7V in an image GT including the plaster board 5A captured by an imaging device 20 into a front-facing image viewed from the front, extract an image 6 of the projected screws 6A, and estimate the pitch of adjacent screws 6A.

[0032] 2. Hardware configuration of the estimation device 10 As shown in FIG. 2, the estimation device 10 is configured as hardware including a ROM, a RAM, etc., and is equipped with a memory device 10A in which the conditions of the gypsum board 5A, a screw pitch estimation program, etc. are recorded, and a calculation device 10B that executes a contour line estimation program.

[0033] An input device 31 and an output device 32 are connected to the estimation device 10. In this embodiment, the input device 31 and the output device 32 may be integrated into a touch panel display. The input device 31 receives data such as the specification conditions of the gypsum board 5A and a program for estimating the screw pitch. In this embodiment, the input device 31 receives image data captured by the imaging device 20. The data input by the input device 31 is stored in the storage device 10A. The output device 32 displays image data captured by the imaging device 20, calculation results calculated by the calculation device 10B, and the like.

[0034] In the present embodiment, the estimation device 10 is configured with the storage device 10A and the calculation device 10B, but may also include, for example, an input device 31 and an output device 32. The estimation device 10 may further include an imaging device 20 in addition to the input device 31 and the output device 32, and may be a mobile terminal such as a smartphone or tablet in which these are integrated.

[0035] 3. Software configuration of the estimation device 10 3, the estimation device 10 at least includes a brightness uniformization processing unit 11, an extraction image generation unit 12, a binarization processed image generation unit 13, a masking processed image generation unit 14, and a contour line estimation unit 15. In addition to these, the estimation device 10 may further include a spacing estimation unit that identifies an image 6 of a screw (fixture) 6A and estimates a pitch P (spacing of the fixture) of the identified screw.

[0036] 3-1. Regarding the brightness equalization processing unit 11 The brightness uniformization processing unit 11 uniforms brightness differences for each pixel constituting an image captured by the imaging device 20. For example, the brightness uniformization processing unit 11 uniforms brightness of each pixel so that the brightness of each pixel falls within a predetermined range. Specifically, the brightness uniformization processing unit 11 may multiply the brightness of each pixel by a correction coefficient corresponding to the difference between the maximum and minimum brightness values ​​of the pixels of the imaging device 20.

[0037] By using the captured image whose brightness has been uniformized by the brightness uniformization processing unit 11, the pixels of the image of the screw, which change depending on the capturing conditions, can be uniformized. Furthermore, in the binarized image GB described later, the images 7h and 7v of the contour lines 7H and 7V can be easily identified, and the accuracy of the estimation of the images 7h and 7v of the contour lines 7H and 7V can be improved. Note that if the images 7h and 7v of the contour lines 7H and 7V can be identified by the following series of steps, the brightness uniformization processing unit 11 may be omitted, and further, noise may be removed from the captured image using a median filter or the like before the brightness uniformization processing.

[0038] 3-2. Extraction image generating unit 12 The extracted image generating unit 12 generates an extracted image GA (see FIG. 4(b)) including the contour lines 7H, 7V of the gypsum board 5A from the captured image GT shown in FIG. 4(a) by using a learning model machine-learned using the contour lines 7H, 7V (images 7h, 7v) of the gypsum board 5A as features. Here, it is preferable to generate the extracted image GA from the captured image GT before the brightness is uniformized by the brightness uniformization processing unit 11.

[0039] The learning model of the extracted image generating unit 12 uses a support vector machine (SVM) or the like to learn images 7h and 7v of the contour lines 7H and 7V from the captured images GT using a plurality of captured images GT captured by the imaging device 20 and feature values ​​of the shape of the gypsum board 5A in these images (for example, images 7h and 7v of the contour lines 7H and 7V, which are the outer edges of the gypsum board 5A) as training data. This makes it possible to extract images 7h and 7v of the contour lines 7H and 7V of the gypsum board 5A from any overall image including the image 5 of the gypsum board 5A. The shape of the gypsum board 5A itself may be identified using a cascade classifier using feature values ​​such as Haar-like features. Furthermore, in a separate process, the image 6 of the screw 6A may be extracted using a learning model that has machine-learned the shape of the screw 6A.

[0040] Furthermore, a CNN (Convolutional Neural Network) may be used to create a learning model for the extracted image generating unit 12. Specifically, preprocessing of data normalization and resizing to an appropriate size are performed from the captured image GT as necessary. Next, annotation data obtained by labeling the image of the background, the image 5 of the plaster board 5A, the image 7v of the contour line 7V along the vertical direction of the plaster board 5A, and the image 7h of the contour line 7H along the horizontal direction of the plaster board 5A for each pixel of the captured image GT are used as training data, and a learning model is created by machine learning using a CNN. From this learning model, an extracted image GA including the contour lines 7H and 7V of the plaster board 5A is generated for the captured image GT.

[0041] In this way, by utilizing a learning model that has been machine-trained using the contour lines 7H and 7V of the gypsum board 5A as features, it is possible to more accurately and quickly identify images ga that include the contour lines 7H and 7V of the gypsum board 5A in the extracted image GA.

[0042] 3-3. About the binary processing image generating unit 13 The binarization processed image generating unit 13 generates a binarized image GB by performing adaptive binarization on the captured image GT. In this embodiment, as shown in Fig. 5, the binarization processed image generating unit 13 generates a binarized image GB from a captured image whose brightness has been uniformized. Since the adaptive binarization does not fix a threshold value for the density of an image, but changes the threshold value for each pixel depending on the surrounding images, it is possible to specify an image gb of an identified part identified depending on the difference in shading in the captured image.

[0043] In Fig. 5, the white parts are images of parts that were not identified, and the black parts are images of parts that were identified by adaptive binarization processing. Adaptive binarization processing is a method of determining a binarization threshold value for each pixel, and can optimally binarize even images with partially varying brightness. Therefore, the image gb of this identified part includes not only images 7h and 7v of the contour lines 7H and 7V of the gypsum board 5A, but also image 6 of the screws and image 7c of the crease lines of the gypsum board 5A.

[0044] As shown in FIG. 1, the gypsum board 5A may be arranged as a wall material (board material) so that the longitudinal direction and the vertical direction coincide with each other. Assuming such an arrangement, if the edge of the gypsum board 5A along the longitudinal direction (vertical direction) is chamfered, a shadow is likely to be generated along the vertical contour line 7V of the gypsum board 5A. Therefore, the image 7v of the contour line 7V is likely to be detected by adaptive binarization processing. On the other hand, the horizontal contour line 7H of the gypsum board 5A is a joint portion with another member, so a shadow is generated, and therefore the image 7h of the contour line 7H may be detected. However, a shadow may not be easily generated on the vertical contour line 7V of the gypsum board 5A, and the image 7h of the contour line 7H may be difficult to detect. In such a case, the edge detection processing shown below may be performed.

[0045] Specifically, the binarized image generating unit 13 generates an edge-detected image GB' (see FIG. 6(a)) by performing edge detection processing on the captured image GT processed by the brightness uniformization processing unit 11. A binarized image GB is generated from this edge-detected image GB'. In the edge detection processing, the Canny method or the like may be used to detect edges in the entire image G1, or the edges of the captured image GT may be detected by sequentially convolving various kernels with image matrices such as Frei-Chen, Prewitt, Roberts, and Sobel. These edge detection processes are generally known and therefore will not be described in detail.

[0046] The edge detection process here is performed by differential processing. Specifically, the captured image GT processed by the brightness uniformization processing unit 11 is converted to grayscale, and differential processing is performed on the captured image GT converted to grayscale so that edges along the horizontal direction are emphasized. This makes it difficult to detect the image gb' (7v, 7c) including the contour line 7V along the vertical direction, and makes it possible to preferably detect an image including the contour line 7H along the horizontal direction.

[0047] In this embodiment, when edge detection processing is performed, an image 7h of a contour 7H along the horizontal direction is estimated using a binarized image GB obtained by performing adaptive binarization processing on the edge-detection-processed image GB'. On the other hand, an image 7v of a contour 7V along the vertical direction is estimated using a binarized image GB that has not been subjected to edge detection processing. In this case, a process may be performed to emphasize edges along the vertical direction as well as the horizontal direction before generating the binarized image GB.

[0048] Specifically, in this edge detection process, as shown in Fig. 6(b), a filter matrix (Sobel filter) is applied to the digitized color values ​​for each pixel according to the color change from black to white. At this time, the digitized color values ​​are converted for each pixel according to the color change from black to white. Specifically, negative values ​​in the filter matrix are set to 0, and all values ​​over 255 are set to 255, so that black is 0 and white is 255.

[0049] The contour line 7H corresponding to the upper and lower edges of the gypsum board 5A along the horizontal direction is often located at the boundary with other components and is difficult to detect as an image due to the influence of lighting, etc., but this type of edge detection process emphasizes the image 7h of the contour line 7H along the horizontal direction, thereby improving the detection accuracy of this image 7h.

[0050] Incidentally, in the extraction image generating unit 12, images other than the contours 7H and 7V are also extracted, but the number of images ga (7v, 7c, etc.) including the contours extracted in the extraction image GA is smaller than the number of images gb of the identification portion included in the binarized image GB. On the other hand, in the extraction image GA, the areas of the images 7h and 7v that are determined to be the contours 7H and 7V are extracted as the contours including the surrounding areas that are slightly wider than the actual contours (see, for example, FIG. 7(a)). Also, in the binarized image GB, the contours can be identified without including the surrounding areas, but the parts other than the contours are also identified. For these reasons, the masking processing image generating unit 14 performs the process described below.

[0051] 3-4. Masking Processing Image Generation Unit 14 The masking processed image generating unit 14 compares the extracted image GA (see FIG. 7(a)) with the binarized image GB (see FIG. 7(b)). Specifically, it performs a masking process such that, of the image ga (7v, 7c, etc.) including the contour extracted in the extracted image GA, an overlapping image gc (7v) of the overlapping portion located at a position overlapping with the image gb (7v) of the identified portion identified by the adaptive binarization process is left, and the other images are deleted. Such an overlapping image gc can be obtained by logically ORing the pixels of the image ga and the pixels of the image gb.

[0052] In Fig. 7(c), an image 7v including a contour line 7V of the gypsum board 5A is included as an overlapping image gc of an overlapping portion along the vertical direction. Although not shown, an image 7h including a contour line 7H of the gypsum board 5A is included as an overlapping image gc of an overlapping portion along the horizontal direction. In this manner, the masking processed image generating unit 14 generates a masking processed image GC (see Fig. 8).

[0053] In this way, when the extracted image GA and the binarized image GB are compared, both of them contain images 7h and 7v including the contour lines 7H and 7V. When the image ga (7c, 7h, 7v, etc.) including the contour lines extracted in the extracted image GA is compared with the image gb (7c, 6, 7h, 7v) of the identification part identified by the adaptive binarization process, the image existing at the common position of the image ga including the contour lines extracted in the extracted image GA and the image gb of the identification part in the entire image area is highly likely to be the image 7h or 7v including the contour lines 7H and 7V. Therefore, specifically, the masking process image generating unit 14 performs a process in which the extracted image GA is used as the original (base) image, the image of the adaptive binarization process is used as the mask image, and the pixel in the mask image that is white (255) is left as it is at the same position in the original image, and the pixel in the mask image that is black (0) is changed to black (0) at the same position in the original image.

[0054] 7(c) and 8, the masked image GC is an image in which images other than images 7h and 7v including contour lines 7H and 7V are deleted from image ga including contour lines extracted in extraction image GA. This masked image GC does not generally reflect the image 6 of the screw 6A, the image of the short scribed line of the plaster board 5A, the image of the elongated stain, and the image of the background other than the plaster board 5A, among the images gb of the identification part identified by the adaptive binarization process. However, assuming that the image 6 of the screw 6A is reflected, for example, a process may be performed to detect the image 6 of the screw 6A as described below and remove the image 6 of the screw 6A from the image gb of the identification part.

[0055] 3-5. Contour Estimation Unit 15 The contour line estimation unit 15 estimates images (7h, 7v) of contour lines 7H, 7V of the gypsum board 5A based on overlap images gc of a plurality of overlapping parts remaining in the masking processed image GC, as shown in FIG. 12(a). Here, when identifying the images 7h, 7v of the contour lines 7H, 7V, a learning model machine-learned using the contour lines 7H, 7V of the gypsum board 5A as feature values ​​may be used for the masking processed image GC. Also, an approximate straight line may be calculated from the coordinates of the pixels constituting the overlap image gc for the entire masking processed image GC by the least squares method, the least median method, or M estimation, and the approximate straight line (image) L may be identified as the contour lines 7H, 7V (images 7h, 7v).

[0056] However, in this embodiment, as shown in FIG. 8, the contour line estimation unit 15 estimates the images 7h and 7v of the contour lines 7H and 7V by the following method. The contour line estimation unit 15 divides the masking processed image GC into regions including the images 7h and 7v of the contour lines 7H and 7V of the gypsum board 5A. For example, when identifying the image 7h of the horizontal contour line 7H, the distance C between the groups of overlapping images gc above and below along the vertical direction of the masking processed image GC is calculated, and the line passing through the center of that distance is set as the division line D to divide the masking processed image GC. The divided division image is an image that mainly includes the image of the horizontal contour line 7H of the gypsum board 5A.

[0057] On the other hand, when identifying the image 7v of the vertical contour line 7V, the distances A1 to A6 between the group of overlapping images gc along the vertical direction of the masking processed image GC and the group of adjacent overlapping images gc are calculated, and the masking processed image GC is divided along the lines passing through the center of each distance as division lines B1 to B6. The divided division images are images that mainly include images of each vertical contour line 7V of the gypsum board 5A.

[0058] In this embodiment, the contour line estimating unit 15 estimates images 7h, 7v of contour lines 7H, 7V along the horizontal and vertical directions of the gypsum board 5A based on overlapping images gc included in the divided section images.

[0059] Here, as shown in FIG. 9(b), when there are multiple overlapping images gc included in the divided image, as shown in FIG. 9(a), the contour line estimation unit 15 may estimate the contour line by linear approximation using a RANSAC (Random Sample Consensus) algorithm. Specifically, the linear approximation is performed by repeating the following methods (1) and (2). Hereinafter, with reference to FIG. 9(b), linear approximation of the image 7h of the contour line 7H along the horizontal direction for divided images obtained by dividing the masking processed image GC for each region including the image 7h of the contour line 7H of the gypsum board 5A will be described. However, linear approximation of the image 7v of the contour line 7V along the vertical direction can be performed in the same manner, so detailed description thereof will be omitted.

[0060] 9(a), the contour estimation unit 15 selects two pixels from among the pixels constituting the multiple overlapping images gc, and calculates a straight line L' connecting the two selected pixels. Specifically, for each pixel of the masking processed image GC, orthogonal coordinates are set in the lengthwise and widthwise directions of the masking processed image GC, and an equation of the straight line L' that passes through the coordinates of the two selected pixels is calculated.

[0061] (2) For all overlapping images gc, the number of pixels (inlier pixels) of the overlapping images that are within a certain distance S from the calculated straight line L' is counted. Note that the circles in FIG. 9(a) indicate the coordinates of the pixels that make up the overlapping images gc with respect to the entire masking processed image GC. The above-mentioned (1) and (2) are repeated for all combinations of pixels that make up the multiple overlapping images gc. However, these repetitions may also be performed for a preset specific number of pixel combinations. In this case, the specific number may be set, for example, by confirming from actual processing the number of times that a certain degree of accuracy in linear approximation is ensured, and the specific number may be, for example, about 50% of the number of combinations.

[0062] Next, from these results, pixels (inlier pixels) within a certain distance S where the count number is maximum are extracted, linear approximation is performed based on the coordinates of the extracted pixels, and the obtained approximate line (image) L is estimated as the contour lines 7H, 7V (images 7h, 7v) of the gypsum board 5A. Note that this approximate line L may be a line connecting the two pixels where the count number is maximum, or the approximate line L may be calculated from the coordinates of the extracted multiple pixels by the least squares method or the like. In addition, when identifying the horizontal contour line 7H, the inlier pixels to be identified may be extracted based on the inlier where the error in the horizontal length of the group of inlier pixels and the distance from the approximate line L is minimal.

[0063] In this way, by extracting the pixels of the overlapping image (inlier pixels) within a certain distance where the count number is maximum, it becomes easier to exclude the overlapping pixels (outlier pixels) that are out of the contour lines 7H, 7V of the gypsum board 5A. As a result, it is possible to obtain an estimated image GD in which the contour lines 7H, 7V of the gypsum board 5A are estimated from the approximation line L based on the extracted pixels. As shown in FIG. 9(b), the overlapping image gc (7d) located approximately parallel to the approximation line L can be excluded from the identification of the contour lines 7H, 7V of the gypsum board 5A.

[0064] A modified example of the contour line estimation unit 15 will be described. The contour line estimation unit 15 selects two overlapping images gc, gc from a plurality of overlapping images gc included in a partitioned image. For example, in the example shown in Fig. 10(a) to (c), overlapping images gc(1) to gc(3) composed of any of pixels "1" to "3" exist in the masking processed image GC. Therefore, in this case, overlapping images gc(1) and gc(2) are selected in Fig. 10(a), overlapping images gc(1) and gc(3) are selected in Fig. 10(b), and overlapping images gc(2) and gc(3) are selected in Fig. 10(c).

[0065] As shown in Fig. 10(a) to (c), for all combinations of the two selected overlapping images gc, gc, a first approximation line La is calculated from the coordinates of all pixels contained in the two selected overlapping images gc, gc. Here, for all overlapping images gc(1) to gc(3), the number of pixels "1" to "3" of the overlapping image gc that are within a certain distance S from the calculated first approximation line La is counted. This is repeated for all combinations of the two overlapping images gc in Fig. 10(a) to (c).

[0066] 10(a)-(c), the combination shown in FIG. 10(b) has the maximum count. Therefore, in the case of FIG. 10(b), pixels "1"-"3" within a certain distance S are extracted, linear approximation is performed based on the coordinates of the extracted pixels, and the obtained approximate line L (second approximate line) is estimated as the contour lines 7H and 7V of the gypsum board 5A. Note that this second approximate line L may be the approximate line La itself in FIG. 10(b), and for example, the approximate line L may be calculated by the least squares method, the least median method, M estimation, or the like from the coordinates of a plurality of pixels (specifically including pixel "2" within the certain distance S) extracted from FIG. 10(b). Although the calculation was performed for all combinations of overlapping images, the number is a preset number, and may be set by, for example, checking the number of times that the accuracy of linear approximation is ensured to a certain degree from actual processing.

[0067] In this modification, a first approximate straight line La is calculated from the coordinates of all pixels included in the two selected overlapping images gc, and the number of pixels "1" to "3" of the overlapping images gc(1) to gc(3) that are within a certain distance from the calculated first approximate straight line La is counted for all overlapping images gc(1) to gc(3). By extracting pixels within a certain distance S at which the count number is maximum, it becomes easier to exclude pixels of the overlapping images gc that are out of the contour line of the gypsum board 5A. As a result, the contour lines 7H and 7V of the gypsum board 5A can be more accurately estimated based on the extracted pixels "1" to "3". This eliminates the need to calculate a straight line by two pixels of the same overlapping image gc and count based on that calculation, and therefore reduces the computation load compared to the case shown in FIG. 9(a).

[0068] In another modified example, as shown in FIG. 11, the contour estimating unit 15 may extract the maximum overlapping image gcmax with the maximum number of pixels from the multiple overlapping images gc(1) to gc(3) included in the divided image. In FIG. 11, the maximum overlapping image gcmax is the overlapping image gc(3). Of the pixels "3" included in the extracted maximum overlapping image gcmax, two pixels "3", "3" may be selected, a straight line Lb connecting the two selected pixels "3", "3" may be calculated, and the number of pixels of the overlapping image within a certain distance S from the calculated straight line Lb may be counted for the pixels "3", "3", ... of all the overlapping images gc(3). This counting is repeated for a specific number of pixels constituting the maximum overlapping image gcmax or for all combinations, and pixels within a certain distance S with the maximum count number are extracted, and the contour of the gypsum board 5A is estimated based on the extracted pixels.

[0069] Furthermore, as shown in FIG. 12(b), the contour estimation unit 15 identifies the position of the images 6 of each screw 6A from the captured image GT, and if the number of images 6 of each screw 6A existing within a certain distance from the estimated contour line (approximate straight line L) is equal to or greater than a predetermined number, it determines that the estimated contour line (approximate straight line L) is an appropriate contour line.

[0070] When identifying the position of the image 6 of each screw 6A from the captured image GT, for example, a learning model that machine-learns the shape of the screw 6A for the captured image GT by the above-mentioned machine learning may be used to superimpose the image 6 of the screw 6A on an estimated image GD estimated from an approximate straight line (straight line) L, and the position of the image 6 of each screw 6A relative to the estimated image GD may be assigned. Also, the image 6 of each screw 6A may be detected from the captured image GT by a commonly known corner detection process. Furthermore, the image 6 of the screw 6A may be detected by performing a mask process on the corner detection process image with the binarized process image GB obtained by the above-mentioned adaptive binarization process, instead of the extracted image GA, by a process similar to that of the above-mentioned masking process image generating unit 14.

[0071] 1 and other figures, a plurality of screws 6A are driven at intervals along contour lines 7H, 7V of a plaster board 5A within a certain distance inside the plaster board 5A from the contour lines 7H, 7V. Therefore, if the number of images 6 of the screws 6A that exist within a certain distance from the estimated contour line (approximation line L) is equal to or greater than a predetermined number, it can be determined with high accuracy that the estimated contour line (approximation line L) is an appropriate contour line.

[0072] A flow chart using the estimation device 10 will be described below with reference to Figs. 13 to 15. First, as shown in Fig. 13, in step S1, the imaging device 20 captures an image of an area to be inspected, including the gypsum board 5A, to obtain an overall image. In step S2, the extraction image generating unit 12 generates an extraction image GA (see Fig. 4(b)).

[0073] Next, in step S3, the brightness uniformization processing unit 11 performs noise removal on the captured image using a median filter or the like. In step S4, the brightness uniformization processing unit 11 uniforms brightness differences for each pixel constituting the captured image.

[0074] Next, in step S5, it is determined whether the mask process for identifying the image 7h of the horizontal contour line 7H has been completed. If the mask process for identifying the image 7h of the horizontal contour line 7H has been completed (YES), the process proceeds to step S6, where an adaptive binarization process is performed on the captured image GT on which the edge detection process shown in Fig. 6 has been performed, thereby generating a binarized image GB. If the image 7h of the horizontal contour line 7H has not been identified (NO), the process proceeds to step S7, where an adaptive binarization process is performed on the captured image GT, thereby generating a binarized image GB (see Fig. 5).

[0075] In step S8, masking is performed by the masking processing image generating unit 14. In step S9, it is determined whether the masking process for identifying the images 7h, 7v of the horizontal and vertical contour lines 7H, 7V has been completed. If the masking process has been completed (YES), the process proceeds to step S10, where the contour line estimating unit 15 creates an approximate straight line L, and in step S11, the approximate straight line L is estimated to be the contour line of the gypsum board 5A. On the other hand, if the identification has not been completed (NO), the process returns to step S5. Finally, in step S11, it is determined whether the estimated contour line (approximate straight line L) is appropriate.

[0076] Here, a modified example of the creation of the approximation line in step S10 will be described with reference to the flow diagram shown in Fig. 14. The process shown in Fig. 14 is performed by the contour line estimation unit 15, and is a process for creating the approximation line L described in the above-mentioned modified example. First, in step S31, for a partitioned partition image, two overlapping images gc, gc are selected from a plurality of overlapping images gc included in the partitioned image. In step S32, the coordinates of all pixels of the selected two overlapping images gc, gc are set. In step S33, a first approximation line La is calculated from the coordinates of all the set pixels.

[0077] In step S34, the number of pixels of a plurality of overlapping images that are within a certain distance from the calculated first approximate line La is counted.

[0078] In step S35, it is determined whether the processes from step S32 to step S34 have been performed for a specific number (or all) of the combinations of overlapping images gc, gc included in the division image. If they have not been performed (if NO), the process returns to step S32, and if they have been performed (if YES), the process proceeds to step S36. In step S36, the pixel with the maximum count is extracted. In step S37, a second approximate straight line L is calculated based on the extracted pixels using the least squares method or the like. This calculated second approximate straight line L is set as the contour lines 5H, 5V of the gypsum board 5A.

[0079] Here, another modified example of the creation of the approximate straight line in step S10 will be described with reference to the flow diagram shown in FIG. 15. The process shown in FIG. 15 is a process performed by the contour line estimation unit 15, and is a process for creating the approximate straight line L described in the other modified example above. First, in step S41, for a partitioned partition image, a maximum overlapping image gcmax having the maximum number of pixels is extracted from a plurality of overlapping images gc contained in the partitioned image. In step S42, the coordinates of the pixels of the maximum overlapping image gcmax are set. A straight line Lb is calculated from the coordinates of two pixels of the maximum overlapping image gcmax. In step S44, the number of pixels of the plurality of overlapping images within a certain distance from the calculated straight line Lb is counted.

[0080] In step S45, it is determined whether the processes from step S42 to step S44 have been performed for a specific number (or all) of the pixel combinations included in the maximum overlapping image gcmax. If they have not been performed (if NO), the process returns to step S42, and if they have been performed (if YES), the process proceeds to step S46. In step S46, the pixel with the maximum count is extracted. In step S47, an approximation line L is calculated based on the extracted pixels using the least squares method or the like. This calculated approximation line L is set as the contour lines 5H, 5V of the gypsum board 5A.

[0081] Although the embodiments of the present invention have been described in detail above, the present invention is not limited to the above-described embodiments, and various design modifications can be made without departing from the spirit of the present invention as described in the claims. [Explanation of symbols]

[0082] 5A: board material (gypsum board), 6A: fastener (screw), 10: estimation device, 11: brightness uniformization processing unit, 12: extracted image generation unit, 13: binarized image generation unit, 14: masking processed image generation unit, 15: contour line estimation unit, GA: extracted image, GB: binarized image, GC: masking processed image, GT: captured image, gc: overlapping image

Claims

1. 1. A plate contour estimation device for estimating a contour of a rectangular plate from a captured image including a plurality of fasteners driven into the plate, comprising: an extraction image generating unit that generates an extraction image by extracting an image including the contour line of the plate material from the captured image using a learning model that has been machine-learned using the contour line of the plate material as a feature; a binarization processed image generating unit that generates a binarization processed image by performing adaptive binarization processing on the captured image; a masking processed image generating unit that compares the extracted image with the binarized processed image, and generates a masking processed image from the extracted image by performing a masking process such that, among images including the contour detected in the extracted image, an overlapping image of an overlapping portion that is located at a position overlapping an image of an identified portion identified by the adaptive binarization process is retained, and other images are deleted; a contour estimating unit that estimates the contour of the plate material based on an overlapping image remaining in the masked image.

2. 2. The plate material contour line estimation device according to claim 1, wherein the binarized image generation unit generates the binarized image from an edge detection processed image obtained by performing edge detection processing on the captured image.

3. The contour line estimation unit divides the masking processed image into regions each including a contour line of the plate material, 3. The device for estimating the contour of the plate according to claim 2, further comprising: a step of estimating the contour of the plate based on the overlapping images included in the divided images.

4. The divided image includes a plurality of overlapping images, the contour estimation unit selects two pixels from among the pixels constituting the multiple overlapping images, calculates a straight line connecting the two selected pixels, and counts the number of pixels of the overlapping images that are within a certain distance from the calculated straight line for all pixels of the overlapping images, the contour estimation unit repeatedly performs this process for all combinations of pixels constituting the multiple overlapping images; extracting pixels within the certain distance where the count number is maximum; 4. The device for estimating the contour of the plate according to claim 3, further comprising: a step of estimating the contour of the plate based on the extracted pixels.

5. The divided image includes a plurality of overlapping images, the contour line estimation unit calculates an approximation line from coordinates of all pixels included in two overlapping images selected from the plurality of overlapping images included in the partitioned image, and counts the number of pixels of the overlapping images that are within a certain distance from the calculated approximation line for all the overlapping images, the contour line estimation unit repeatedly performs the above-mentioned steps for a specific number or all combinations of the two overlapping images; extracting pixels within the certain distance where the count number is maximum; 4. The device for estimating the contour of the plate according to claim 3, further comprising: a step of estimating the contour of the plate based on the extracted pixels.

6. The divided image includes a plurality of overlapping images, the contour estimating unit extracts a maximum overlapping image having a maximum number of pixels from the plurality of overlapping images included in the partitioned image; selecting two pixels from among the pixels included in the maximum overlapping image, calculating a straight line connecting the two selected pixels, and counting the number of pixels of the overlapping image that are within a certain distance from the calculated straight line for all pixels of the overlapping image, for a specific number or all combinations of pixels that constitute the maximum overlapping image; extracting pixels within the certain distance where the count number is maximum; 4. The device for estimating the contour of the plate according to claim 3, further comprising: a step of estimating the contour of the plate based on the extracted pixels.

7. The contour line estimation unit identifies a position of an image of each of the fixing tools from the captured image, The plate material contour estimation device according to claim 1, characterized in that if the number of images of the fastener that exist within a certain distance range from the estimated contour is equal to or greater than a predetermined number, the estimated contour is determined to be an appropriate contour.

Images