Fixture spacing estimation device

Abstract

To provide an estimation device of a screw pitch which can correctly estimate a pitch of screws driven to a gypsum board.SOLUTION: An estimation device 10 estimates a screw pitch of a plurality of screws 6, 6... for attaching a rectangular gypsum board 5. The estimation device 10 comprises at least: a board image extraction unit 11 which extracts an image G2 of the gypsum board 5 from the whole image G1 including the gypsum board 5 imaged by an imaging device 20; a position specification unit 12 which specifies positions of the plurality of screws 6 with respect to the whole image; a projective transformation unit 13 which performs the projective transformation of the image of the gypsum board 5 into a front image G3 viewed from the front; and a pitch calculation unit 14 which calculates the screw pitch P of the adjacent screws 6, 6 on the basis of the positions of the plurality of screws 6 with respect to the front image G3 subjected to the projective transformation.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to an estimation device that estimates the distance between adjacent fasteners among a plurality of fasteners driven into a rectangular plate material. [Background technology]

[0002] Conventionally, as shown in Patent Document 1, board materials such as gypsum boards used for interior wall materials or ceiling materials are attached to a substrate via a plurality of fasteners such as screws. The plurality of fasteners are driven into the board at specified intervals along at least the periphery of the board. After the board is installed, it is inspected to see if the spacing between the fasteners is within the specified intervals. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2018-031241 Summary of the Invention [Problem to be solved by the invention]

[0004] While such inspections can be performed visually, it is also conceivable to photograph the plate into which the fasteners have been driven and measure the spacing between the fasteners from the photographed image. However, if the plate is photographed from an oblique direction (i.e., if the photograph is taken with the optical axis of the imaging device tilted relative to the normal to the surface of the plate), the shape of the rectangular plate may not be accurately captured in the image. Even if an attempt is made to estimate the spacing between the fasteners on the plate using image processing on such an image, it is difficult to accurately estimate the spacing between the fasteners.

[0005] The present invention has been made in consideration of the above points, and its object is to provide a fastener spacing estimation device that can accurately estimate the spacing between fasteners driven into a plate material. [Means for solving the problem]

[0006] In view of the above-mentioned problems, the fastener spacing estimation device of the present invention is an estimation device that estimates the spacing between adjacent fasteners among a plurality of fasteners driven into a rectangular plate material, and is characterized by comprising at least an image extraction unit that extracts an image of the plate material from an overall image including the plate material captured by an imaging device, a position identification unit that identifies the positions of the plurality of fasteners relative to the overall image from the extracted image of the plate material, a projection transformation unit that projectively transforms the extracted image of the plate material into a front image viewed from the front, and a spacing calculation unit that calculates the spacing between the adjacent fasteners based on the positions of the plurality of fasteners relative to the projectively transformed front image.

[0007] According to the present invention, the image of the plate extracted by the image extraction unit is projectively transformed into a front image viewed from the front by the projection transformation unit. As a result, even when the plate is imaged from an oblique direction, the image of the plate together with the fasteners is projectively transformed into the front image. Therefore, the positions (position coordinates) of the fasteners in the front image of the plate after projection transformation can be identified from the positions (position coordinates) of the fasteners before projection transformation. That is, according to the present invention, rather than directly identifying the positions of multiple fasteners from the projectively transformed front image, the positions of the fasteners are identified from the image of the plate before projection transformation, in which the fastener images are easily identified, and then the positions of the multiple fasteners in the projectively transformed front image can be identified based on those positions. As a result, the spacing calculation unit can accurately calculate the spacing between the fasteners regardless of the imaging conditions.

[0008] Here, the projection transformation unit may, for example, projectively transform the extracted image of the rectangular plate material into an image viewed from the front based on the pre-input lengths of the short and long sides of the rectangular plate material and the edges of the four sides extracted by the image extraction unit, and the method is not particularly limited as long as it is possible to projectively transform into a front image viewed from the front.

[0009] Here, in a more preferred embodiment, the image extraction unit detects edges consisting of four sides of the plate material, and the projection transformation unit includes a four-corner identification unit that identifies the four corners of the extracted plate material from the detected edges, a position coordinate estimation unit that estimates the three-dimensional coordinates of the identified four corners, and a front image conversion unit that estimates the lengths of the four sides of the plate material based on the three-dimensional coordinates of the four corners and converts the extracted image of the plate material into the front image so that the rectangular area surrounded by the four sides becomes the front image of the plate material.

[0010] According to this aspect, the position coordinate estimation unit estimates the three-dimensional coordinates of the four corners of the board identified by the four-corner identification unit. This makes it possible to estimate the lengths of the four sides of the board in the virtual space. The front image conversion unit estimates the lengths of the four sides of the board based on the three-dimensional coordinates of the four corners, and converts the extracted image of the board into a front image so that a rectangular area surrounded by the four sides becomes a front image of the board, thereby making it possible to obtain a more accurate front image of the board.

[0011] In a more preferred embodiment, the position coordinate estimation unit sets the optical center point O of the imaging device as the origin of a three-dimensional Cartesian coordinate system defined by the X-axis, Y-axis, and Z-axis, sets the optical axis of the imaging device as the Z-axis, sets the entire image on an image plane that is a plane perpendicular to the Z-axis, and estimates the three-dimensional coordinates of the four corners of the plate using the three-dimensional coordinates of the pixels at the four corners on the image plane and the three-dimensional coordinates of the pixel at the center of the plate on the image plane.

[0012] According to this aspect, the position coordinate estimation unit sets the optical axis of the imaging device as the Z-axis and a plane perpendicular to the Z-axis as the image plane, thereby enabling the entire image to be set on the image plane so that the entire image is reflected in accordance with the angle of view of the imaging device. As a result, the optical axis of the imaging device coincides with the center pixel of the entire image, and in the Cartesian coordinate system, the entire image captured on the image plane is positioned (on the Z-axis) so as to fit exactly within the imaging range of the imaging device. As a result, the X- and Y-coordinates of the pixels at the four corners of the entire image can be determined by the number of pixels constituting the entire image, and the distance from the optical center point O to the image plane (i.e., the Z-coordinate positions of the four corners on the image plane) can be calculated using the same number of pixels as the pixels of the entire image.

[0013] In this way, by using the XY coordinates of the pixels at the four corners on the image plane and the XY coordinates of the pixel at the center of the plate, and taking into account that the intersection of the lines connecting the diagonally opposite corners of the four three-dimensional coordinates of the plate corresponds to the three-dimensional coordinate of the pixel at the center of the plate, the three-dimensional coordinates of the four corners of the plate, including the Z coordinate, can be easily estimated.

[0014] Furthermore, in a preferred embodiment, the position coordinate estimation unit estimates the three-dimensional coordinates of the four corners by further using information on the distance measured from the imaging device to the plate. According to this embodiment, by using the information on the distance measured, it is possible to accurately calculate the three-dimensional coordinates of the four corners (each corner portion) in real space and the length and width of the plate without inputting the length and width of the plate into the estimation device in advance.

[0015] In a more preferred embodiment, the estimation device further includes a spacing determination unit that determines whether the calculated spacing between the fasteners falls within a specified range. According to this embodiment, the spacing determination unit determines whether the spacing between the fasteners calculated by the spacing calculation unit falls within a predetermined range, making it possible to determine whether the fasteners are driven into the plate material at appropriate spacing between the fasteners. [Effects of the Invention]

[0016] According to the present invention, the spacing between fasteners driven into a plate material can be accurately estimated. [Brief explanation of the drawings]

[0017] [Figure 1] FIG. 1 is a schematic diagram of a screw pitch estimation device according to an embodiment of the present invention. [Figure 2] FIG. 2 is a control block diagram of a calculation unit of the estimation device shown in FIG. [Figure 3] FIG. 3 is a control block diagram of a projective transformation unit shown in FIG. 2. [Figure 4] FIG. 1 is a schematic diagram showing an overall image including a gypsum board captured by an imaging device. [Figure 5] 3 is a schematic diagram showing an image of a gypsum board extracted by a board image extraction unit shown in FIG. 2. FIG. [Figure 6] 4(a) to 4(c) are schematic diagrams illustrating the projective transformation unit shown in FIG. 3. [Figure 7] 4 is a schematic diagram for explaining a front image of a gypsum board generated by the front image generating unit shown in FIG. 3. FIG. [Figure 8] FIG. 2 is a schematic diagram for explaining extraction of an image of a screw shown in FIG. 1 and calculation of a screw pitch. [Figure 9] FIG. 2 is an estimation flow diagram using the estimation device shown in FIG. [Figure 10] 4 is a schematic diagram for explaining a modified example of the position coordinate estimation unit shown in FIG. 3. FIG. [Figure 11] 10, (a) is a schematic diagram viewed from the Y-axis direction to explain a modified example of the position coordinate estimation unit shown in FIG. 10, and (b) is a schematic diagram viewed from the Y-axis direction to explain a further modified example of (a). DETAILED DESCRIPTION OF THE INVENTION

[0018] An estimation device 10 according to this embodiment will be described below with reference to FIGS. 1. About boards and fixtures In this embodiment, the estimation device 10 is a device that estimates the spacing between adjacent fasteners among a plurality of fasteners driven into a rectangular board material. Here, the fasteners are driven at predetermined intervals at least along the periphery of the board material. Examples of combinations of board material and fasteners include eaves soffit material (decorative material under the eaves) and nails that secure it, structural plywood in wooden houses and nails that secure it, and floor underlayment and nails / screws that secure it.

[0019] In the following embodiment, gypsum board 5 is exemplified as the plate material, and screws 6 are exemplified as the fasteners. Therefore, gypsum board 5 corresponds to the "plate material" of the present invention, screws 6 correspond to the "fasteners" of the present invention, and the pitch of screws 6 (screw pitch) described below corresponds to the "spacing of fasteners" of the present invention.

[0020] The fasteners for fixing the gypsum board 5 may be nails or staples. The gypsum board 5 is used as an interior wall material or ceiling material, etc. The gypsum board 5 is fixed with screws 6 along the long and short sides of the gypsum board 5 at a pitch equal to or less than that specified by laws and regulations. The screw pitch here refers to the distance between adjacent screws 6, 6 along the long and short sides of the gypsum board 5.

[0021] Here, the multiple screws 6, 6, ... are driven into the gypsum board 5 at a specified pitch along the peripheral edge 5a of the gypsum board 5, inside the peripheral edge 5a. Furthermore, in this embodiment, the multiple screws 6, 6, ... are driven into the gypsum board 5 at a specified pitch in the center of the width direction of the gypsum board 5 along the long side direction of the gypsum board 5. After the gypsum board 5 is installed, the screw pitch is checked to see if it is within the specified pitch. The screw pitch is the center-to-center distance between adjacent screws 6 along the long side direction and the short side direction when the gypsum board 5 is viewed from the front.

[0022] The screw pitch estimation device 10 according to this embodiment is a device that estimates the pitch (screw pitch) of screws 6 driven into a gypsum board 5 after the gypsum board 5 has been installed. The estimation device 10 ultimately extracts images of the screws 6 from an image G1 including the gypsum board 5 captured by an imaging device 20, and estimates the pitch of adjacent screws 6.

[0023] 2. Hardware configuration of the estimation device 10 The estimation device 10 is composed of hardware such as ROM, RAM, etc., and is equipped with a memory device 10A in which the conditions of the gypsum board 5, a screw pitch estimation program, etc. are recorded, and an arithmetic device 10B that executes the screw pitch estimation program.

[0024] An input device 31 and an output device 32 are connected to the estimation device 10. In this embodiment, the input device 31 and the output device 32 may be integrated into a touch panel display. Data such as the specifications of the gypsum board 5 and a screw pitch estimation program is input to the input device 31. In this embodiment, image data captured by the imaging device 20 is input to the input device 31. The data input by the input device 31 is stored in the storage device 10A. The output device 32 displays the image data captured by the imaging device 20, the calculation results calculated by the calculation device 10B, etc.

[0025] In this embodiment, the estimation device 10 is configured with the storage device 10A and the calculation device 10B, but may also include, for example, an input device 31 and an output device 32. The estimation device 10 may further include an imaging device 20 in addition to the input device 31 and the output device 32, and may be a mobile terminal such as a smartphone or tablet that integrates these.

[0026] 3. Software configuration of the estimation device 10 In this embodiment, as shown in FIG. 2, the estimation device 10 includes at least a board image extraction unit 11, a position identification unit 12, a projective transformation unit 13, a pitch calculation unit 14, and a pitch determination unit 15.

[0027] 3-1. About the board image extraction unit 11 The board image extraction unit 11 extracts an image G2 (see FIG. 5) of the gypsum board 5 from an image G1 (see FIG. 4) including the gypsum board 5 for identifying the screw pitch captured by the imaging device 20. Specifically, since the gypsum board 5 is rectangular, edges c1 to c4 consisting of the four sides of the gypsum board 5 may be detected by edge detection (such as edge detection using the Canny method) and commonly known line detection (such as line detection using the Hough method).

[0028] A quadrangular area surrounded by this edge 5b may be extracted as an image G2 of the gypsum board 5. In this embodiment, a plurality of gypsum boards 5 are detected, and the images G2 of these gypsum boards 5 are extracted individually, but for example, these gypsum boards 5 may be extracted as a single aggregate.

[0029] Alternatively, the board image extraction unit 11 may use, for example, a support vector machine (SVM) or the like to learn the features of the gypsum board from the captured image using an image of the gypsum board captured by an imaging device and the features of the shape of the gypsum board in the image (for example, multiple points along the edge of the gypsum board) as training data. This makes it possible to extract an image G2 of the gypsum board from any overall image G1 that includes an image of the gypsum board. Alternatively, after identifying the gypsum board itself using a cascade classifier that uses features such as Haar-like features, the image G2 of the gypsum board 5 may be extracted from the image of the identification range using the machine learning described above.

[0030] 3-2. About the position identification unit 12 The position identification unit 12 identifies the positions of multiple screws 6 relative to the overall image G1 from the extracted image of the gypsum board 5. For example, in this embodiment, it is sufficient to be able to calculate the screw pitch, and ultimately, it is sufficient to identify the center position of the screws 6. Therefore, since it is not necessary to accurately identify the image size of the screws 6 or the shape of the screws 6, the screws 6 may be identified using a cascade classifier or the like that has machine learning knowledge of the screw shapes. In this way, by identifying the position of the screws 6 in the image shown in FIG. 4 before projective transformation, the screws 6 can be identified from the front image of the gypsum board 5 after projective transformation, as shown in FIG. 7. For example, a circular mark m indicating the identified screw 6 is added around the screw 6. This mark may be a square, diamond, or other mark other than a circle. Alternatively, the position identification unit 12 may perform binarization or grayscale processing on the transformed front image and identify the screws 6 based on the number of pixels that have a predetermined brightness difference from the surrounding pixels.

[0031] 3-3. About the projective transformation unit 13 The projection transformation unit 13 performs projection transformation on the extracted image G2 of the gypsum board 5 to form a front image G3 viewed from the front. The projection transformation method used by the projection transformation unit 13 is not particularly limited as long as it is possible to perform projection transformation on the extracted image G2 of the gypsum board 5 to form a front image G3 viewed from the front. For example, the unit 13 may input the lengths of the short and long sides of the rectangular gypsum board 5 that have been input in advance, determine whether the corresponding edges c1 to c4 of the image G1 of the gypsum board 5 correspond to the short side or the long side, and convert the extracted image G2 of the gypsum board 5 to the front image G3 so that the aspect ratio of the edges c1 to c4 of the image G1 of the gypsum board 5 matches the ratio of the short side to the long side.

[0032] 3, the projective transformation unit 13 includes a four-corner identification unit 13A, a center coordinate identification unit 13B, a position coordinate estimation unit 13C, and a front image generation unit 13D. The four-corner identification unit 13A obtains the intersections of the detected edges c1 to c4 in the image G1 of the plasterboard 5, and identifies the extracted four corner pixels g1 to g4 of the plasterboard 5 in the overall image G1 (specifically, the two-dimensional coordinates of the pixels g1 to g4 in the overall image G1).

[0033] The center coordinate identifying unit 13B identifies the coordinates of the pixel gc at the center of the extracted gypsum board from the coordinates of the pixels g1 to g4 at the four corners. Specifically, the coordinates of the center pixel gc are the intersection of the line segment connecting the diagonal pixels g1 and g3 at the four corners and the line segment connecting the diagonal pixels g2 and g4 at the four corners. Furthermore, the center coordinate identifying unit 13B identifies the coordinates of the pixel GC at the center of the entire image G1.

[0034] The position coordinate estimation unit 13C estimates the three-dimensional coordinates of the identified four corners in the virtual space V represented by the entire image G1. The position coordinate estimation unit 13C estimates the depth coordinate z of the identified four corner pixels in the virtual space V represented by the entire image G1 with respect to the captured plasterboard 5, based on the coordinates of the images g1 to g4 of the four corners, the coordinates of the pixels gc and GC at the center of the image G2 and the entire image G1, and the field of view FOV of the imaging device 20. For example, in FIG. 5, in the virtual space V, the corner point of pixel g2 appears to be closer than the corner point of pixel g1. Similarly, the corner point of pixel g3 appears to be closer than the corner point of pixel g4. Therefore, the position coordinate estimation unit 13C estimates the depth coordinate z so that the relative positions of the four corner points in the three-dimensional coordinate system within the virtual space V are identified. Furthermore, the estimated coordinate z may be used to correct the two-dimensional coordinate values ​​of the four corner points identified with respect to the entire image G1. The following series of equations calculates the depth coordinates of the corner points of pixels g1 and g3, but it goes without saying that the depth coordinates of the corner points of pixels g2 and g4 can also be calculated in a similar manner. In the following calculations, the three-dimensional position coordinates of the four corners g1 to g4 are estimated by utilizing the fact that the distances from the center coordinate gc of the gypsum board 5 to the four corners g1 to g4 are all the same.

[0035] TIFF0007752501000001.tif783

[0036] The variables shown in equation (1) are as follows, and equation (1) can be derived from FIG. 6(a). α: The number of pixels from the center pixel gc of the gypsum board image to the corner pixel g1 β: Number of pixels from the center pixel gc of the gypsum board image to the corner pixel g3 γ: When a perpendicular line is drawn from the pixel GC at the center of the entire image G1 to the diagonal line of pixels g1 and g3 of the plaster board 5, the intersection point p13 with the diagonal line of the plaster board 5 (see FIG. 5) is the angle formed by a line segment D13 from the optical center point O of the imaging device 20 to the intersection point p13 and a line segment Dc from the optical center point O of the imaging device 20 to the pixel gc at the center of the image G2 of the plaster board 5 δ: the angle formed by the line segment Dc from the optical center point O of the imaging device 20 to the pixel gc at the center of the image G2 of the gypsum board 5 and the line segment D1 from the optical center point O of the imaging device 20 to the pixel g1 at the corner of the gypsum board 5 ε: the angle formed by the line segment Dc from the optical center point O of the imaging device 20 to the pixel gc at the center of the image G2 of the gypsum board 5 and the line segment D3 from the optical center point O of the imaging device 20 to the pixel g3 at the corner of the gypsum board 5 z: Depth coordinate of pixel g1 at the corner of the gypsum board

[0037] Note that the line segment Dp from the optical center point O of the imaging device 20 to the pixel GC at the center of the entire image G1 can be calculated using the following formula (2). The line segment D1 from the optical center point O of the imaging device 20 to the pixel g1 at the corner of the plaster board 5 can be calculated using the following formula (3). The line segment Dc from the optical center point O of the imaging device 20 to the pixel gc at the center of the image G2 of the plaster board 5 can be calculated using the following formula (4). The line segment D13 from the optical center point O of the imaging device 20 to the intersection point p13 can be calculated using the following formula (5).

[0038] TIFF0007752501000002.tif5052

[0039] Among the variables in these equations, FOV, Hp, Wp, l1, lc, and l13 are as follows, and equations (2) to (5) can be derived from FIGS. 6(b) and (c). FOV: angle of view of the imaging device 20 (may be calculated from the focal length) Hp: number of vertical pixels of the whole image G1 Wp: number of horizontal pixels of the whole image G1 l1: The number of pixels from the center pixel GC of the entire image G1 to the corner pixel g1 lc: The number of pixels from the center pixel GC of the whole image G1 to the center pixel gc of the plasterboard image l13: The number of pixels from the center pixel GC of the entire image G1 to the intersection point p13 (see Figure 5)

[0040] In this way, using the values ​​calculated by equations (2) to (5) and the law of cosines, angle δ can be expressed using equation (6) shown below, and in a similar manner, angles ε and γ can also be expressed using equations (7) and (8) shown below.

[0041] TIFF0007752501000003.tif3369

[0042] After calculating the z coordinates of the four corners of the gypsum board 5 in the depth direction in this way, correction amounts Δd1 and Δd2 are calculated on the diagonal of the corners corresponding to pixels g1 and g3 of the gypsum board 5 as necessary using Figure 6(a) and the following equations (9) and (10). This is then decomposed into components on a plane to correct the coordinates of each corner identified earlier. In Figure 6(a), the corner coordinate of pixel g1 is corrected by Δd1 so that it is moved toward the center along the diagonal of the gypsum board 5, and the corner coordinate of pixel g3 is corrected by Δd2 so that it is moved toward the center along the diagonal of the gypsum board 5. This series of methods for correcting the three-dimensional coordinates of the corners corresponding to pixels g1 and g3 can also be used to correct the three-dimensional image of the corners corresponding to pixels g2 and g4.

[0043] TIFF0007752501000004.tif1658

[0044] In this way, the position coordinate estimation unit 13C calculates the depth coordinates of the corner pixels g1 and g3 that make up the gypsum board 5 from a series of equations, and calculates the three-dimensional coordinates of each pixel g1 and g3 in pixel units from the geometric relationship.In a similar manner, the position coordinate estimation unit 13C calculates the depth coordinates of the corner pixels g2 and g4 that make up the gypsum board 5, and calculates the three-dimensional coordinates of each pixel g3 and g4 in pixel units from the geometric relationship.

[0045] The front image generating unit 13D estimates the lengths of the four sides of the gypsum board 5 based on the three-dimensional coordinates of the four corners, as shown in Figure 7, and converts the extracted image of the gypsum board 5 into a front image so that the rectangular area surrounded by the four sides becomes the front image of the gypsum board 5.

[0046] The position coordinate estimation unit 13C calculates the length and width of the gypsum board 5 from the three-dimensional coordinates of the obtained corner (four corner) pixels g1 to g4 using Pythagoras' theorem, and performs projective transformation on each corner using the following transformation formulas (11) and (12) so that the corners are positioned at the corner positions of the length and width (length of the long side and length of the short side) calculated from the three-dimensional coordinates.

[0047] TIFF0007752501000005.tif2258

[0048] Here, the parameters in equations (11) and (12) are as follows: x: horizontal coordinate of the pixel before conversion y: vertical coordinate of the pixel before conversion x': horizontal coordinate of the pixel after transformation y': vertical coordinate of the pixel after transformation Other a 0、 b 0、 Coefficients such as c0 are coefficients for the transformation. Note that equations (11) and (12) are commonly known equations for projective transformation.

[0049] The front image generating unit 12D algebraically calculates the coefficients a for conversion by using the coordinates of the pixels at the four corners of the gypsum board before conversion and the coordinates after conversion determined based on the length and width (length of the long side and length of the short side) estimated from the three-dimensional shape of the gypsum board. 0、 b 0、 For example, the conversion positions are determined and calculated as follows: In the above equations (11) and (12), the coefficients are uniquely determined if the coordinates and positional relationships of the pixels at the four corners of the gypsum board are known.

[0050] The vertical and horizontal lengths of the gypsum board 5 are calculated as described above. Next, for example, the coordinates of pixel g1 are made the same before and after the transformation. The horizontal coordinate of pixel g4 after the transformation is made the same as the horizontal coordinate of pixel g1, and the vertical coordinate of pixel g4 after the transformation is calculated by adding the estimated vertical length of the gypsum board to the vertical coordinate of pixel g1. Using this method, the coordinates of the pixels at the other corners g2 and g3 are estimated. By substituting these coordinate values ​​into equations (11) and (12), the coefficients of the transformation equations (11) and (12) can be calculated. Note that if a pixel gap appears in the front image G3 after the projective transformation, for example, pixels with a color tone matching the surrounding pixels are assigned. In this way, the extracted image of the gypsum board 5 is converted into a front image so that the rectangular area surrounded by the four sides of the gypsum board 5 becomes the front image G3, thereby obtaining a more accurate front image G3 of the gypsum board 5.

[0051] 3-4. Pitch calculation unit 14 The pitch calculation unit 14 calculates the screw pitch P between adjacent screws 6, 6 along the vertical direction (lengthwise direction) of the front image G3 and the screw pitch P between adjacent screws 6, 6 along the horizontal direction (lengthwise direction) of the front image G3 based on the positions of the plurality of screws 6, 6 relative to the front image G3 after projection transformation. The pitch calculation unit 14 corresponds to the "spacing calculation unit" in the present invention.

[0052] Specifically, the distance between the centers of circles m1 and m2, which serve as identification marks for the screws 6, is calculated as the number of pixels. If the length of the long or short side of the gypsum board 5 is input in advance, the actual length and width of each pixel can be calculated from these long and short sides, and the screw pitch P can be calculated from these actual lengths. Alternatively, if the entire image G1 is acquired by the imaging device 20 together with a scale of the actual lengths or a mark corresponding to this, the actual length and width of each pixel can be calculated.

[0053] 3-5. Pitch determination unit 15 The pitch determination unit 15 determines whether the screw pitch P calculated by the pitch calculation unit 14 falls within a specified range. Specifically, as shown in FIG. 7, if the measured screw pitch falls within the specified range, each screw is surrounded by a solid-line circle m1, and if it does not fall within the specified range, it is surrounded by a dashed-line circle m2. For example, the screw pitch P1 between screws 6a and 6b and the screw pitch P2 between screws 6c and 6d do not fall within the specified range, so these screws 6a to 6d are surrounded by dashed-line circles. The pitch determination unit 15 corresponds to the "spacing determination unit" defined in this invention.

[0054] According to this embodiment, the image G2 of the gypsum board 5 extracted by the board image extraction unit 11 is projectively transformed by the projection transformation unit into a front image G3 viewed from the front. As a result, even when the gypsum board 5 is imaged from an oblique direction, the image G2 of the gypsum board 5 is transformed into the front image G3 together with the screws 6, and therefore the position (position coordinates) of the screws 6 in the front image G3 of the gypsum board 5 after the projection transformation can also be identified from the position (position coordinates) of the screws 6 before the projection transformation.

[0055] Rather than directly identifying the positions of the multiple screws 6, 6, ... from the projection-transformed front image G3, the positions of the screws 6 are identified from the image G2 of the gypsum board G1 before projection transformation, in which the image of the screws 6 is easy to identify, and then the positions of the multiple screws 6, 6, ... in the projection-transformed front image G3 can be identified based on those positions. As a result, the pitch calculation unit 14 can accurately calculate the spacing between the screws 6, 6 regardless of the imaging conditions.

[0056] An estimation flow diagram using the estimation device will be described below with reference to FIG. First, in step S1, the imaging device 20 captures an image of an area including the gypsum board 5 as an inspection area, and acquires an entire image G1.

[0057] Next, in step S2, the board image extraction unit 11 extracts an image G2 of the gypsum board 5 from the overall image G1 including the gypsum board 5, and detects edges c1 to c4 consisting of the four sides of the gypsum board. At the same time, the position identification unit 12 identifies the positions of the screws relative to the overall image G1.

[0058] Next, in step S3, the four corner specifying unit 13A specifies the pixels g1 to g4 at the four corners of the gypsum board 5 from the edges c1 to c4 detected in step S2. In step S4, the center coordinate specifying unit 13B specifies the coordinates of the pixel gc at the center of the image G2 of the gypsum board 5 from the coordinates of the pixels g1 to g4 at the four corners, and also specifies the coordinates of the pixel GC at the center of the entire image G1.

[0059] Next, in step S5, the position coordinate estimation unit 13C estimates the three-dimensional coordinates of the four corners in the virtual space V of the captured overall image G1 of the gypsum board 5 from the coordinates of the four corner images g1 to g4 (two-dimensional coordinates relative to the overall image G1), the coordinates GC and gc of the central pixel, and the field of view FOV of the imaging device.

[0060] Next, in step S6, the front image generation unit 12D performs projective transformation from the three-dimensional coordinates of the four corners in the virtual space V to generate a front image G3 viewed from the front. Since the front image generation unit 12D generates the front image G3 from the three-dimensional coordinates of the four corners in the virtual space V, the front image G3 of the gypsum board 5 can be generated with higher accuracy.

[0061] Next, in step S7, the positions (center positions) of the multiple screws 6 are identified from the front image G3 of the gypsum board 5, and in step S8, the pitch calculation unit 14 calculates the screw pitch P of adjacent screws from the image of the identified screws 6. Finally, in step S9, the pitch determination unit 15 determines whether the screw pitch is within a specified range.

[0062] 4. Modification of the position coordinate estimation unit 13C of the projective transformation unit 13 The position coordinate estimation unit 13C estimates the three-dimensional coordinates of the identified four corners in the virtual space represented by the entire image G1. Specifically, as shown in FIG. 10, the position coordinate estimation unit 13C sets the optical center point O of the image capture device 20 as the origin of a three-dimensional Cartesian coordinate system defined by the X-axis, Y-axis, and Z-axis. More specifically, the position coordinate estimation unit 13C sets the optical axis of the image capture device 20 as the Z-axis, and sets the entire image G1 on an image plane that is a plane orthogonal to the Z-axis so that the entire image G is reflected in the image plane in accordance with the field of view FOV of the image capture device 20. Here, the image plane is a plane orthogonal to the Z-axis that is formed at a position a focal length away from the optical center point O along the Z-axis.

[0063] In this modification, the three-dimensional coordinates of pixels g1 and g3 among the four corner pixels of the captured whole image G1 are respectively expressed as (x g1 ,y g1 ,z g1 ), (x g3 ,y g3 ,z g3 ), and the three-dimensional coordinates of the center pixel gc of the gypsum board 5 are (x gc ,y gc ,z gc ), and these coordinates are set in units of pixels or actual length (length in real space). As will be described later, the Z coordinates of these three-dimensional coordinates have the same value, and when set in units of pixels, they correspond to Dp shown in equation (2) already shown.

[0064] With the above settings, as shown in FIG. 10, the optical axis of the imaging device 20 coincides with the center pixel gc of the entire image G1, and in the Cartesian coordinate system, the entire image G1 captured on the image plane is positioned (on the Z axis) so as to fit exactly within the imaging range R of the imaging device 20. As a result, the X and Y coordinates of the pixels at the four corners g1 to g4 of the entire image G1 can be determined by, for example, the number of pixels constituting the entire image G1. Furthermore, the distance from the optical center point O of the imaging device 20 to the image plane (i.e., the Z coordinate position of the images of the four corners on the image plane) can be calculated using the same number of pixels as the pixels of the entire image. Note that, as shown in FIG. 10, the intersection of the image plane and a line connecting the four corners B1 to B4 of the gypsum board 5 and the optical center point O of the imaging device 20 in the imaging range (field of view) R of the imaging device 20 corresponds to the coordinates of the pixels g1 to g4 at the four corners of the gypsum board 5. In this embodiment, the three-dimensional coordinates of the four corners of the gypsum board 5 or their relationships are estimated using the three-dimensional coordinates of pixels g1 to g4 at the four corners on the image plane and the three-dimensional coordinates of a pixel gc at the center of the gypsum board 5 on the image plane. Below, the three-dimensional coordinates of the corners B1 and B3 of the gypsum board 5 are shown as examples, and a method for calculating these three-dimensional coordinates will be described.

[0065] First, in the above-mentioned Cartesian coordinate system, the coordinates of the corners B1 and B3 of the four corners of the actual gypsum board and the coordinates of the center Bc of the gypsum board 5 are expressed as (x B1 ,y B1 ,z B1 ), (x B3 ,y B3 ,z B3 ), (x Bc ,y Bc ,z Bc )

[0066] As shown in FIG. 11(a), by utilizing the geometrical similarity relationship, the X coordinate x of the center Bc of the gypsum board 5 and the corners B1 and B3 located diagonally out of the four corners of the gypsum board 5 are B1 ,x B3 ,x Bccan be expressed as the following formulas (13) to (15). In addition, since the center Bc of the gypsum board 5 is the midpoint between the corner B1 and the corner B3, the X coordinate x Bc The X coordinate x of the corners B1 and B3 B1 ,x B3 Using this, it can be expressed as in equation (16).

[0067] TIFF0007752501000006.tif5053

[0068] Next, in equation (16), we add x in equations (13) to (15). B1 ,x B3 ,x Bc Substituting the above, we can obtain equation (17).

[0069] TIFF0007752501000007.tif1770

[0070] Next, the Z coordinate z of the center Bc of the gypsum board 5 Bc Since the center Bc is the midpoint between the corner B1 and the corner B3, the Z coordinate z of the center Bc of the gypsum board 5 is Bc The Z coordinates of the corners B1 and B3 are z B1 ,z B3 By using the formula (18), the Z coordinate z of the corner B3 can be expressed as follows: B3 can be expressed as in equation (19).

[0071] TIFF0007752501000008.tif2054

[0072] Next, in equation (17), Z in equation (19) B3 Substituting Z from equation (17), B3 By eliminating, we can obtain equation (20).

[0073] TIFF0007752501000009.tif1881

[0074] Here, the image plane is a plane perpendicular to the Z axis, and is formed along the Z axis at a position spaced apart from the optical center point O by the focal length. Therefore, the z coordinate z of the pixels g1 and g3 on the image plane and the center pixel gc of the gypsum board 5 are g1 ,z g3 ,z gc are all the same value. Therefore, equation (21) can be obtained from equation (20).

[0075] TIFF0007752501000010.tif1382

[0076] By transforming this equation (21), we can obtain equation (22). Here, the Z coordinate z of the corner B1 of the gypsum board 5 is B1 is the X coordinate x of the pixels g1 and g3 at the corners of the gypsum board 5 in the whole image G1. g1 ,x g3 , the X coordinate x of the center pixel gc of the gypsum board 4 gc , and the distance z from the optical center point O to the center Bc of the gypsum board 5 Bc In the same way, the Z coordinate z of the corner B3 of the gypsum board 5 can be calculated. B3 can also be derived as in equation (23), and by substituting equation (22) into equation (19), the Z coordinate z of the corner B3 of the gypsum board 5 can be calculated. B3 can be calculated.

[0077] TIFF0007752501000011.tif3373

[0078] And z in equation (22) B1 Substituting this into equation (13), the X coordinate x of the corner B1 of the gypsum board 5 is B1 The Y coordinate of the corner B1 can be obtained by B1 For this, we create an equation similar to equation (13), and add z in equation (22) to this equation. B1 Substituting this, the Y coordinate of corner B1 is y B1 In this way, the following three-dimensional coordinates (24) of the corner B1 of the gypsum board 5 can be obtained.

[0079] TIFF0007752501000012.tif15135

[0080] Furthermore, similarly, using equation (23), the following three-dimensional coordinates (25) can be obtained as the three-dimensional coordinates of the corner B1 of the gypsum board 5. Although detailed explanation will be omitted, the three-dimensional coordinates of B2 and B4 of the gypsum board 5 can also be calculated using a similar method.

[0081] TIFF0007752501000013.tif20146

[0082] In this way, use are made of the three-dimensional coordinates of the pixels g1 to g4 at the four corners on the image plane and the three-dimensional coordinates of the pixel gc at the center of the gypsum board 5. Using these three-dimensional coordinates, and taking advantage of the fact that the intersection of the lines connecting the diagonally positioned corners B1 and B3 (B2, B4) among the three-dimensional coordinates of the four corners of the gypsum board 5 is the three-dimensional coordinate of the center Bc of the gypsum board 5, which is the coordinate corresponding to the three-dimensional coordinate of the pixel gc at the center of the gypsum board 5, the three-dimensional coordinates including the Z coordinates of the four corners of the gypsum board 5 can be easily estimated.

[0083] If the length of either the vertical or horizontal side of the rectangular gypsum board 5 is known in advance, the Z coordinate z Bc can be set to any value. In this case, the ratio of the three-dimensional coordinates of each of the corners B1 to B4 can be calculated from the three-dimensional coordinates (24) of each of the corners B1 to B4 described above, and by combining the side length based on this ratio (calculated side length) with either the length or width of the rectangular gypsum board 5 (actual side length), the ratio can be calculated, and the other side length and the three-dimensional coordinates of each of the corners B1 to B4 can be accurately calculated.

[0084] Furthermore, for example, as shown in FIG. 11(b), the Z coordinate z Bc , the Z coordinate z of the center pixel of the gypsum board 5 on the image plane (whole image G1) gcIn this case, it is possible to obtain the position coordinates of the four corners of the entire image G1 (image plane) assuming that the plasterboard 5 is the subject of the image capture (i.e., the position coordinates of the four corners of the plasterboard 5 reduced so that the center Bc of the plasterboard coincides with the center pixel gc). Specifically, the distance z from the optical center point O to the image plane converted into the number of pixels can be obtained by gc By substituting the coordinates (specifically, focal length) into each equation shown in the three-dimensional coordinates (24) and the like, the three-dimensional coordinates (including the Z coordinate) of each pixel can be calculated in terms of the number of pixels. This method is similar to the embodiment described with reference to FIGS. 6(a) to 6(c). As in the above-described embodiment, this method is used to calculate the length of each side in terms of the number of pixels from the position coordinates of the four corners before projective transformation, and the actual length corresponding to the length and width of one pixel can be calculated from the length calculated using this number of pixels and the actual side length of the gypsum board 5, and the position coordinates of the four corners can also be found to perform projective transformation to an image that looks like it was taken from the front.

[0085] The Z coordinate z, which corresponds to the distance from the optical center point O to the image plane (whole image G1), gc corresponds to Dp shown in the formula (2) already shown, and the Z coordinate z of the gypsum board 5 Bc Calculate the value (number of pixels) and calculate the Z coordinate z Bc Using these values, the three-dimensional coordinates (number of pixels) of each of the corners B1 to B4 can be calculated.

[0086] Furthermore, the Z coordinate z of the center Bc of the gypsum board 5 Bc corresponds to the distance from the optical center point O of the imaging device 20 to the center Bc of the gypsum board 5. Therefore, by measuring this distance using a displacement meter (for example, a laser displacement meter) mounted on the imaging device 20, the Z coordinate z Bc This allows all values ​​shown in the three-dimensional coordinate equations (24) and (25) to be known, and the Z coordinate z Bc Furthermore, the three-dimensional coordinates of the four corners of the gypsum board 5 in the real space can be estimated by using the above.

[0087] As a result, the Z coordinate z of the center point Bc of the gypsum board 5 was calculated from the distance measured by the displacement meter. Bc Therefore, as described above, the actual coordinates (coordinates in real space) of the four corners of the gypsum board 5 can be obtained without inputting the length of either the vertical or horizontal side of the rectangular gypsum board 5, which is the scale for each pixel of the captured overall image G1, into the estimation device 10.

[0088] In this way, the three-dimensional coordinates of the four corners of the gypsum board 5 can be calculated, and in the same manner as in the above-described embodiment, the calculated three-dimensional coordinates of the four corners are used to generate a front image of the gypsum board 5 in the above-described front image generation unit 13D, and the screw pitch is calculated in the pitch determination unit 15.

[0089] Although the embodiments of the present invention have been described in detail above, the present invention is not limited to the above-described embodiments, and various design modifications can be made without departing from the spirit of the present invention as set forth in the claims.

[0090] For example, in this embodiment, the optical center point of the imaging device is set as the origin of a three-dimensional Cartesian coordinate system, and an image plane is used that is advanced from the optical center point along the Z axis by the focal length. However, if the above-described calculation method can be used, a Cartesian coordinate system may be set in which at least one of the origin, X axis, Y axis, or Z axis is translated in a predetermined direction. [Explanation of symbols]

[0091] 5: plasterboard, 6: screws, 10: estimation device, 11: board image extraction unit, 12: position identification unit, 13: projective transformation unit, 13A: four corner coordinate identification unit, 13B: center coordinate identification unit, 13C: position coordinate estimation unit, 13D: front image generation unit, 14: pitch calculation unit (spacing calculation unit), 15: pitch determination unit (spacing determination unit), 20: imaging device, G1: overall image, G2: plasterboard image, G3: front image

Claims

1. An estimation device that estimates the distance between adjacent fasteners among a plurality of fasteners driven into a rectangular plate material, an image extraction unit that extracts an image of the plate from an overall image including the plate captured by an imaging device; a position specifying unit that specifies position coordinates of the plurality of fasteners relative to the entire image from the extracted image of the plate material; a projection transformation unit that performs projection transformation on the extracted image of the plate material into a front image viewed from the front; At least the image extraction unit detects edges consisting of four sides of the plate material; The projection transformation unit includes a four-corner identification unit that identifies four corners of the extracted plate material from the detected edges; a position coordinate estimation unit that estimates three-dimensional coordinates of the identified four corners; a front image conversion unit that estimates lengths of four sides of the plate based on the three-dimensional coordinates of the four corners and converts the extracted image of the plate into the front image so that a rectangular area surrounded by the four sides becomes the front image of the plate; a fixture position specifying unit that specifies position coordinates of the plurality of fixtures relative to the projectively transformed front image from the position coordinates of the plurality of fixtures specified by the position specifying unit, The estimation device is characterized in that it further includes a spacing calculation unit that calculates the spacing between adjacent fixing devices based on the position coordinates of the plurality of fixing devices identified by the fixing device position identification unit.

2. the position coordinate estimation unit sets an optical center point of the imaging device as an origin in a three-dimensional Cartesian coordinate system defined by an X axis, a Y axis, and a Z axis, sets an optical axis of the imaging device as the Z axis, and sets the entire image on an image plane that is a plane perpendicular to the Z axis; 2. The fastener spacing estimation device according to claim 1, wherein the three-dimensional coordinates of the four corners of the plate are estimated using the three-dimensional coordinates of the pixels at the four corners on the image plane and the three-dimensional coordinates of the pixel at the center of the plate on the image plane.

3. 3. The fastener spacing estimation device according to claim 2, wherein the position coordinate estimation unit estimates the three-dimensional coordinates of the four corners by further using information on the distance measured from the imaging device to the plate material.

4. The fastener spacing estimation device according to any one of claims 1 to 3, further comprising a spacing determination unit that determines whether the calculated spacing between the fasteners falls within a specified range.

Images