Image matching device, image matching method, and image matching program

Abstract

To provide an image matching device capable of inspecting sleeves in an accurate and efficient manner.SOLUTION: An image matching device includes: an ellipse detection unit for detecting ellipses around a top surface for each sleeve from an image in which a plurality of sleeves is captured; an ellipse matching unit for associating the detected ellipse with a circle of the sleeve shown in a design drawing, frontalizing the ellipses based on a correspondence relation, and measuring a position error between the frontalized ellipse and the circle in the design drawing; and a matching result display unit for displaying the frontalized ellipse and the circle in the design drawing.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to an image matching device, an image matching method, and an image matching program. [Background technology]

[0002] In order to smoothly carry out work at a construction site, it is important to carry out construction management and progress management accurately and quickly.

[0003] One of the targets of construction management is sleeve construction. Sleeve construction is the work of installing sleeves to leave holes for piping and cables before pouring concrete (see, for example, Patent Document 1). If the sleeve is installed in the wrong position that differs from the design drawings, or if the sleeve is forgotten to be installed, work will be required to re-drill the hole (coring) after the concrete has hardened. Coring can cause rebar to break off, resulting in a loss of necessary earthquake resistance. Furthermore, it may be necessary to rebuild the entire building. The larger the building, such as an apartment building or office building, the more sleeves are required. Therefore, the larger the building, the more accurate and fast the inspection of installed sleeves is required. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] International Publication No. 2018 / 066614 [Non-patent literature]

[0005] [Non-Patent Document 1] Juho Kannala, Mikko Salo, and Janne Heikkila. “Algorithms for Computing a Planar Homography from Conics in Correspondence,” BMVC 2006 [Non-patent document 2] Hisatoshi Toriya, Itaru Kitahara, Yuichi Ohta, “Mobile Camera Localization Using Aerial-View Images,” IPSJ Transactions on Computer Vision and Applications (CVA), Vol.6, pp.111-119, October 2014 Summary of the Invention [Problem to be solved by the invention]

[0006] However, sleeve inspection requires measuring each sleeve with a tape measure and comparing it with the design drawings. This can lead to issues such as a lack of accuracy and installation errors. Furthermore, since it is done manually, it requires a lot of man-hours.

[0007] Patent Document 1 describes a sleeve position inspection device that reduces the burden on an operator when inspecting whether a sleeve is in the correct position. The sleeve position inspection device uses a camera or the like to enable the operator to visually check whether the sleeve is in the appropriate position.

[0008] However, when using the sleeve position inspection device described in Patent Document 1, a sleeve marker must be installed on each sleeve. In other words, since the work of installing the sleeve markers is required, the number of inspection steps increases for large-scale buildings.

[0009] An object of the present invention is to provide an image matching device, an image matching method, and an image matching program that are capable of performing sleeve inspection accurately and efficiently. [Means for solving the problem]

[0010] An image matching device based on the present disclosure includes an ellipse detection unit that detects an ellipse around the top surface of each sleeve from an image of multiple sleeves, an ellipse matching unit that matches the detected ellipse with the circle of the sleeve shown in the design drawing, frontalizes the ellipse based on the correspondence, and measures the position error between the frontalized ellipse and the circle on the design drawing, and a matching result display unit that displays the frontalized ellipse and the circle on the design drawing.

[0011] The image matching method according to the present disclosure detects an ellipse around the top surface of each sleeve from an image of multiple sleeves, matches the detected ellipse with the circle of the sleeve shown in the design drawing, front-onizes the ellipse based on the correspondence, measures the positional error between the front-on ellipse and the circle in the design drawing, and displays the front-on ellipse and the circle in the design drawing.

[0012] An image matching program based on the present disclosure causes a computer to execute the following processes: detect an ellipse around the top surface of each sleeve from an image of multiple sleeves; match the detected ellipse with the circle of the sleeve shown in the design drawing; frontalize the ellipse based on the correspondence; measure the positional error between the frontalized ellipse and the circle in the design drawing; and display the frontalized ellipse and the circle in the design drawing. [Effects of the Invention]

[0013] According to the present invention, sleeve inspection can be performed accurately and efficiently. [Brief explanation of the drawings]

[0014] [Figure 1] FIG. 1 is a block diagram showing an example of the configuration of an image matching device. [Figure 2] 10 is a flowchart showing the processing of the image matching device. [Figure 3] FIG. 10 is an explanatory diagram illustrating a specific example of image matching. [Figure 4] FIG. 2 is an explanatory diagram illustrating a specific example of input and output of the image matching device. [Figure 5] FIG. 10 is a block diagram showing another example of the configuration of the image matching device. [Figure 6]10 is a flowchart showing the processing of the image matching device. [Figure 7] FIG. 10 is a block diagram showing yet another example configuration of the image matching device. [Figure 8] 10 is a flowchart showing the processing of the image matching device. [Figure 9A] 10A and 10B are explanatory diagrams illustrating specific examples of sleeve height deviations. [Figure 9B] 10A and 10B are explanatory diagrams illustrating specific examples of sleeve height deviations. [Figure 10] FIG. 10 is a block diagram showing another example of the configuration of the image matching device. [Figure 11] 10 is a flowchart showing the processing of the image matching device. [Figure 12] FIG. 1 is a block diagram illustrating an example of the configuration of a computer. DETAILED DESCRIPTION OF THE INVENTION

[0015] Hereinafter, an embodiment will be described with reference to the drawings.

[0016] Embodiment 1. Fig. 1 is a block diagram showing an example of the configuration of an image matching device. Image matching device 10 shown in Fig. 1 is a device that receives image data of a sleeve and design drawing data of the sleeve as input, and matches the sleeve position in the image data with the sleeve position in the drawing data. Image matching device 10 includes an ellipse detection unit 11, an ellipse matching unit 12, and a matching result display unit 13.

[0017] [Configuration Description] Image data showing a sleeve captured by an imaging device (not shown) such as a camera is input to the ellipse detection unit 11. The ellipse detection unit 11 detects the periphery of the top surface of the sleeve as an ellipse from the image based on the input image data. It is assumed that two or more sleeves are captured in the image. This is because if there is only one sleeve at the site, it is easy to measure it manually. The image is one or more images taken from a bird's-eye view of the location where the sleeve is installed.

[0018] Further, design drawing data of the sleeve is input to the image matching device 10. The design drawing data is input, for example, by an inspector via an input device (not shown). The ellipse matching unit 12 first determines which sleeve in the design drawing corresponds to each of the two or more detected ellipses. On a plan view, a sleeve is depicted as a circle. Therefore, the ellipse matching unit 12 associates the ellipses with the circles. Next, the ellipse matching unit 12 front-faces the ellipses based on the correspondence between the two or more ellipses and the circles. The ellipse matching unit 12 then aligns the front-faced ellipses with the circles in the design drawing. Hereinafter, the circles in the design drawing are referred to as drawing circles. Furthermore, the ellipse matching unit 12 measures a position error. The position error is the difference between the center coordinates of the front-faced ellipse and the drawing circle. Note that front-facedness refers to coordinate transformation (frontal transformation) to orient an object so that it faces forward.

[0019] The matching result display unit 13 displays the frontal ellipse and the drawing circle superimposed on a display device (not shown). The matching result display unit 13 also displays the measured position errors near each sleeve on the display screen of the display device. The matching result display unit 13 may include a display device.

[0020] [Explanation of operation] Next, the operation of the image matching device 10 will be described with reference to Fig. 2. Fig. 2 is a flowchart showing the processing of the image matching device 10.

[0021] When image data showing a sleeve and design drawing data of the sleeve are input to the image matching device 10, the ellipse detection unit 11 detects the periphery of the upper surface of the sleeve from the image as an ellipse and outputs it (step S11).

[0022] Next, when the drawing data and the ellipses are input from the ellipse detection unit 11, the ellipse matching unit 12 first determines which circle on the design drawing corresponds to which sleeve for each ellipse. Next, the ellipse matching unit 12 front-faces the ellipses based on the correspondence between the ellipses and the circles. The ellipse matching unit 12 then aligns the front-faced ellipses with the drawing circles and outputs the measured position error to the matching result display unit 13 (step S12).

[0023] When the frontal ellipse, the drawing circle, and the position error are input to the matching result display unit 13, the matching result display unit 13 displays the frontal ellipse and the drawing circle in an overlapping manner, and displays the position error near each sleeve (step S13).

[0024] Next, a specific example of image matching will be described.

[0025] Fig. 3 is an explanatory diagram illustrating a specific example of image matching. Fig. 3 shows a scene in which three sleeves are installed, in which an inspector takes pictures of the three sleeves (A, B, C) with an inspection terminal (including an imaging unit) and matches the images with the design drawings. Fig. 4 is an explanatory diagram illustrating a specific example of input and output of the image matching device.

[0026] In step S11 above, the ellipse detection unit 11 detects the periphery of the upper surface of the sleeve as an ellipse from the image captured by the inspection terminal. Generally, a sleeve is cylindrical. Therefore, its upper surface is observed as an ellipse in the image. To detect the ellipse, the ellipse detection unit 11 may, for example, apply image processing that binarizes the image and performs ellipse fitting to the edges of the upper surface of the sleeve. Note that the upper surface of the sleeve may be colored or patterned to make the edges stand out, and the ellipse detection unit 11 may detect the ellipse by color extraction or pattern recognition. Alternatively, the inspector may manually specify the ellipse by tracing the upper surface of the sleeve on the screen with his or her finger.

[0027] Next, in step S12, the ellipse matching unit 12 determines which circle on the design drawing indicates which sleeve each ellipse corresponds to. In the example shown in FIGS. 3 and 4, the ellipse matching unit 12 determines which of the drawing circles A to C each ellipse corresponds to. Note that A to C in "ellipse detection" in FIG. 4 correspond to the ellipses A to C. Also, A to C in "design drawing" in FIG. 4 correspond to the drawing circles A to C.

[0028] When associating ellipses with circles, for example, an inspector may manually determine the correspondence one by one. Furthermore, for example, if the sleeves are different sizes, the size relationship between the ellipses on the screen and the circles on the drawing should generally match, so the ellipse matching unit 12 may determine the correspondence based on the size of the sleeve. Furthermore, the ellipse matching unit 12 may determine the distribution of the center coordinates of the ellipses A to C and the center coordinates of the circles A to C on the drawing using the ICP (Iterative Closest Point) method.

[0029] Once the ellipses A to C and the drawing circles A to C have been associated with each other, the ellipse matching unit 12 frontalizes the ellipses A to C using the correspondence information (information indicating the correspondence, i.e., information indicating which ellipse corresponds to which drawing circle). As a method for frontalization, for example, the method described in Non-Patent Document 1 may be used. That is, the ellipse matching unit 12 may calculate a planar projective transformation matrix from the ellipses A to C to the drawing circles A to C and apply the planar projective transformation to the ellipses A to C. Once the ellipses A to C have been frontalized, the ellipse matching unit 12 aligns the frontalized ellipses A to C with the drawing circles A to C based on the correspondence information. The alignment is a two-dimensional rigid transformation (rotation and translation) from the frontalized ellipses (frontalized ellipses) A ​​to C to the drawing circles A to C.

[0030] The following method may be used to automatically associate and align the above-mentioned ellipses A to C with the drawing circles A to C. The simplest method is to perform an exhaustive search for combinations of correspondences between the ellipses A to C and the drawing circles A to C, and then perform planar projective transformation for each combination using the method described in Non-Patent Document 1. The number of combinations is proportional to the square of the number of ellipses. However, there is only one combination in which the ellipses A to C become perfect circles after planar projective transformation. Therefore, the search can be terminated when the frontalized ellipse becomes a perfect circle. Furthermore, by combining RANSAC (Random Sample Consensus) with the exhaustive search, the search can be terminated even earlier.

[0031] Then, in step S13, the matching result display unit 13 displays the frontalized ellipse and the drawing circle superimposed on the display device. The matching result display unit 13 also displays the position error near each sleeve on the display screen of the display device. If the image resolution of the frontalized ellipses A to C and the drawing circles A to C are matched, the error expressed in pixels on the image can be converted into an actual size error in meters. To improve visibility, the matching result display unit 13 may display the position error only when the position error is large (for example, 1 mm or more). In other words, the matching result display unit 13 may display the position error when the position error exceeds a predetermined threshold.

[0032] [Advantages of the first embodiment] The image matching device 10 of this embodiment can perform sleeve inspection accurately and efficiently.

[0033] This is because the sleeve positions are compared between the image of the sleeve and the design drawing, eliminating the need to measure the position of each sleeve with a tape measure. Specifying the center of the sleeve manually is difficult and involves large measurement errors. However, in this embodiment, the center of the ellipse can be estimated with high accuracy using image processing. If the sleeve measurement error is large, it becomes possible to correct the sleeve position before pouring the concrete. Furthermore, if a sleeve is left unattended, the number of sleeves differs between the photographed image and the drawing, causing a failure to match. As a result, the inspector can notice that a sleeve has been left unattended. Therefore, the worker can install the forgotten sleeve before pouring the concrete.

[0034] Embodiment 2. Fig. 5 is a block diagram showing another example of the configuration of an image matching device. Image matching device 20 shown in Fig. 5 includes ellipse detection unit 11 that performs ellipse matching using IMU sensor data in ellipse matching unit 22, ellipse matching unit 22, and matching result display unit 13. Image matching device 20 receives as input IMU (Inertial Measurement Unit) sensor data corresponding to the image data of the sleeve, in addition to image data of the sleeve and design drawing data of the sleeve.

[0035] In the image matching device 20, the ellipse matching unit 22 performs ellipse matching using IMU sensor data. The IMU sensor is a sensor that measures inertia and is made up of an acceleration sensor and a gyro.

[0036] The configurations and functions of the ellipse detection unit 11 and the matching result display unit 13 are the same as those in the first embodiment.

[0037] [Configuration Description] The ellipse matching unit 12 uses the IMU sensor data to frontalize the ellipse detected by the ellipse detection unit 11. Next, the ellipse matching unit 12 associates the frontalized ellipse with the drawing circle. Then, the ellipse matching unit 12 aligns the frontalized ellipse with the drawing circle and measures the position error.

[0038] [Explanation of operation] Next, the operation of the image matching device 20 will be described with reference to Fig. 6. Fig. 6 is a flowchart showing the processing of the image matching device 20.

[0039] The process in step S11 is the same as the process in the first embodiment.

[0040] When IMU sensor data corresponding to an image captured by an imaging device such as a camera, drawing data, and an ellipse output from the ellipse detection unit 11 are input to the ellipse matching unit 22, the ellipse matching unit 22 frontalizes the ellipse detected by the ellipse detection unit 11 using the IMU sensor data. Next, the ellipse matching unit 22 associates the frontalized ellipse with a drawing circle. Then, the ellipse matching unit 22 aligns the position of the frontalized ellipse with the drawing circle and outputs a position error (step S22).

[0041] The process in step S13 is the same as the process in the first embodiment.

[0042] Next, a specific example of image matching will be described. It is known that by using IMU sensor data, an image captured from a bird's-eye view of a floor surface can be converted into a frontal view as if it were captured vertically from above (see, for example, Non-Patent Document 2).

[0043] The ellipse matching unit 22 can directly convert the detected ellipse into a frontal view by using the sensor data and the method described in Non-Patent Document 2. After the frontal view is performed, the ellipse matching unit 22 can complete the correspondence and alignment by performing the ICP method of two-dimensional rigid body transformation in the same way as the example in the first embodiment.

[0044] [Effects of the second embodiment] In the image matching device 20 of this embodiment, the ellipse matching unit 22 can perform matching processing at higher speed.

[0045] The reason is that the ellipse matching unit 22 performs frontal transformation using IMU sensor data, and therefore does not need to calculate planar projective transformation as described in Non-Patent Document 1. In other words, it is not necessary to search all combinations to associate the ellipses A to C with the drawing circles A to C, and therefore the processing speed is increased.

[0046] Embodiment 3. Fig. 7 is a block diagram showing yet another example configuration of an image matching device. Image matching device 30 shown in Fig. 7 includes ellipse detection unit 31, ellipse matching unit 32, height correction unit 33, and matching result display unit 13. In addition to design drawing data of the sleeve, image matching device 30 receives as input image data of multiple images (image group) of the sleeve taken from multiple locations (for example, two locations) and IMU sensor data corresponding to each image data. Height correction unit 33 corrects the height direction of the sleeve using the IMU sensor data and the frontalization ellipse.

[0047] The configuration and functions of the collation result display unit 13 are the same as those in the first embodiment.

[0048] [Configuration Description] The ellipse detection unit 31 detects the periphery of the upper surface of the sleeve as an ellipse from each of a plurality of images (image group) of the sleeve taken from multiple points. The ellipse matching unit 32 front-on shapes each of the detected ellipses based on the correspondence between ellipses and circles or by using IMU sensor data. The ellipse matching unit 32 then aligns each front-on ellipse with the drawing circle and measures the position error for each.

[0049] The height correction unit 33 estimates the height deviation of the sleeve from the reference plane using the IMU sensor data and the frontal ellipse input from the ellipse matching unit 12. The height correction unit 33 also corrects the position error between the frontal ellipse and the drawing circle.

[0050] [Explanation of operation] Next, the operation of the image matching device 30 will be described with reference to Fig. 8. Fig. 8 is a flowchart showing the processing of the image matching device 30.

[0051] When image data of multiple images of a sleeve taken from multiple locations and design drawing data of the sleeve are input into the image matching device 30, the ellipse detection unit 31 detects the periphery of the top surface of the sleeve as an ellipse from each of the multiple images and outputs it (step S31).

[0052] Next, when the drawing data and the ellipses are input from the ellipse detection unit 31, the ellipse matching unit 32 first determines which circle on the design drawing indicates which sleeve each ellipse corresponds to. Next, the ellipse matching unit 32 front-faces the ellipses. The ellipse matching unit 32 then aligns the front-faced ellipses with the drawing circles and outputs the measured position error to the height correction unit 33 (step S32).

[0053] The ellipse matching unit 32 performs the above-described process for each of a plurality of images taken from a plurality of points.

[0054] In this embodiment, when the IMU sensor data and the frontal ellipse from the ellipse comparison unit 32 are input, the height correction unit 33 first estimates the height deviation of the sleeve from the reference plane. Next, the height correction unit 33 corrects the position error between the frontal ellipse and the drawing circle (step S33).

[0055] The processing by the height correction unit 33 will be described below. FIGS. 9A and 9B are explanatory diagrams illustrating a specific example of sleeve height deviation. FIG. 9A shows a schematic side view of the sleeve when photographed from two different points, showing the state in which the sleeve height deviates from the design reference plane. FIG. 9B shows a schematic top view of the sleeve when photographed from two different points, showing the state in which the sleeve height deviates from the design reference plane. In FIG. 9B, image 1 is an image photographed from a certain point. Image 2 is an image photographed from another point. The frontalized ellipses of image 1 and image 2 correspond to the frontalized ellipses of the respective images associated with the same circle (one circle) in the design drawing.

[0056] In FIG. 9A, the deviation (height deviation) of the actual sleeve height from the design reference plane is z, and the observation angle between the camera that observed the i-th image and the vertical direction is θ i , the apparent length of z in the i-th image is r i In Figure 9B, the center position of the frontal ellipse of the i-th image is (x i , y i ), and the actual center position is (x c , y c ), and the angle between the actual center position and the center position of the two frontal ellipses is φ. Here, z and (x c , y c ) and φ are unknown values.

[0057] The height correction unit 33 corrects the observation angle θ from the gravity direction of the IMU sensor data. i The height deviation z and apparent length r can be obtained. i between r i = z * tan(θ i The relationship between the center position of the frontal ellipse and the actual center position is expressed by the following equation.

[0058] (x1- x c , y1- y c ) = (0, r1) (x2- x c , y2- y c ) = (r²*sin(φ), r²*cos(φ)) ...Formula 1

[0059] The upper equation in Equation 1 is x c = x1, y c This can be transformed into =y1-r1, and by substituting this into the lower equation in equation 1, we obtain the following equation.

[0060] x2- x1= z * tan(θ2) * sin(φ) y2- y1+ z * tan(θ1) = z * tan(θ2) * cos(φ) ...Formula 2

[0061] If we square both sides and add them, we can eliminate φ from cos(φ)^2 + sin(φ)^2=1, and it becomes a quadratic equation for z. In other words, we get two solutions for z. Substituting these into equation 2 gives us (x c , y c ) and φ can be calculated. The reason two solutions are obtained is because Equation 2 is squared.

[0062] The height correction unit 33 can uniquely determine a solution as follows. For example, if images from three different locations are used, a total of six different z values ​​will be calculated. However, three of these values ​​should theoretically be the same value. Therefore, the height correction unit 33 can search for three values ​​that are close to each other and calculate the average value. Also, if it is assumed that the deviation in z is small, for example, the height correction unit 33 can select the image with the smaller absolute value of z from two images from two locations.

[0063] The above specific example will be explained in association with the operation of the image matching device 30.

[0064] In step S33, when the IMU sensor data of images taken from two or more different points and the frontal ellipse output by the ellipse matching unit 12 using the images are input, the height correction unit 33 first solves Equation 2 to calculate the height deviation z of the sleeve from the reference plane. Next, the height correction unit 33 calculates the center position (x c , y c ) is calculated. The height correction unit 33 then recalculates the position error between the frontal ellipse and the drawing circle.

[0065] The process in step S13 is the same as the process in the first embodiment.

[0066] [Variations] The correction process by the height corrector 33 in the third embodiment is not limited to the process described above. Below, several modifications of the correction process will be described.

[0067] Variation 3-1. The method by which the height correction unit 33 estimates the height deviation and the actual center position is not limited to solving Equation 2. For example, the detected ellipse may be converted into a three-dimensional shape using a method called Structure-from-Motion, which restores the three-dimensional motion of the camera from corresponding points in images. Note that Structure-from-Motion is a theory and method that are widely known in computer vision.

[0068] Variation 3-2. The ellipse matcher 12 may perform frontalization of the detected ellipse as the ellipse matcher 22 using IMU sensor data.

[0069] [Effects of the third embodiment] The image matching device 30 of this embodiment can measure position errors with high precision even for sleeves of different heights.

[0070] The reasons are as follows: In the first and second embodiments, it is assumed that the heights of the sleeves are the same (i.e., the upper surfaces of the sleeves are on the same plane). If the heights are not the same, the position of the frontal ellipse will shift horizontally as shown in FIG. 9B. This will result in an excessively large deviation compared to the actual deviation. By using image data captured from multiple points as in this embodiment, the height correction unit 33 can solve, for example, Equation 2 or Structure-from-Motion. As a result, the height correction unit 33 can correct the height deviation of the sleeve and calculate the actual center position of the frontal ellipse.

[0071] Embodiment 4. Fig. 10 is a block diagram showing another example of the configuration of an image matching device. Image matching device 40 shown in Fig. 10 includes ellipse detection unit 31, ellipse matching unit 32, height correction unit 33, matching result display unit 13, matching correction unit 41, and matching result recording unit 42.

[0072] The configuration of image matching device 40 is such that matching and correction unit 41 and matching result recording unit 42 are incorporated into image matching device 30 of the third embodiment. However, matching and correction unit 41 and matching result recording unit 42 may also be incorporated into image matching device 10 of the first embodiment or image matching device 20 of the second embodiment.

[0073] [Configuration Description] When the inspector inputs a correction to the correspondence information between the frontalized ellipse and the drawing circle, the matching correction unit 41 overwrites the correspondence information output from the ellipse matching unit 32, i.e., the correspondence information determined by the ellipse matching unit 32, with the correction information, and outputs the overwriting result as corrected correspondence information to the ellipse matching unit 12. When the positional error between the frontalized ellipse and the drawing circle is input from the ellipse matching unit 32, the matching result recording unit 42 records the positional error in the inspection slip.

[0074] [Explanation of operation] Next, the operation of the image matching device 40 will be described with reference to Fig. 11. Fig. 11 is a flowchart showing the processing of the image matching device 40.

[0075] The processes in steps S31 to S33 and S13 are the same as those in the third embodiment.

[0076] When the inspector inputs a correction to the correspondence information between the frontal ellipse and the drawing circle, i.e., when the correspondence information is incorrect (step S40), the matching and correction unit 41 corrects the correspondence information already determined by the ellipse matching unit 32 (step S41). Then, the matching and correction unit 41 outputs the corrected correspondence information to the ellipse matching unit 32.

[0077] The matching result recording unit 42 records the position error between the frontal ellipse output from the ellipse matching unit 42 and the drawing circle in an inspection form (step S42). The inspection form is stored as inspection form data in a storage device (not shown) that is included in the image matching device 40 or that is external to the image matching device 40, for example.

[0078] A specific example of the process of correcting the correspondence information and the process of recording the inspection form will be described below.

[0079] The inspector checks the correspondence information between the frontal ellipse and the drawing circle displayed on a display device (not shown) by matching result display unit 13. If the inspector determines that the correspondence information is incorrect, he or she inputs the correct correspondence information into image matching device 40 via an input device (not shown). In image matching device 40, ellipse matching unit 32 and height correction unit 33 use the corrected correspondence information to perform the ellipse matching process and height correction process (processing of steps S32 and S33) again.

[0080] Next, the inspector checks the correspondence information and positional error between the frontalized ellipse and the drawing circle displayed on the display device. If the inspector determines that there are no errors, he or she inputs a recording command to the image matching device 40 via the input device. In the image matching device 40, the matching result recording unit 42 records the positional error between the frontalized ellipse and the drawing circle on an inspection form. If the information is recorded correctly, the image matching device 40 ends its operation.

[0081] [Variations] The processing by the verifying and correcting unit 41 in the fourth embodiment is not limited to the processing described above. Modifications will be described below.

[0082] Variation 4-1. The matching and correction unit 41 may receive as input the three-dimensional coordinates of one or more frontal ellipses in the world coordinate system. The world coordinate system is a reference point (reference line) called a "relief mark" or "return mark" that matches the coordinate system of the measurement site and the design drawing.

[0083] In all of the first to fourth embodiments, the accuracy of the relative distance between the sleeves is measured. Therefore, it is unclear whether the absolute distance from the "relief mark" is correct. For example, if the distance from the relief mark of only one sleeve is measured with a tape measure and that value is fixed as the coordinate value of the frontal ellipse, the output of the ellipse matching units 12, 22, and 32 can be converted into a measurement result in the world coordinate system.

[0084] [Effects of the fourth embodiment] Even if incorrect correspondence information is output from the ellipse matching unit 32, the image matching device 40 of this embodiment can appropriately correct the correspondence information in cooperation with the inspector. Therefore, an accurate inspection form can be automatically created.

[0085] In this embodiment, both the matching correction unit 41 and the matching result recording unit 42 are provided, but it is also possible to provide only the matching correction unit 41. Furthermore, the matching result recording unit 42 may be provided in the image matching devices 10 to 30 of the first to third embodiments.

[0086] Each function (each process) in the above-described embodiments can be realized by a computer having a processor such as a CPU, a memory, etc. For example, a program for implementing the method (process) in the above-described embodiments may be stored in a storage device (storage medium), and each function may be realized by executing the program stored in the storage device by a CPU.

[0087] 12 is a block diagram showing an example of the configuration of a computer that can realize image matching devices 10 to 40. The computer shown in FIG.

[0088] The image matching devices 10 to 40 shown in Figures 1, 5, 7, and 10 can be realized by software. That is, for example, the functions of each block in the image matching devices 10 to 40 can be realized by a processor 1000 executing processing in accordance with a program stored in a program memory 1001 in a computer shown in Figure 12. When multiple processors are installed, the functions of the image matching devices 10 to 40 can also be realized by the multiple processors working together.

[0089] The program memory 1001 is, for example, a non-transitory computer readable medium. The non-transitory computer readable medium includes various types of tangible storage medium. For example, a semiconductor storage medium such as a flash ROM (Read Only Memory) or a magnetic storage medium such as a hard disk can be used as the program memory 1001. The program memory 1001 stores an image matching program for realizing the functions of the image matching devices 10 to 40 of the above embodiments.

[0090] A semiconductor storage medium or a magnetic storage medium can be used as memory 1002. Memory 1002 stores temporary data and the like that is generated when image matching devices 10 to 40 are executing processing. It is also possible to assume a configuration in which an image matching program is transferred to memory 1002, and processor 1000 executes processing based on the program in memory 1002. Note that program memory 1001 and memory 1002 may be integrated.

[0091] The memory 1002 can also be used as a storage device for storing inspection forms.

[0092] The input / output interface 1003 realizes a function for inputting and outputting information (data), and includes, for example, an input circuit for an inspector to input information to the image matching devices 10-40.

[0093] Although the present invention has been described above with reference to the embodiments, the present invention is not limited to the above-described embodiments. Various modifications that can be understood by those skilled in the art can be made to the configuration and details of the present invention within the scope of the present invention.

[0094] Although the present invention has been described above with reference to the embodiments, the present invention is not limited to the above-described embodiments. Various modifications that can be understood by those skilled in the art can be made to the configuration and details of the present invention within the scope of the present invention.

[0095] (Supplementary Note 1) An ellipse detection unit that detects an ellipse around the top surface of each sleeve from an image of a plurality of sleeves; an ellipse comparison unit that associates the detected ellipse with a circle of the sleeve shown in a design drawing, front-faces the ellipse based on the correspondence, and measures a position error between the front-faced ellipse and the circle on the design drawing; a comparison result display unit that displays the frontal ellipse and the circle in the design drawing; An image matching device comprising:

[0096] (Note 2) The matching result display unit displays the position error near the frontal ellipse and the circle on the design drawing. 2. The image matching device according to claim 1.

[0097] (Appendix 3) The ellipse matching unit inputs IMU sensor data corresponding to the image of the sleeve, uses the IMU sensor data to frontalize the detected ellipse, and measures the position error by associating the frontalized ellipse with a circle on the design drawing. 3. The image matching device according to claim 1 or 2.

[0098] (Supplementary Note 4) The ellipse matching unit frontally matches the detected ellipse with respect to each image in an image group consisting of a plurality of images of the sleeve photographed from a plurality of points; The image processing device further includes a height correction unit that estimates a height from a reference plane from a position difference of the frontalized ellipse of each image associated with the same circle on the design drawing, and corrects a position error between the frontalized ellipse and the circle on the design drawing. 4. The image matching device according to claim 3.

[0099] (Supplementary Note 5) The present invention further includes a matching correction unit that, when a correction of correspondence information indicating a correspondence relationship between the frontalized ellipse and a circle in the design drawing is input, corrects the correspondence information determined by the ellipse matching unit based on the input correction, and outputs the corrected correspondence information to the ellipse matching unit. 5. The image matching device according to claim 1, wherein the image matching device is a computer.

[0100] (Appendix 6) The apparatus further includes a matching result recording unit that records the position error between the frontalized ellipse output from the ellipse matching unit and the circle on the design drawing in a document. 6. The image matching device according to claim 1,

[0101] (Supplementary Note 7) The collation and correction unit receives, as input, three-dimensional coordinates in a world coordinate system of one or more of the frontalized ellipses. 6. The image matching device according to claim 5.

[0102] (Appendix 8) Detecting an ellipse around the top surface of each sleeve from an image of multiple sleeves, Corresponding the detected ellipse to a circle of the sleeve shown in the design drawing, front-facing the ellipse based on the correspondence, and measuring a position error between the front-facing ellipse and the circle on the design drawing; Display the frontal ellipse and the circle in the design drawing Image matching methods.

[0103] (Appendix 9) To the computer, A process of detecting an ellipse around the top surface of each sleeve from an image of a plurality of sleeves; a process of associating the detected ellipse with a circle of the sleeve shown in the design drawing, front-facing the ellipse based on the correspondence, and measuring a position error between the front-facing ellipse and the circle on the design drawing; and executing a process of displaying the frontalized ellipse and the circle in the design drawing. Image matching program for.

[0104] (Appendix 10) To the computer, A process of detecting an ellipse around the top surface of each sleeve from an image of a plurality of sleeves; a process of associating the detected ellipse with a circle of the sleeve shown in the design drawing, front-facing the ellipse based on the correspondence, and measuring a position error between the front-facing ellipse and the circle on the design drawing; a computer-readable recording medium having recorded thereon an image matching program for executing a process of displaying the frontalized ellipse and the circle in the design drawing;

[0105] Some or all of the configurations described in Supplementary Notes 2 to 7, which are dependent on Supplementary Note 1, may be made dependent on Supplementary Notes 8 to 10 in accordance with the same dependency relationships as Supplementary Notes 2 to 7. Furthermore, not limited to Supplementary Notes 1, 8, 9, and 10, some or all of the configurations described as the above Supplements may be made dependent on various hardware, software, various recording means for recording software, or systems, provided that they do not deviate from the above-described embodiments. [Explanation of symbols]

[0106] 10,20,30,40 Image matching device 11,31 Ellipse detection unit 12,22,32 Ellipse matching section 13 Matching result display section 33 Height correction unit 41 Verification and Correction Section 42 Matching result recording section 1000 processors 1001 program memory 1002 memory 1003 Input / Output Interface

Claims

1. an ellipse detection unit that detects an ellipse around the upper surface of each sleeve from an image of the plurality of sleeves; an ellipse comparison unit that associates the detected ellipse with a circle of the sleeve shown in a design drawing, front-faces the ellipse based on the correspondence, and measures a position error between the front-faced ellipse and the circle on the design drawing; a comparison result display unit that displays the frontal ellipse and the circle in the design drawing; An image matching device comprising:

2. The matching result display unit displays the position error near the frontal ellipse and the circle on the design drawing.

2. The image matching device according to claim 1.

3. The ellipse matching unit inputs IMU sensor data corresponding to the image of the sleeve, frontalizes the detected ellipse using the IMU sensor data, and measures the position error by correlating the frontalized ellipse with a circle on the design drawing.

2. The image matching device according to claim 1.

4. the ellipse matching unit frontally compares the detected ellipse with respect to each image in an image group consisting of a plurality of images of the sleeve photographed from a plurality of points; The image processing device further includes a height correction unit that estimates a height from a reference plane from a position difference of the frontalized ellipse of each image associated with the same circle on the design drawing, and corrects a position error between the frontalized ellipse and the circle on the design drawing.

4. The image matching device according to claim 3.

5. The method further includes a matching correction unit that, when a correction of correspondence information indicating a correspondence relationship between the frontal ellipse and a circle in the design drawing is input, corrects the correspondence information determined by the ellipse matching unit based on the input correction, and outputs the corrected correspondence information to the ellipse matching unit.

5. The image matching device according to claim 1.

6. The system further includes a comparison result recording unit that records a position error between the frontalized ellipse output from the ellipse comparison unit and the circle on the design drawing in a document.

5. The image matching device according to claim 1.

7. The collation and correction unit receives, as input, three-dimensional coordinates of one or more of the frontalized ellipses in a world coordinate system.

6. The image matching device according to claim 5.

8. Detecting an ellipse around the top surface of each sleeve from an image of a plurality of sleeves; Corresponding the detected ellipse to a circle of the sleeve shown in the design drawing, front-facing the ellipse based on the correspondence, and measuring a position error between the front-facing ellipse and the circle on the design drawing; Display the frontal ellipse and the circle in the design drawing Image matching methods.

9. On the computer, A process of detecting an ellipse around the top surface of each sleeve from an image of a plurality of sleeves; a process of associating the detected ellipse with a circle of the sleeve shown in the design drawing, front-facing the ellipse based on the correspondence, and measuring a position error between the front-facing ellipse and the circle on the design drawing; and executing a process of displaying the frontalized ellipse and the circle in the design drawing. Image matching program for.

Images