Measurement system using laser-measured point clouds and photogrammetry point clouds

Abstract

To provide a measurement system that uses laser-measured point clouds and photogrammetry point clouds, which can automatically perform laser measurement and photogrammetry without human intervention during the measurement process, and efficiently acquire point clouds that are accurate in size and free of defects. [Solution] The first shooting angle θ1 is defined as the shooting direction of the first camera 4-1 relative to the glass table 2, and the second shooting angle θ2 is defined as the shooting direction of the second camera 4-2 relative to the glass table 2. The first shooting angle θ1 is made larger than the second shooting angle θ2, and the first camera 4-1 is positioned further away from the glass table 2 than the second camera 4-2. The control device 6 determines the shooting arm trajectory and the measurement arm trajectory of the arm 5 based on the object placement range 10. The arm 5 is moved according to the shooting arm trajectory to perform initial shooting with the camera 4, and after shooting is completed, the arm 5 is moved according to the measurement arm trajectory to perform measurement with the laser device 3.

Description

Technical Field

[0001] The present invention relates to a measurement system using a laser measurement point cloud and a photographic measurement point cloud.

Background Art

[0002] The point cloud measured by a three-dimensional measuring device is used not only for recording the surface shape of an object but also for various applications such as shape similarity evaluation and shape modeling. Patent Document 1 proposes a multi-directional large-scale simultaneous measurement system capable of automatically measuring a large number of objects in a batch. Four laser distance sensors are attached to the arm, which is a movable part, covering 360° around the object. Then, by arranging a large number of objects on a glass table, the front and back surfaces of the objects are measured in a batch. Methods for obtaining the surface point cloud of an object include laser measurement and photographic measurement. Laser measurement can measure in actual size, but if the laser does not reach the surface of the object, the point cloud will be missing. In addition, black parts of the object are likely to cause missing because the laser is absorbed. Moreover, even if the object is measured with a laser, the color information of the object is not reflected in the point cloud. On the other hand, photographic measurement cannot measure the object in actual size, but missing is less likely to occur on the shape surface. And since the point cloud is created based on the photograph, color information can also be obtained. However, white parts of the object are likely to cause overexposure in the photograph, so missing is likely to occur.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In the multi-directional large-scale simultaneous measurement system according to Patent Document 1, a point cloud representing the shape surface can be obtained, but the color information of the object cannot be obtained. Furthermore, combining laser measurement and photogrammetry results is time-consuming and laborious.

[0005] Therefore, the present invention aims to provide a measurement system that uses laser-measured point clouds and photo-measured point clouds, which can perform laser measurement and photo-measurement automatically without human intervention during the measurement process, and can efficiently acquire point clouds that are accurate in size and free of defects. [Means for solving the problem]

[0006] The measurement system using laser measurement point clouds and photometric point clouds according to claim 1 of the present invention comprises a glass table 2 on which an object to be measured 1 is placed, a laser device 3 for acquiring a laser measurement point cloud from the object to be measured 1, a plurality of cameras 4 for photographing the object to be measured 1 to obtain a photometric point cloud, an arm 5 to which the laser device 3 and the cameras 4 are attached, and a control device 6 for controlling the operation of the laser device 3, the cameras 4 and the arm 5, wherein the arm 5 positions the laser device 3 and the cameras 4 on the surface side of the glass table 2, the shooting direction of the first camera 4-1 relative to the glass table 2 is set to a first shooting angle θ1, and the shooting direction of the second camera 4-2 relative to the glass table 2 If the direction is the second shooting angle θ2, the first shooting angle θ1 is made larger than the second shooting angle θ2, and the first camera 4-1 is positioned further away from the glass table 2 than the second camera 4-2. The control device 6 determines the shooting arm trajectory of the arm 5 that performs shooting with the camera 4 and the measurement arm trajectory of the arm 5 that performs measurement with the laser device 3, based on the object placement range 10 of the object to be measured 1 that is placed on it. The arm 5 is operated according to the shooting arm trajectory to perform shooting with the camera 4 first, and after the shooting with the camera is completed, the arm 5 is operated according to the measurement arm trajectory to perform measurement with the laser device 3. The present invention as described in claim 2 is a measurement system using a laser measurement point cloud and a photometric measurement point cloud as described in claim 1, characterized in that the arm 5 creates the photometric measurement point cloud using photographic data of the object to be measured 1 while the arm 5 is moving along the measurement arm trajectory. The present invention as described in claim 3 is a measurement system using a laser measurement point cloud and a photometric measurement point cloud as described in claim 1, characterized in that an effective shooting range E is set in advance for each of the cameras 4 attached to the arm 5 with respect to specific coordinates, the movement range of the arm 5 in which the object placement range 10 falls within the effective shooting range E is calculated, the camera 4 takes photographs within the calculated movement range of the arm 5, and the camera 4 does not take photographs outside the calculated movement range of the arm 5. The present invention as described in claim 4 is a measurement system using a laser measurement point cloud and a photometric measurement point cloud as described in claim 3, characterized in that the effective shooting range E is set to the central part of the shooting range F that can be captured by the camera 4, excluding the peripheral part. The present invention as described in claim 5 is a measurement system using a laser measurement point cloud and a photometric point cloud as described in claim 2, wherein a plurality of measurement objects 1 are placed on the glass table 2, the area including the plurality of measurement objects 1 is defined as the object placement area 10, and after creating the photometric point cloud, the height data of the photometric point cloud is used to remove as noise any space on the glass table 2 where the measurement objects 1 do not exist. [Effects of the Invention]

[0007] According to the present invention, it is possible to obtain photographs for obtaining laser measurement point clouds and photometric point clouds from an object placed on a glass table. By acquiring images that cover the entire shape surface from both images taken of the top surface and images taken of the side surface of the object to be measured, the image can be captured in a way that minimizes the loss of points in the point cloud generated from the image. Furthermore, by creating a photometric point cloud using photographic data during measurement with a laser device, the total time until measurement point cloud creation can be shortened. [Brief explanation of the drawing]

[0008] [Figure 1] Configuration diagram of a measurement system using laser measurement point clouds and photometric point clouds in one embodiment of the present invention. [Figure 2] This diagram shows how to pre-set the effective shooting range for each camera mounted on an arm, targeting specific coordinates. [Figure 3] A graph showing the position coordinates of the arm when the four corners A, B, C, and D of the effective shooting range overlap with the marker. [Figure 4] Diagram explaining the shooting point for the second camera. [Figure 5] Flowchart of a measurement system using laser measurement point clouds and photometric point clouds in one embodiment of the present invention. [Figure 6] A photograph showing a measurement system with cameras positioned on both the front and back surfaces of a glass table. [Figure 7] A diagram illustrating noise reduction using photometric markers. [Figure 8] Overview of the point cloud synthesis procedure in S25 [Figure 9] Explanatory diagram illustrating the verification of the present invention. [Figure 10] Explanatory diagram illustrating the verification of the present invention. [Modes for carrying out the invention]

[0009] In the first embodiment of the present invention, a measurement system using laser measurement point clouds and photometric point clouds is configured such that an arm positions the laser device and camera on the surface side of a glass table, the shooting direction of the first camera relative to the glass table is defined as the first shooting angle, and the shooting direction of the second camera relative to the glass table is defined as the second shooting angle. The first shooting angle is made larger than the second shooting angle, and the first camera is positioned further away from the glass table than the second camera. The control device determines the shooting arm trajectory of the arm that performs photography with the camera and the measurement arm trajectory of the arm that performs measurement with the laser device, based on the object placement range of the object to be measured. The arm is operated according to the shooting arm trajectory to perform photography with the camera first, and after photography with the camera is completed, the arm is operated according to the measurement arm trajectory to perform measurement with the laser device. According to this embodiment, by arranging the laser device and camera on the surface side of the glass table, it is possible to obtain both a photograph for obtaining a photometric point cloud and a laser-measured point cloud from an object placed on the glass table. Furthermore, according to this embodiment, by making the first shooting angle of the first camera greater than the second shooting angle of the second camera, and positioning the first camera further away from the glass table than the second camera, an image covering the entire shape surface can be obtained from both the image of the top surface and the image of the side surface of the object to be measured, thereby reducing the loss of points in the point cloud generated from the image. Furthermore, according to this embodiment, by performing measurement with a laser device using the measurement arm trajectory after the camera has finished taking pictures, it is possible to create a photometric point cloud using photographic data while the laser device is performing measurement, thereby shortening the total time until the measurement point cloud is created.

[0010] A second embodiment of the present invention is a measurement system using a laser measurement point cloud and a photometric point cloud according to the first embodiment, wherein a photometric point cloud is created using photographic data of the object to be measured while the arm is moving along the measurement arm trajectory. Although measurement by a laser device and creation of a photographic measurement point cloud require time, according to this embodiment, by creating a photographic measurement point cloud using photographic data during measurement by the laser device, the total time until creation of the measurement point cloud can be shortened.

[0011] In a third embodiment of the present invention, in a measurement system using the laser measurement point cloud and the photographic measurement point cloud according to the first embodiment, for each camera attached to the arm, an effective shooting range is set in advance for a specific coordinate, a movement range of the arm in which the object placement range enters the effective shooting range is calculated, shooting is performed by the camera within the calculated movement range of the arm, and shooting by the camera is not performed outside the calculated movement range of the arm. According to this embodiment, it is possible to prevent a quality degradation during creation of a point cloud from a photograph in which the object to be measured is not clearly shown, and to create a highly reliable photographic measurement point cloud.

[0012] In a fourth embodiment of the present invention, in a measurement system using the laser measurement point cloud and the photographic measurement point cloud according to the third embodiment, the effective shooting range is set in the central part excluding the peripheral part from the shooting range that can be shot by the camera. According to this embodiment, by setting the effective shooting range in the central part, it is possible to remove an image that may have distortion or defocus in the peripheral part, and to create a highly reliable photographic measurement point cloud.

[0013] In a fifth embodiment of the present invention, in a measurement system using the laser measurement point cloud and the photographic measurement point cloud according to the second embodiment, a plurality of measurement objects are placed on a glass table, a range including the plurality of measurement objects is set as the object placement range, and after creating the photographic measurement point cloud, using the height data of the photographic measurement point cloud, a space on the glass table where no measurement object exists is removed as noise. According to this embodiment, it is possible to facilitate creation of a photographic measurement point cloud for each measurement object.

Example

[0014] An embodiment of a measurement system using the laser measurement point cloud and photometric measurement point cloud of the present invention will be described below. Figure 1 is a diagram illustrating the configuration of a measurement system using laser-measured point clouds and photometric point clouds in one embodiment of the present invention. As shown in Figure 1(a), the measurement system using laser measurement point clouds and photometric point clouds in this embodiment comprises a glass table 2 on which the object to be measured 1 is placed, a laser device 3 for acquiring a laser measurement point cloud from the object to be measured 1, a plurality of cameras 4 for photographing the object to be measured 1 to obtain a photometric point cloud, an arm 5 to which the laser device 3 and cameras 4 are attached, and a control device 6 for controlling the operation of the laser device 3, cameras 4, and arm 5. The laser device 3 and camera 4 are positioned on the surface side of the glass table 2 by the arm 5. Arm 5 moves parallel to the glass table 2 in the X-axis and Y-axis directions. In this way, by arranging the laser device 3 and camera 4 on the surface side of the glass table 2, it is possible to obtain both a photograph for obtaining a photometric point cloud and a laser-measured point cloud from the object 1 placed on the glass table 2.

[0015] As shown in Figure 1(b), this embodiment is equipped with six cameras 4. The six cameras 4 are arranged radially around the object to be measured 1, with the target shooting position A as the center, and all cameras 4 have the target shooting position A as their shooting direction. Of the six cameras 4, the first camera 4-1 and the fourth camera 4-4 are positioned parallel to the X-axis, the second camera 4-2 and the fifth camera 4-5 are positioned at a 45° angle to the line connecting the first camera 4-1 and the fourth camera 4-4, and the third camera 4-3 and the sixth camera 4-6 are also positioned at a 45° angle to the line connecting the first camera 4-1 and the fourth camera 4-4.

[0016] As shown in Figure 1(c), the first camera 4-1, the third camera 4-3, and the fifth camera 4-5 are set to a first shooting angle θ1, while the second camera 4-2, the fourth camera 4-4, and the sixth camera 4-6 are set to a second shooting angle θ2. The shooting direction of the first camera 4-1, the third camera 4-3, and the fifth camera 4-5 relative to the glass table 2 is the first shooting angle θ1, and the shooting direction of the second camera 4-2, the fourth camera 4-4, and the sixth camera 4-6 relative to the glass table 2 is the second shooting angle θ2. The first shooting angle θ1 is between 45° and 90°, and the second shooting angle θ2 is between 0° and less than 45°, with the first shooting angle θ1 being greater than the second shooting angle θ2. Furthermore, the first camera 4-1, the third camera 4-3, and the fifth camera 4-5 are positioned further away from the glass table 2 than the second camera 4-2, the fourth camera 4-4, and the sixth camera 4-6. In this way, by making the first shooting angle θ1 of the first camera 4-1, the third camera 4-3, and the fifth camera 4-5 larger than the second shooting angle θ2 of the second camera 4-2, the fourth camera 4-4, and the sixth camera 4-6, and by positioning the first camera 4-1, the third camera 4-3, and the fifth camera 4-5 further away from the glass table 2 than the second camera 4-2, the fourth camera 4-4, and the sixth camera 4-6, it is possible to acquire an image that covers the entire shape surface from both the image of the object to be measured 1 and the image of its side, thereby reducing the loss of points in the point cloud generated from the image.

[0017] Figures 1(d) and 1(e) show the difference in image overlap when photographing the object 1 due to the difference in the angle of camera 4. As shown in Figure 1(d), when shooting at a large first shooting angle θ1, the top surface of the object to be measured 1 can be photographed by multiple cameras 4, making it easier for the photographs of the top surface of the object to be measured 1 to overlap. However, the sides of the object to be measured 1 are not photographed, so the entire surface shape cannot be covered. As a result, the point cloud generated from the images may have missing parts in the side portions or have a low point cloud density. In contrast, as shown in Figure 1(e), when imaging is performed at a smaller second imaging angle θ2, the side of the object being measured 1 is mainly captured. As a result, the images of the side of the object being measured 1 tend to overlap, but the top surface of the object being measured 1 does not overlap as much. Consequently, the point cloud generated from the images may have missing parts in the top surface area or a lower point cloud density. Furthermore, depending on the shape of the object being measured 1, the measurement results may become unstable. Therefore, by alternately mounting the first camera 4-1, the third camera 4-3, and the fifth camera 4-5 with a large first shooting angle θ1, and the second camera 4-2, the fourth camera 4-4, and the sixth camera 4-6 with a small second shooting angle θ2, it is possible to ensure overlap of the images obtained from each camera 4 and generate a point cloud without missing data. As shown in Figure 1(a), when multiple objects to be measured 1 are placed on the glass table 2, the area including the multiple objects to be measured 1 is set as the object placement area 10.

[0018] Figures 2 to 4 illustrate the shooting range for each camera. Figure 2 shows a method for pre-setting the effective shooting range for each camera 4 mounted on an arm, targeting specific coordinates. Since each of the six cameras 4 has a different shooting range F, it is necessary to consider the shooting range F of each of the six cameras 4 in order to ensure overlap of the captured images. Photogrammetry requires many photographs that show the object to be measured (object 1). However, photographs that do not show object 1, or photographs where object 1 is out of focus because it is at the edge of the frame, will lead to a decrease in the quality of the point cloud created. Therefore, as shown in Figure 2(a), when the object to be measured 1 is in the center (effective shooting range E) of the image obtained from one camera 4, which is divided into four equal parts vertically and horizontally, a photograph is taken. The area enclosed by A, B, C, and D shown in Figure 2(a) is the effective shooting range E. The effective shooting range E is set to the central part of the shooting range F that can be captured by camera 4, excluding the peripheral areas. By setting the effective shooting range E to the central part, it is possible to remove images with potential distortion or out-of-focus areas in the peripheral areas, thereby creating a highly reliable photogrammetry point cloud.

[0019] Figure 2(b) is a photograph showing a marker placed at coordinate (0,0) on glass table 2. The method for determining the shooting range will be explained using the second camera, 4-2. First, the marker at coordinate (0,0) is photographed using the second camera 4-2. The arm 5 is moved to align the marker with the four corners A, B, C, and D of the effective shooting range E shown in Figure 2(a). Figure 2(c) shows the image when the corner A of the effective shooting range E of the second camera 4-2 overlaps with the marker.

[0020] Figure 3 is a graph showing the position coordinates of the arm when the four corners A, B, C, and D of the effective shooting range overlap with the markers. Figure 3(a) is from the second camera, Figure 3(b) is from the first camera, and Figure 3(c) is from the third camera. Point A in Figure 3(a) represents the position coordinates (250,230) of arm 5 when the corner A of the effective shooting range E of the second camera 4-2, as shown in Figure 2(c), is superimposed on the marker. Point B in Figure 3(a) represents the position coordinates (50,45) of arm 5 when the corner B of the effective shooting range E of the second camera 4-2, as shown in Figure 2(c), is superimposed on the marker. Point C in Figure 3(a) represents the position coordinates (-60, 200) of arm 5 when the corner C of the effective shooting range E of the second camera 4-2, as shown in Figure 2(c), is superimposed on the marker. Point D in Figure 3(a) represents the position coordinates (40,320) of arm 5 when the corner D of the effective shooting range E of the second camera 4-2, as shown in Figure 2(c), is superimposed on the marker.

[0021] In other words, if the coordinates of arm 5 are within the rectangle formed by connecting the four points A, B, C, and D at the four corners shown in Figure 3(a), then the marker placed at coordinate (0,0) will always be captured within the effective shooting range E of the second camera 4-2. Similarly, if the coordinates of arm 5 are within the rectangle formed by connecting the four points A, B, C, and D at the four corners shown in Figure 3(b), then the marker placed at coordinate (0,0) will always be captured within the effective shooting range E of the first camera 4-1. Furthermore, if the coordinates of arm 5 are within the rectangle formed by connecting the four points A, B, C, and D shown in Figure 3(c), the marker placed at coordinate (0,0) will always be captured within the effective shooting range E of the third camera 4-3. In this way, for each of the six cameras 4, the coordinates of the arm 5 that captures the marker placed at coordinate (0,0) within the effective shooting range E of each camera can be determined.

[0022] Figure 4 is an explanatory diagram of the shooting points for the second camera. Figure 4(a) shows the object placement area, and Figure 4(b) is a graph showing the effective shooting range that the second camera can capture within the object placement area shown in Figure 4(a). Using the second camera 4-2, the position of the arm 5 that can capture the coordinates (0,0) of the object placement area 10 shown in Figure 4(a) is as shown in Figure 3(a), and is the range G shown in Figure 4(b). When using the second camera 4-2 to photograph the coordinate (200,0) of the object placement area 10 shown in Figure 4(a), moving the arm 5 200 units in the X-axis direction will change the area G shown in Figure 4(b) to area Gx, making it possible to photograph with the second camera 4-2. When using the second camera 4-2 to photograph the coordinates (0,200) of the object placement area 10 shown in Figure 4(a), moving the arm 5 200 units in the Y-axis direction will change the area G shown in Figure 4(b) to area Gy, making it possible to photograph with the second camera 4-2.

[0023] However, since arm 5 cannot move beyond its movable range (850 in the X-axis direction and 350 in the Y-axis direction), the filled-in area in Figure 4(b) represents the shooting range F of the second camera 4-2.

[0024] Figure 4(c) shows the shooting arm trajectory and shooting points of arm 5 within the shooting range of the second camera 4-2. When the shooting interval is set to 50 mm, the shooting points are indicated by the black dots. Then, as indicated by the arrows, arm 5 moves in a way that turns around at the shooting points, and takes pictures while temporarily stopping at the shooting points. Also, Figure 4(c) shows the position and shooting direction of the second camera 4-2 at the shooting points indicated by the circles.

[0025] In this way, for each camera 4 attached to the arm 5, an effective shooting range E is set in advance for a specific coordinate, the movement range of the arm 5 that includes the object placement range 10 within the effective shooting range E is calculated, and it is decided that the camera 4 will take pictures within the calculated movement range of the arm 5. Therefore, since the camera 4 does not take pictures outside the calculated movement range of the arm 5, it is possible to prevent a decrease in quality when creating point clouds from photographs that do not adequately capture the object 1, and to create highly reliable photogrammetry point clouds.

[0026] Figure 5 is a flowchart of a measurement system using laser-measured point clouds and photometric point clouds in one embodiment of the present invention. Data is input to the measurement system according to this embodiment (S10). In S10, data input includes the object placement range 10, the camera 4's photo-taking interval, and the laser device 3's measurement pitch. When data is input in S10, the control device 6a determines the trajectory of the shooting arm 5 that takes pictures with the camera 4 and the trajectory of the measurement arm 5 that takes measurements with the laser device 3, based on the object placement range 10 (S11). Then, the control device 6a issues instructions for shooting operations based on the shooting arm trajectory determined in S11 (S12). In response to the shooting operation instruction in S12, arm 5 moves according to the shooting arm trajectory and takes a picture with camera 4 at a predetermined position (S13). Note that the shooting position differs for each camera 4, so the camera 4 corresponding to the shooting position takes the picture. In S13, the photographic data captured by camera 4 is stored (S14).

[0027] Until all shooting is complete (No in S15), the shooting operation instruction is given in S12. When all imaging is completed (Yes in S15), the control device 6a generates an imaging completion message (S16) and issues a laser measurement operation instruction based on the measurement arm trajectory determined in S11 (S17). In response to the laser measurement operation instruction in S17, arm 5 moves along the measurement arm trajectory, and the laser device 3 performs measurement at the input measurement pitch (S18). In S18, the measurement data measured by the laser device 3 is stored (S19).

[0028] The control device 6b monitors the creation of the shooting completion message in S16 (No in S20). When the image capture completion message is created in S16 (Yes in S20), the control device 6b starts the photogrammetry point cloud creation software (S21). In S21, when the photogrammetry point cloud creation software is launched, the photo data stored in S14 is imported and a photogrammetry point cloud is created (S22). It is preferable to denoise the photogrammetry point cloud created in S22 (S23). The acquired photogrammetry point cloud contains noise caused by the glass table 2 and shadows, but this noise has very low reliability. Therefore, by filtering out the points with low reliability, the noise in the photogrammetry point cloud can be roughly removed. For example, using the height data from the photogrammetry point cloud, it is possible to remove noise from the space on the glass table 2 where the object to be measured 1 does not exist. Although Figure 1 illustrates the case where the camera 4 is placed on the surface side of the glass table 2 together with the laser device 3, it is even more preferable to place multiple cameras 4 on the underside of the glass table 2 and use photographic data from the underside of the glass table 2. In this manner, when multiple cameras 4 are placed on the underside of the glass table 2 to acquire photographic data from the underside of the glass table 2, in S22, the photometric point clouds for the surface and the underside are created separately, and the photometric point clouds for the surface and the underside are combined into a single photometric point cloud using the chunk merging function (S24). It is preferable to perform noise reduction on the photometric point cloud combined in S24.

[0029] The photometric point clouds combined in S24, or the photometric point clouds of the surface, and the laser measurement point clouds stored in S19 are aligned using the SICP algorithm (S25), and the measurement point cloud is completed. Although Figure 5 describes control devices 6a and 6b as separate devices, it is also possible to perform the processing with a single control device 6.

[0030] Figure 6 is a photograph showing a measurement system with cameras positioned on both the front and back surfaces of a glass table. As shown in Figure 6, the measurement system using laser measurement point clouds and photometric point clouds in this embodiment comprises a glass table 2 on which the object to be measured 1 is placed, a laser device 3 that acquires a laser measurement point cloud from the object to be measured 1, a plurality of cameras 4F, 4B that photograph the object to be measured 1 to obtain a photometric point cloud, an arm 5 to which the laser device 3 and cameras 4F, 4B are attached, and a control device (not shown) that controls the operation of the laser device 3, cameras 4F, 4B, and arm 5. The laser device 3 and camera 4F are positioned on the front side of the glass table 2 by the arm 5, and camera 4B is positioned on the back side of the glass table 2 by the arm 5. Arm 5 moves parallel to the glass table 2 in the X-axis and Y-axis directions. In this way, by arranging the laser device 3 and camera 4F on the front side of the glass table 2 and camera 4B on the back side of the glass table 2, it is possible to obtain both a photograph for obtaining a photometric point cloud and a laser-measured point cloud from the object 1 placed on the glass table 2. As shown in Figure 6, photogrammetry markers 7 are placed around the object to be measured 1. When performing photogrammetry using cameras 4F and 4B, arranging photogrammetry markers 7 around the object to be measured 1 can improve the accuracy of the point cloud created by photogrammetry.

[0031] Figure 7 illustrates noise reduction using photogrammetry markers; Figure 7(a) shows the arrangement of the photogrammetry markers, and Figure 7(b) is a photograph showing noise reduction using photogrammetry markers. As shown in Figure 7(a), a photometric marker 7 for setting the object placement range (measurement range) 10 is placed on the glass table 2. Four photometric markers 7a, 7b, 7c, and 7d are placed at the four corners of the measurement range 10, including the origin, on the glass table 2, and the photometric marker 7e is placed on the y-axis of the glass table 2 at a predetermined distance from the origin (for example, 10 cm away). Furthermore, for photometric measurement from the back, different photometric markers 7f, 7g, 7h, 7i, and 7j are arranged (for example, printed) in sequence on the back of each of the photometric markers 7a, 7b, 7c, 7d, and 7e.

[0032] The photometric marker 7 has three functions. Firstly, it is to improve the accuracy of photogrammetry. The captured photographs are processed using photogrammetry point cloud creation software. At this processing stage, the photographs need to be aligned, but by limiting the measurement range through the placement of photogrammetry markers 7, the number of feature points acquired from the images increases, making it easier to match between photographs and improving the accuracy of the alignment.

[0033] Secondly, it can remove noise other than that from the object being measured (object 1). In photogrammetry, if objects other than those in the vicinity of the object being measured are captured in the photograph, extraneous point clouds outside the vicinity of the object being measured may also be acquired. Therefore, by limiting the measurement range to only the point clouds within the range of the photogrammetry marker 7, noise outside the range of the photogrammetry marker 7, i.e., outside the vicinity of the object being measured, can be removed. As shown in Figure 7(b), noise is removed by the photometric marker 7. Furthermore, the photogrammetry point cloud creation software can calculate the reliability of the points. The acquired photogrammetry point cloud contains noise caused by the glass table 2 and shadows, but the reliability of the noise is very low. Therefore, by filtering out points with low reliability, noise near the object 1 in the photogrammetry point cloud can be removed. In the case of laser-measured point clouds, the point clouds of the two glass table surfaces are also acquired. Therefore, by removing the points corresponding to the coordinates of the two glass table surfaces, most of the noise can be automatically removed. However, for both laser-measured and photometric point clouds, any fine noise that cannot be automatically removed must be removed manually.

[0034] Thirdly, by using the photogrammetry marker 7 for coordinate management, the initial positioning can be automatically determined when performing chunk compositing or point cloud compositing. Chunk stacking (combining front and back images of photogrammetry point clouds) A photometric point cloud is acquired using photometric point cloud creation software. The photometric point cloud creation software has a chunk merging function, which allows point clouds to be merged when the position and scale of the chunks match. By using the photometric marker 7, the position and scale of the chunks can be matched and merged. The chunks contain position information, depth maps, and point cloud information for cameras 4F and 4B. In this embodiment, the surface of the object to be measured is measured by camera 4F and the back surface of the object to be measured by camera 4B. Therefore, the photometric point cloud creation software creates separate chunks for the front and back surfaces, and the point clouds of the front and back surfaces of the object to be measured can be merged into a single photometric point cloud using the chunk merging function. The coordinates of the point cloud created by the photometric point cloud creation software differ from the actual dimensions of the object being measured 1. Therefore, by ensuring that all photometric markers 7 are positioned on the two surfaces of the glass table, and by using the coordinates of the origin photometric marker 7 and photometric markers 7e and 7j, which are located on the y-axis and 10 cm away from the origin, the coordinate axes of the created point cloud can be aligned with the positions of the photometric markers 7. This makes it possible to align the coordinate axes of the photometric point cloud on the front side of the glass table 2, the photometric point cloud on the back side of the glass table 2, and the laser measurement point cloud.

[0035] Figure 8 is an overview of the point cloud synthesis procedure in S25. Figure 8(a) is an illustrative diagram showing the difference between a laser-measured point cloud obtained using a laser device and a photometric point cloud obtained using photographic data. The laser-measured point cloud represents actual size, while the photometric point cloud represents relative shape. As shown in Figure 8(b), the initial sizes of the two point clouds before applying the SICP algorithm are set so that the photometric point cloud is approximately half the size of the laser-measured point cloud. Then, as shown in Figure 8(c), the direction of the point cloud is determined, and the position is adjusted so that the centroids of each point cloud coincide. Here, the centroid O of the point cloud is calculated as the average of the x, y, and z values ​​of all points (c1), and principal component analysis is applied to the point cloud to obtain the three directions (x, y, and z axis directions) (c2). Then, (c1) and (c2) are applied to each point cloud to perform a coordinate transformation so that the origin O-xyz axis of each point cloud coincides. Finally, as shown in Figure 8(d), the two point clouds are aligned using the SICP algorithm.

[0036] As described above, after the camera capture is complete, the arm 5 is moved according to the measurement arm trajectory to perform measurements with the laser device 3. This allows for the creation of a photometric point cloud using photographic data while the laser device 3 is performing measurements, thereby shortening the total time required to create the measurement point cloud. Measurement using the laser device 3 and the creation of the photometric point cloud take time, but by creating the photometric point cloud using photographic data while the measurement is being performed by the laser device 3, the total time until the point cloud is created can be shortened.

[0037] <Verification> Figures 9 and 10 are explanatory diagrams illustrating the verification of the present invention. As shown in Figure 9(a), oyster shells were used as the object to be measured 1. The experimental environment consisted of Windows 10 Pro as the OS and an Intel Core® CPU. TM It has an i7-5820k processor and 64.0GB of RAM. The oyster shell measures 133.19 mm x 280.80 mm. The shooting interval for photogrammetry was set to 50 mm. The number of photos taken was 24 from the first camera 4-1, 45 from the second camera 4-2, 38 from the third camera 4-3, 28 from the fourth camera 4-4, 52 from the fifth camera 4-5, and 47 from the sixth camera 4-6, for a total of 234 photos. The shooting time was 16 minutes and 59 seconds. Furthermore, 233 of the 234 obtained photographs were aligned using photogrammetry point cloud creation software. The time taken for point cloud creation software was 24 minutes and 41 seconds, and the resulting photogrammetry point cloud is shown in Figure 9(b).

[0038] The laser measurement pitch was set to 0.24 mm, and the laser measurement time was 1 hour, 16 minutes, and 43 seconds. Figure 10(a) shows the laser measurement point cloud with noise removed, containing 177,424 points. Additionally, 60 photographs were taken of the underside of the oyster shells using a digital camera, and a point cloud was created using photogrammetry point cloud creation software. This point cloud was then chunk-combined with the photogrammetry point cloud of the upper side of the oyster shells, which was obtained automatically, to create a photogrammetry point cloud that included both sides. Photographing the underside of the oyster shells and creating the point cloud took approximately 30 minutes, while chunk combining took approximately 40 minutes. Figure 10(b) shows the photogrammetry point cloud after chunk combining and noise reduction. The number of points is 583,017. Figure 10(c) shows the results of aligning laser-measured point clouds and photometric point clouds using the SICP algorithm. The red point clouds are laser-measured point clouds, and the colored point clouds are photometric point clouds. The threshold for the SICP algorithm was set to 0.001, and the processing time for point cloud merging was 22 minutes and 50 seconds. As shown in Figure 10(c), we were able to obtain a point cloud that is both actual size and contains color information. [Industrial applicability]

[0039] According to the present invention, laser measurement and photogrammetry can be performed automatically without human intervention during the measurement process, and a point cloud that is accurate in size and free of defects can be efficiently acquired. [Explanation of Symbols]

[0040] 1. Object to be measured 2 Glass Table 3. Laser device Cameras 4, 4F, 4B 4-1 First Camera 4-2 Second Camera 4-3 Third Camera 4-4 The fourth camera 4-5 The fifth camera 4-6 The sixth camera 5 Arms 6, 6a, 6b Control devices 7, 7a, 7b, 7c, 7d, 7e, 7f, 7g, 7h, 7i, 7j photo measurement markers 10. Object placement range (measurement range) A Target location E Effective shooting range F Shooting range G, Gx, Gy range θ1 First shooting angle θ2 Second shooting angle

Claims

1. A glass table on which the object to be measured is placed, A laser device that acquires a laser measurement point cloud from the object to be measured, Multiple cameras for photographing the object to be measured in order to obtain a photogrammetry point cloud, The laser device and the arm to which the camera is attached, The laser device, the camera, and the control device for controlling the movement of the arm Equipped with, The arm positions the laser device and the camera on the surface side of the glass table. If the first camera's shooting direction relative to the glass table is defined as the first shooting angle, and the second camera's shooting direction relative to the glass table is defined as the second shooting angle, then the first shooting angle is made larger than the second shooting angle. The first camera is positioned further away from the glass table than the second camera. In the aforementioned control device, The trajectory of the camera arm that performs photography and the trajectory of the laser arm that performs measurement are determined based on the object placement area of ​​the object to be measured. The arm is moved according to the trajectory of the shooting arm to perform the initial shooting with the camera. After the aforementioned camera has finished taking the image, the arm is moved according to the measurement arm trajectory to perform the measurement using the laser device. A measurement system using laser-measured point clouds and photographic point clouds, characterized by the above.

2. During the movement of the arm along the measurement arm trajectory, the photometric point cloud is created using photographic data of the object to be measured. A measurement system using a laser-measured point cloud and a photo-measured point cloud as described in claim 1.

3. With each of the cameras attached to the arm, The effective shooting range is pre-set for a specific coordinate system. The range of movement of the arm is calculated such that the object placement area falls within the effective shooting range. Within the calculated range of movement of the arm, the camera takes photographs. Outside the calculated range of motion of the arm, the camera will not take photographs. A measurement system using a laser-measured point cloud and a photo-measured point cloud as described in claim 1.

4. The effective shooting range is set to the central part of the shooting range that can be captured by the camera, excluding the peripheral area. A measurement system using a laser-measured point cloud and a photo-measured point cloud as described in claim 3.

5. Multiple objects to be measured are placed on the glass table. The range including multiple objects to be measured is defined as the object placement range. After creating the aforementioned photometric point cloud, the height data of the photometric point cloud is used to remove noise from the space on the glass table where the object to be measured does not exist. A measurement system using a laser-measured point cloud and a photometric point cloud as described in feature 2.

Images