Calibration method for stereo measurement device

Abstract

To provide a calibration method that makes it possible to calibrate with high accuracy in a stereo measurement device using a line sensor camera.SOLUTION: A calibration method includes: a first adjustment step of adjusting positions in an X axis direction of line sensor cameras 2A and 2B and rotation angles around a Y axis and Z axis thereof respectively; a second adjustment step of adjusting a rotation angle around the X axis of the line sensor cameras 2A and 2B; a first computation step of computing a distance in the Z axis direction between the line sensor cameras 2A and 2B and a calibration piece; and a second computation step of computing a distance in the Y axis between a cross point where a straight line extending in the Z axis direction from an image center of the line sensor cameras 2A and 2B crosses the calibration piece and a reference coordinate in the Y axis direction provided in the calibration piece.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to a calibration method for a stereo measurement device.

Background Art

[0002] A stereo measurement device that measures the distance to an object based on images of the object taken by a pair of cameras is known. In a stereo measurement device, it is necessary to perform a calibration operation for specifying the positional relationship between a pair of cameras using a calibration target in advance. When a line sensor camera is used as the camera of the stereo measurement device, it is difficult to make each line sensor camera capture the same location of the calibration target, so various contrivances are required for the calibration operation. For example, Patent Document 1 discloses a calibration method in which a cylinder with a checkerboard attached is photographed while rotating around an axis, and the position and orientation of a pair of line sensor cameras are adjusted using the photographed images.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In the calibration method of Patent Document 1, the manufacturing accuracy and rotation accuracy of the calibration target itself are reflected in the photographed image, and the calibration accuracy of the line sensor camera is reduced. For this reason, there is a problem that the line sensor camera cannot be sufficiently calibrated and the distance measurement to the object cannot be performed with high accuracy.

[0005] The present invention has been made based on such a background, and an object thereof is to provide a calibration method capable of performing calibration with high accuracy in a stereo measurement device using a line sensor camera.

Means for Solving the Problems

[0006] To achieve the above objectives, this Clearly The calibration method is, A calibration method for calibrating the external parameters of a line sensor camera in a stereo measuring device comprising a pair of line sensor cameras and a line light source positioned side by side between the pair of line sensor cameras to form a plane of light in space, wherein the device comprises a pair of line sensor cameras and a line light source positioned between the pair of line sensor cameras to form a plane of light in space, In a Cartesian coordinate system where the direction of light emission from the line light source is the Y-axis direction, the direction of light emission is the Z-axis direction, and the direction perpendicular to the Y-axis and Z-axis is the X-axis direction, a first adjustment step is to adjust the position of the line sensor camera in the X-axis direction and the rotation angles around the Y-axis and Z-axis, respectively, so that the plane of the imaging range formed by the line sensor camera coincides with the plane of light formed by the line light source. A second adjustment step involves adjusting the rotation angle of the line sensor camera around the X-axis so that the optical axis of the line sensor camera, whose position and rotation angle have been adjusted by the first adjustment step, extends in the Z-axis direction. A first calculation step calculates the distance in the Z-axis direction between the line sensor camera and the calibration piece based on the image captured by the line sensor camera whose rotation angle has been adjusted by the second adjustment step and the focal length of the lens of the line sensor camera, A second calculation step calculates the distance in the Y-axis direction between a line extending in the Z-axis direction from the center of the image of the line sensor camera and the calibration piece, based on the distance in the Z-axis direction between the line sensor camera and the calibration piece, and the Y-axis direction reference coordinate provided on the calibration piece, based on the distance in the Z-axis direction between the line sensor camera and the calibration piece, calculated in the first calculation step, Includes fruit, The calibration piece is a slitted calibration piece having multiple slits arranged at equal intervals in the longitudinal direction of the substrate and capable of reflecting light, In the second adjustment step described above, The calibration piece, illuminated by light from the line light source, is photographed by the line sensor camera. The orientation of the line sensor camera is adjusted so that the spacing between the brightness peaks corresponding to the slits in the image captured by the line sensor camera becomes uniform. .

[0007] In the first adjustment step described above, The screen illuminated by light from the line light source is photographed by the line sensor camera. The position and orientation of the line sensor camera may be adjusted so that the brightness is uniform in each pixel of the image captured by the line sensor camera.

[0008] In the first adjustment step, if the difference between the maximum and minimum brightness values ​​for all pixels in the image captured by the line sensor camera is less than or equal to a threshold, it may be determined that the brightness is uniform for each pixel in the image captured by the line sensor camera.

[0009] In the first adjustment step, the screen may be placed at two different positions in the Z-axis direction, and the position and orientation of the line sensor camera may be adjusted so that the brightness is uniform in each pixel of the image captured by the line sensor camera at each position.

[0012] In the first calculation step, the distance in the Z-axis direction between the line sensor camera and the calibration piece may be calculated based on the spacing of the slits in the calibration piece, the spacing of the brightness peaks corresponding to the slits in the image captured by the line sensor camera, and the focal length of the lens of the line sensor camera.

[0013] The aforementioned slitted calibration piece is provided at the reference coordinates and includes a marking that is detectable on the image, In the second calculation step described above, the distance in the Y-axis direction between the intersection point where a straight line extending in the Z-axis direction from the image center of the line sensor camera intersects the calibration piece and the reference coordinate in the Y-axis direction provided on the calibration piece may be calculated based on the distance between the reference coordinate marking obtained from the image of the line sensor camera and the image center, the distance in the Z-axis direction between the line sensor camera and the calibration piece, and the focal length of the lens of the line sensor camera. [Effects of the Invention]

[0015] According to the present invention, it is possible to provide a calibration method capable of performing highly accurate calibration in a stereo measurement device using a line sensor camera.

Brief Description of the Drawings

[0016] [Figure 1] It is a schematic diagram showing the configuration of a stereo measurement device according to an embodiment of the present invention. [Figure 2] It is a side view showing the configuration of a stereo measurement device according to an embodiment of the present invention. [Figure 3] It is a flowchart showing the procedure of calibration work according to an embodiment of the present invention. [Figure 4] It is a flowchart showing the procedure of calibration work for internal parameters according to an embodiment of the present invention. [Figure 5] It is a front view showing a checkerboard used in Zhang's method. [Figure 6] It is a flowchart showing the procedure of calibration work for external parameters according to an embodiment of the present invention. [Figure 7] It is a schematic diagram showing the state of calibration work using a screen in a stereo measurement device according to an embodiment of the present invention. [Figure 8] (a) is a graph showing the distribution of luminance values before adjustment, and (b) is a graph showing the distribution of luminance values after adjustment. [Figure 9] It is a schematic diagram showing the state of calibration work using a calibration piece with slits in a stereo measurement device according to an embodiment of the present invention. [Figure 10] (a) is a graph showing the distribution of slits before adjustment, and (b) is a graph showing the distribution of slits after adjustment in a line sensor camera. [Figure 11] It is a schematic diagram showing the relationship between the distance from the reference coordinate to the pixel center in the calibration piece with slits and the distance from the reference coordinate to the pixel center in the line sensor. [Figure 12] It is a block diagram showing the hardware configuration of a processing unit according to an embodiment of the present invention. [Figure 13]This flowchart shows the flow of the calibration support process according to an embodiment of the present invention. [Modes for carrying out the invention]

[0017] The calibration method for a stereo measuring device according to an embodiment of the present invention will be described in detail below with reference to the drawings. In each drawing, the same or equivalent parts are denoted by the same reference numerals.

[0018] The stereo measurement device according to this embodiment is a measurement device that uses a pair of line sensor cameras to image the same location on an object and calculates the distance to the same location on the object based on the parallax caused by the difference in the imaging positions. As shown in Figure 1, the stereo measurement device 1 comprises a pair of line sensor cameras 2A and 2B arranged at a distance from each other, a line laser light source 3 positioned between the pair of line sensor cameras 2A and 2B, and a processing unit 100 that is communicatively connected to the pair of line sensor cameras 2A and 2B and the line laser light source 3 and controls the operation of the pair of line sensor cameras 2A and 2B and the line laser light source 3.

[0019] Each line sensor camera 2A and 2B captures an object as a one-dimensional image and transmits brightness (pixel value) data distributed in one direction in the image to the processing unit 100. Each line sensor camera 2A and 2B includes a lens 21 and a line sensor 22 into which light entering from the lens 21 is imaged. The line sensor 22 is an image sensor in which a large number of pixels are arranged in a row as a pixel column, and the light received by each pixel is converted into an electrical signal and output. Each line sensor camera 2A and 2B forms a planar imaging range in space that includes the optical axis and acquires an image of the surface portion of the object that intersects the plane of the imaging range.

[0020] The line laser light source 3 is an example of a line light source that forms a plane of light (laser plane) in space and draws a straight line extending on the plane of an object when light is shone on the plane of that object. The line laser light source 3 is placed side by side with a pair of line sensor cameras 2A and 2B, and emits laser light to form a fan-shaped laser plane, which is shone onto the surface of the object. The line drawn on the plane of the object by the line laser light source 3 is used as a reference line during calibration of the line sensor cameras 2A and 2B, and also as a guide when setting up the line sensor cameras 2A and 2B. With only the line sensor cameras 2A and 2B, it is difficult to determine the shooting position, but by using the line laser light source 3 as a guide, it is possible to easily determine the shooting position.

[0021] In the following embodiments, two coordinate systems are used: a reference coordinate system and a camera coordinate system. The reference coordinate system is an orthogonal coordinate system composed of mutually orthogonal X, Y, and Z axes. The direction of laser beam propagation is the Y-axis direction, the direction of laser beam emission is the Z-axis direction, and the directions perpendicular to the Y and Z axes are the X-axis directions. The camera reference system is an orthogonal coordinate system set based on each line sensor camera 2A and 2B, composed of mutually orthogonal X1, Y1, and Z1 axes. The direction in which the optical axes of each line sensor camera 2A and 2B extend is the Z1 axis direction, the direction in which each line sensor camera 2A and 2B and the line laser light source 3 are aligned is the Y1 axis direction, and the directions perpendicular to the Y1 and Z1 axes are the X1 axis direction.

[0022] As shown in Figure 2, the line sensor cameras 2A and 2B and the line laser light source 3 are each supported on the housing 4 so as to be able to adjust their position and orientation. The housing 4 is positioned above the moving stage 5 and is supported by a stand 6. The moving stage 5 is configured to move in two directions (the X-axis and Z-axis directions) perpendicular to the direction in which the line sensor 22 extends, with an object placed on it. The stereo measurement device 1 captures images of the object with the line sensor cameras 2A and 2B while the moving stage 5 is moving in the X-axis direction, and sequentially calculates the distance to the object based on each captured image.

[0023] Returning to Figure 1, the processing unit 100 is, for example, a general-purpose computer. The processing unit 100 calculates the distance to the same point on the object based on the brightness data at the same point on the object acquired by the line sensor cameras 2A and 2B. Specifically, it captures the same point on the object with a pair of line sensor cameras 2A and 2B, estimates the parallax by matching each point in each captured image based on brightness, and calculates the distance to each point on the object based on the estimated parallax. The processing unit 100 also functions as a calibration device that performs processing to support the calibration work of the line sensor cameras 2A and 2B. The above describes the configuration of the stereo measurement device 1.

[0024] In the stereo measurement device 1, calibration is required for each of the line sensor cameras 2A and 2B in order to determine their relative positions. Calibration includes both initial calibration before operation and periodic calibration during operation. Calibration of each line sensor camera 2A and 2B is performed for both internal and external parameters.

[0025] Internal parameters are parameters that indicate the relationship between the three-dimensional camera coordinates and the one-dimensional image coordinates set for the image acquired by the camera, and specifically include parameters related to the focal length, lens distortion, and image center of line sensor cameras 2A and 2B. External parameters are parameters that indicate the relationship between the reference coordinate system and the camera coordinate system, and specifically include parameters related to the position and orientation of line sensor cameras 2A and 2B in the reference coordinate system.

[0026] In the stereo measurement device 1, since the line sensor cameras 2A and 2B have a large number of pixels arranged on a single straight line, a higher resolution 2D image can be generated than that of an area camera with the same number of pixels by arranging the brightness data acquired over time while moving the line sensor cameras 2A and 2B. For this reason, the distance to the object can be calculated with higher accuracy compared to a stereo measurement device using an area camera. However, unlike an area camera, it is difficult for line sensor cameras 2A and 2B to photograph the same point on the object, so calibration is performed using the following procedure.

[0027] Next, referring to Figure 3, the procedure for calibration work performed by the user on the line sensor cameras 2A and 2B of the stereo measurement device 1 according to the embodiment will be described. Since the calibration of line sensor cameras 2A and 2B is performed using the same procedure, the following explanation will use the calibration of line sensor camera 2A as an example.

[0028] First, the internal parameters of the line sensor camera 2A are calibrated (step S1). For internal parameter calibration, for example, the Zhang method is used. The Zhang method is a technique in which a sample with a pattern that allows estimation of how it would look without distortion is drawn is photographed with the camera, and the internal parameters of the camera are calculated based on the captured image.

[0029] In this process, the focal length and lens distortion parameters are calculated as internal parameters. The other internal parameter, the image center parameter, is determined by the longitudinal position of the line sensor 22 and therefore cannot be calculated in this process; it is calculated at the same time as the calibration of the external parameters of the line sensor camera 2A. The flow of the calibration work for the internal parameters of the line sensor camera 2A according to this embodiment will be explained below with reference to Figure 4.

[0030] First, the line sensor 22 is removed from the lens 21 and replaced with an area sensor having a complementary shape (step S11). This is because the line sensor 22 cannot be used to acquire a two-dimensional image of the checkerboard, as described later. The mounting portion of the line sensor 22 on the lens 21 is standardized to allow the attachment of different types of sensors, so the line sensor 22 can be easily replaced with an area sensor of the same standard.

[0031] Next, the area sensor installed in step S11 captures an image of the checkerboard, and the focal length and lens distortion parameters of the line sensor camera 2A are calculated based on the captured image of the checkerboard (step S12).

[0032] In this process, a checkerboard, as shown in Figure 5, is imaged as an example of a sample pattern that allows for estimation of how it would look without distortion. The processing unit 100 is then used to optimize the focal length and lens distortion parameters so that the vertices of the black and white squares on the checkerboard are aligned in a straight line. Specifically, the real-space positions of the vertices of the squares on the checkerboard can be calculated in advance based on the regularity of the checkerboard, and the positions of these vertices on the captured image can be detected by applying image processing to the captured image. Next, the lens distortion parameters are calculated based on the relationship between the real-space positions and the image positions.

[0033] Furthermore, in Zhang's method, the focal length is calculated as a dimensionless quantity normalized by the pixel size. Based on the ratio of the pixel sizes in the line sensor 22 and the area sensor, the focal length parameter value calculated based on the image captured by the area sensor can be corrected to the focal length parameter value when the line sensor 22 is attached.

[0034] Next, the area sensor is removed from the lens 21, and the line sensor 22 is attached to the lens 21 (step S13). The above outlines the procedure for calibrating internal parameters.

[0035] Returning to Figure 3, the external parameters of the line sensor camera 2A are calibrated (step S2). The external parameters are six parameters that indicate the position (x1, y1, z1) of the line sensor camera 2A in the reference coordinate system and the rotation angles (θx1, θy1, θz1) around the X, Y, and Z axes. The following describes the procedure for calibrating the external parameters of the line sensor camera 2A according to the embodiment, with reference to Figure 6.

[0036] First, a first adjustment step is performed (step S21) to adjust the position of the line sensor camera 2A in the X-axis direction and the rotation angles around the Y-axis and Z-axis, respectively, so that the position x1=0 and the rotation angles θy1,θz1=0. Specifically, as shown in Figure 7, a planar screen is placed in front of the line sensor camera 2A and the line laser light source 3. The screen is an example of an adjustment target, and is, for example, a plate-shaped member having a flat surface that can reflect laser light over its entire surface. The screen is installed in an orientation parallel to the direction in which the line sensor camera 2A and the line laser light source 3 are arranged. Next, laser light is shone toward the screen, and the position and orientation of the line sensor camera 2A are tentatively determined so that the lines drawn on the screen by the line laser light source 3 and the pixel rows of the line sensor camera 2A are as parallel as possible.

[0037] Next, the position and orientation of the line sensor camera 2A are adjusted so that the brightness of the reflected light from the screen is uniform at each pixel of the line sensor camera 2A. More specifically, if the brightness of the reflected light cannot be detected at each pixel of the line sensor camera 2A, the line sensor camera 2A is either shifted in the X-axis direction relative to the reference laser plane or rotated around the Y-axis. Also, if the brightness of the reflected light is not uniform at each pixel of the line sensor camera 2A, the line sensor camera 2A is rotated around the Z-axis relative to the laser plane, and the line drawn on the screen by the line laser light source 3 intersects with the pixel row of the line sensor camera 2A. The user adjusts the position and angle of the line sensor camera 2A so that the brightness of the reflected light from the screen is uniform at each pixel of the line sensor camera 2A.

[0038] Whether the luminance of the reflected light from the screen is uniform for each pixel of the line sensor camera 2A is determined, for example, from the maximum value and the minimum value of the luminance. Specifically, when the difference ΔL between the maximum value and the minimum value of the luminance is within the threshold value Lth, it is determined that the luminance of the reflected light from the screen is uniform for each pixel. For example, in the graph of FIG. 8(a), since ΔL≧Lth, it can be determined that the distribution of the luminance acquired by the line sensor camera 2A is not uniform. On the other hand, in the graph of FIG. 8(b), since ΔL<Lth, it can be determined that the distribution of the luminance acquired by the line sensor camera 2A is uniform.

[0039] Next, the screen is moved to different positions in the Z-axis direction, and similarly, it is confirmed whether the luminance of the reflected light from the screen is uniform for each pixel of the line sensor camera 2A. This is because even when θy1≠0, depending on the emission direction of the laser light, the luminance at each pixel of the captured image may accidentally become uniform. In this way, the position and orientation of the line sensor camera 2A are changed, and when the luminance becomes uniform at two positions in the Z-axis direction, the position of the line sensor camera 2A in the X-axis direction is set to x1 = 0, and the rotation angles in the Y-axis direction and the Z-axis direction are set to θy1, θz1 = 0, respectively. As a result, the plane of the imaging range formed by the line sensor camera 2A can be adjusted to overlap with the plane of the light formed by the line laser light source 3, and as a result, all the lines drawn on the screen by the line laser light source 3 can be detected within the range of the pixel column of the line sensor camera 2A.

[0040] Returning to FIG. 6, a second adjustment step of adjusting the rotation angle of the line sensor camera 2A around the X-axis so that the rotation θx1 = 0 is performed (step S22). Specifically, first, the screen used in the step of step S1 is replaced with a calibration piece with a slit, and as shown in FIG. 9, the laser light is irradiated from the line laser light source 3 toward the calibration piece with a slit.

[0041] A slitted calibration piece is an example of an adjustment target and comprises a substrate made of a transparent material and a plurality of slits provided on the main body substrate and made of a material capable of reflecting laser light. Each slit extends in the width direction of the substrate and is arranged at equal intervals in the longitudinal direction of the substrate. The slitted calibration piece is placed in front of the line sensor camera 2A and the line laser light source 3 so that the lines drawn on the screen by the line laser light source 3 are perpendicular to each slit. At this time, the slitted calibration piece is placed in an orientation parallel to the direction in which the line sensor camera 2A and the line laser light source 3 are aligned.

[0042] Next, a calibration piece with slits, illuminated by laser light from the line laser light source 3, is photographed, and the orientation of the line sensor camera 2A is adjusted so that the spacing between the slits indicating the brightness peaks in the captured image is uniform. As shown in Figure 10(a), when θx1≠0, the spacing between the slits indicating the brightness peaks in the image captured by the line sensor 22 gradually changes. On the other hand, as shown in Figure 10(b), when θx1=0, the spacing between the slits indicating the brightness peaks in the image captured by the line sensor 22 is constant. The orientation of the line sensor camera 2A is changed, and the rotation angle of the line sensor camera 2A in the X-axis direction is set to θx1=0 when the spacing between the slits indicating the brightness peaks in the image becomes constant. This allows the optical axis of the line sensor camera 2A to be adjusted to extend in the Z-axis direction.

[0043] Returning to Figure 6, the first calculation step is performed to calculate the distance z1 in the Z-axis direction between the line sensor camera 2A and the slit calibration piece (step S23). Since the slit spacing d in the slit calibration piece is known, the magnification M = d' / d is calculated by reading the interval d' of the brightness peaks from the captured image. Next, the distance z1 is calculated based on the calculated magnification M and focal length f. The distance z1 is expressed by the following equation (1). z1 = f / M …(1)

[0044] Next, a second calculation step is performed (step S24) to calculate the distance y1 in the Y-axis direction between the intersection of a straight line extending in the Z-axis direction from the image center of the line sensor camera 2A and the slit calibration piece, and the reference coordinate y0 of the slit calibration piece. The image center is represented by a one-dimensional coordinate on the pixels of the line sensor 22 and can be calculated based on the number of pixels. The reference coordinate y0 is the reference Y-axis coordinate on the slit calibration piece. The slit calibration piece is provided at the reference coordinate y0 and is equipped with a marking that can be detected on the image. Specifically, the marking is formed so that the slit located at the reference coordinate y0 can be distinguished from other slits on the image. For example, the slit spacing or slit width of the slit located at the reference coordinate y0 is formed to be different from that of other slits. As a result, as shown in Figure 10(b), it is possible to determine on the image which pixel of the line sensor 22 is capturing the laser light reflected from the reference coordinate y0 of the slit calibration piece.

[0045] As shown in Figure 11, if p1 is the distance between the reference coordinate y0 and the center of the image on the image captured by the line sensor 22, then the ratio of distance y1 to distance p1 is equal to the ratio of distance z1 to focal length f. Therefore, distance y1 is expressed by the following equation (2). y1 = z1 * p1 / f …(2) The distance p1 can be calculated using the number of pixels on the image from the center of the image to the reference coordinate y0, as shown in Figure 10(b). Since the distance z1 and focal length f are known from the process described above, the distance y1 can be calculated using equation (2) above.

[0046] Through the above steps, the relative position and rotation relationship between the reference coordinate system XYZ and the camera coordinate system X1Y1Z1 is determined, and the calibration of external parameters is completed. The above outlines the procedure for calibrating external parameters.

[0047] Next, with reference to Figure 12, the hardware configuration of the processing unit 100 according to the embodiment will be described. The processing unit 100 includes an operation unit 110, a display unit 120, a communication unit 130, a storage unit 140, and a control unit 150. Each part of the processing unit 100 is interconnected via an internal bus (not shown).

[0048] The operation unit 110 receives user instructions and supplies operation signals corresponding to the received operations to the control unit 150. The operation unit 110 includes, for example, a mouse and a keyboard.

[0049] The display unit 120 is equipped with a display drive circuit and displays various images to the user based on data supplied from the control unit 150.

[0050] The communication unit 130 is a communication interface for the processing unit 100 to communicate with external devices. The communication unit 130 communicates with external devices, for example, via a communication network or input / output terminals. The input / output terminals are, for example, USB (Universal Serial Bus).

[0051] The memory unit 140 includes, for example, RAM (Random Access Memory), ROM (Read Only Memory), flash memory, and a hard disk. The memory unit 140 stores programs and various data executed by the control unit 150. The memory unit 140 also temporarily stores various data and functions as work memory for the control unit 150 to execute processing.

[0052] The control unit 150 includes a processor and controls each part of the processing unit 100. The processor is, for example, a CPU (Central Processing Unit). The control unit 150 executes the calibration support process shown in Figure 13 by executing a program stored in the storage unit 140. Functionally, the control unit 150 includes an acquisition unit 151, a guide unit 152, a calculation unit 153, and an output unit 154.

[0053] The acquisition unit 151 periodically acquires brightness data for each pixel of the images captured by the line sensor cameras 2A and 2B.

[0054] The guide unit 152 guides the user in adjusting the position and orientation of the line sensor cameras 2A and 2B based on the brightness of each pixel in the image acquired by the acquisition unit 151. Specifically, it determines whether the brightness of each pixel acquired by the acquisition unit 151 is uniform, and if it determines that the brightness is uniform, it instructs the user to stop adjusting the position of the line sensor cameras 2A and 2B in the X-axis direction and the rotation angles around the Y-axis and Z-axis. It also determines whether the intervals between brightness peaks in the image acquired by the acquisition unit 151 are uniform, and if it determines that the intervals between peaks are uniform, it instructs the user to stop adjusting the rotation angles of the line sensor cameras 2A and 2B around the X-axis.

[0055] The calculation unit 153 calculates distances z1 and y1 based on the images captured by line sensor cameras 2A and 2B, whose position and orientation have been adjusted by the guide unit 152, and the focal length f. Distances z1 and y1 are calculated, for example, based on equations (1) and (2) stored in the storage unit 140.

[0056] The output unit 154 displays on the display unit 120 a graph showing the brightness distribution as shown in Figure 8 or a graph showing the interval between brightness peaks as shown in Figure 10, based on the brightness data for each pixel acquired by the acquisition unit 151. The above describes the hardware configuration of the processing unit 100.

[0057] (Calibration support processing) Next, with reference to Figure 13, the flow of the calibration support process executed by the processing unit 100 according to the embodiment will be explained. The calibration support process is a process to assist the user in performing the external parameter calibration work (step S2) of the calibration work of line sensor cameras 2A and 2B, and starts when instructions from the user are received. Since the calibration of line sensor cameras 2A and 2B is performed using the same procedure, the following explanation will use the calibration of line sensor camera 2A as an example.

[0058] First, the user places the screen shown in Figure 7 in front of the line sensor camera 2A and the line laser light source 3. Upon receiving instructions from the user, the acquisition unit 151 operates the line sensor camera 2A and the line laser light source 3 and begins capturing images of lines drawn on the screen by the line laser light source 3 (step S101). At this time, the output unit 154 displays a graph showing the brightness distribution, as shown in Figure 8(a), on the display unit 120. The acquisition unit 151 periodically acquires brightness data for each pixel of the captured image until the calibration support process is completed, and the output unit 154 updates the graph each time the acquisition unit 151 acquires brightness data for the captured image. The user adjusts the position and orientation of the line sensor camera 2A so that the brightness is uniform, as shown in Figure 8(b), while referring to the graph displayed on the display unit 120.

[0059] Next, the guide unit 152 determines whether the brightness distribution acquired by the line sensor camera 2A is uniform (step S102). If it is determined that the brightness distribution is uniform (step S102; Yes), the guide unit 152 instructs the user to move the screen to a different position in the Z-axis direction (step S103). The user moves the screen to a different position in the Z-axis direction and readjusts the position and orientation of the line sensor camera 2A so that the brightness becomes uniform. On the other hand, if it is determined that the brightness distribution is not uniform (step S102; No), the process waits until it is determined that the brightness distribution is uniform.

[0060] Next, the guide unit 152 determines whether the brightness distribution acquired by the line sensor camera 2A is uniform again (step S104). If it is determined that the brightness distribution is uniform (step S104; Yes), the process in step S21 of the external parameter calibration work is completed, and the guide unit 152 instructs the user to replace the screen with the slit calibration piece (step S105). On the other hand, if it is determined that the brightness distribution is not uniform (step S104; No), the process waits until it is determined that the brightness distribution is uniform.

[0061] Upon receiving instructions in step S105, the user replaces the screen shown in Figure 7 with the slitted calibration piece shown in Figure 9 and adjusts the position and orientation of the line sensor camera 2A so that the intervals between brightness peaks are uniform on the image. At this time, the output unit 154 displays a graph showing the intervals between brightness peaks, as shown in Figure 10(a), on the display unit 120, and updates the graph each time the acquisition unit 151 acquires brightness data from the captured image. The user adjusts the orientation of the line sensor camera 2A so that the intervals between brightness peaks are uniform, as shown in Figure 10(b), while referring to the graph displayed on the display unit 120.

[0062] Next, the guide unit 152 determines whether the intervals between the brightness peaks acquired by the line sensor camera 2A are uniform (step S106). If it is determined that the intervals between the brightness peaks are uniform (step S106; Yes), the process in step S22 of the external parameter calibration work is completed, and the user is instructed to complete the adjustment work of the position and orientation of the line sensor camera 2A (step S107). On the other hand, if it is determined that the intervals between the brightness peaks are not uniform (step S106; No), the process waits until it is determined that the brightness slit intervals are uniform.

[0063] After the completion of step S107, the calculation unit 153 calculates distances z1 and y1 based on the captured image acquired by the line sensor camera 2A (step S108). The process in step S108 corresponds to steps S23 and S24 in the external parameter calibration process. Distances z1 and y1 are calculated using equations (1) and (2) stored in the storage unit 140, respectively. After the calculation of distances z1 and y1 is completed, the calculation unit 153 notifies the user that the calibration support process has been completed. The above is the flow of the calibration support process.

[0064] As described above, the calibration method according to the embodiment includes a first adjustment step (step S21) for adjusting the position and rotation angle of line sensor cameras 2A and 2B, a second adjustment step (step S22) for adjusting the rotation angle of line sensor cameras 2A and 2B, a first calculation step (step S23) for calculating the distance in the Z-axis direction between line sensor cameras 2A and 2B and the slit calibration piece, and a second calculation step (step S24) for calculating the distance in the Y-axis direction between the intersection point where a straight line extending in the Z-axis direction from the image center of line sensor cameras 2A and 2B intersects with the slit calibration piece and a reference coordinate in the Y-axis direction provided on the slit calibration piece. Therefore, even with a stereo measurement device 1 equipped with line sensor cameras 2A and 2B, the calibration work of external parameters can be performed with high accuracy.

[0065] The calibration method according to this embodiment allows for the calibration of internal parameters by replacing the line sensors 22 of the line sensor cameras 2A and 2B with area sensors. This allows for the adjustment of internal parameters such as the focal length and lens distortion of the lens 21 in the line sensor cameras 2A and 2B, thereby further improving the calibration accuracy of the stereo measurement device 1.

[0066] The present invention is not limited to the embodiments described above, and the following modifications are also possible.

[0067] (modified version) In the above embodiment, line sensor cameras 2A and 2B were used in which the pixels of the line sensor 22 were arranged in a single row, but the present invention is not limited to this. The line sensor 22 of each line sensor camera 2A and 2B may have multiple rows of pixels.

[0068] In the above embodiment, a line laser light source 3 was used as a line light source to draw a reference line on the plane of the object, but the present invention is not limited to this. The line drawn on the plane of the object by the line light source does not need to be strictly linear, and may have a certain width and fluctuation. The line light source may be something other than a laser light source, for example, it may be configured by aligning multiple LEDs (Light Emitting Diodes) in a straight line.

[0069] In the above embodiment, the line sensor cameras 2A and 2B are separated from each other, so the fields of view they capture are also offset from each other. However, the present invention is not limited to this. For example, tilt-shift lenses (tilt-shift lenses) capable of tilt imaging may be attached to each line sensor camera 2A and 2B, and the inclination of the tilt-shift lenses relative to the line sensor 22 may be adjusted so that the fields of view they capture coincide.

[0070] In the above embodiment, the calibration piece with slits comprised a substrate made of a transparent material and a plurality of slits arranged at equal intervals along the longitudinal direction of the substrate, but the present invention is not limited thereto. The substrate only needs to be made of a material that reflects laser light less than the slits, and may be, for example, a translucent material.

[0071] In the above embodiment, various data were stored in the storage unit 140 of the processing unit 100, but the present invention is not limited thereto. For example, all or part of the various data may be stored in an external control device or computer via a communication network.

[0072] In the above embodiment, each processing unit 100 operated based on a program stored in the storage unit 140, but the present invention is not limited thereto. For example, a functional configuration realized by a program may be realized by hardware.

[0073] In the above embodiment, the processing unit 100 was, for example, a general-purpose computer, but the present invention is not limited thereto. For example, the processing unit 100 may be implemented as a computer located on the cloud.

[0074] In the above embodiment, the processing performed by the processing unit 100 was realized by the device having the above-described physical configuration executing a program stored in the storage unit 140. However, the present invention may be realized as a program, or as a storage medium on which that program is recorded.

[0075] Alternatively, a device that performs the above-mentioned processing operations may be configured by distributing a program for executing the above-mentioned processing operations on a computer-readable non-temporary recording medium such as a flexible disk, CD-ROM (Compact Disk Read-Only Memory), DVD (Digital Versatile Disk), or MO (Magneto-Optical Disk), and then installing that program on a computer.

[0076] The embodiments described above are illustrative, and the present invention is not limited thereto. Various embodiments are possible without departing from the spirit of the invention as described in the claims. The components described in the embodiments and modifications can be freely combined. Furthermore, inventions equivalent to the invention described in the claims are also included in the present invention. [Explanation of symbols]

[0077] 1. Stereo measurement device 2A, 2B Line Sensor Camera 3-line laser light source 4 Housing 5 Moving Stages 6. Stand 21 lenses 22-line sensor 100 processing units 110 Operation section 120 Display section 130 Communications Department 140 Storage section 150 Control Unit 151 Acquisition Department 152 Guide section 153 Arithmetic section Output section of 154

Claims

1. A calibration method for calibrating the external parameters of a line sensor camera in a stereo measuring device comprising a pair of line sensor cameras and a line light source positioned side by side between the pair of line sensor cameras to form a plane of light in space, wherein the device comprises a pair of line sensor cameras and a line light source positioned between the pair of line sensor cameras to form a plane of light in space, In a Cartesian coordinate system where the direction of light emission from the line light source is the Y-axis direction, the direction of light emission is the Z-axis direction, and the direction perpendicular to the Y-axis and Z-axis is the X-axis direction, a first adjustment step involves adjusting the position of the line sensor camera in the X-axis direction and the rotation angles around the Y-axis and Z-axis, respectively, so that the plane of the imaging range formed by the line sensor camera coincides with the plane of light formed by the line light source. A second adjustment step involves adjusting the rotation angle of the line sensor camera around the X-axis so that the optical axis of the line sensor camera, whose position and rotation angle have been adjusted by the first adjustment step, extends in the Z-axis direction. A first calculation step calculates the distance in the Z-axis direction between the line sensor camera and the calibration piece based on the image captured by the line sensor camera whose rotation angle has been adjusted by the second adjustment step and the focal length of the lens of the line sensor camera, A second calculation step calculates the distance in the Y-axis direction between a line extending in the Z-axis direction from the center of the image of the line sensor camera and the calibration piece, based on the distance in the Z-axis direction between the line sensor camera and the calibration piece, and a reference coordinate in the Y-axis direction provided on the calibration piece, based on the distance in the Z-axis direction between the line sensor camera and the calibration piece, calculated in the first calculation step, Includes, The calibration piece is a slitted calibration piece having multiple slits arranged at equal intervals in the longitudinal direction of the substrate and capable of reflecting light, In the second adjustment step described above, The calibration piece, illuminated by light from the line light source, is photographed by the line sensor camera. The orientation of the line sensor camera is adjusted so that the spacing between the brightness peaks corresponding to the slits in the image captured by the line sensor camera becomes uniform. Calibration method.

2. In the first adjustment step described above, The screen illuminated by light from the line light source is photographed by the line sensor camera. The position and orientation of the line sensor camera are adjusted so that the brightness is uniform in each pixel of the image captured by the line sensor camera. The calibration method according to claim 1.

3. In the first adjustment step, if the difference between the maximum and minimum brightness values ​​of all pixels in the image captured by the line sensor camera is less than or equal to a threshold, it is determined that the brightness is uniform in each pixel of the image captured by the line sensor camera. The calibration method according to claim 2.

4. In the first adjustment step, the screen is placed at two different positions in the Z-axis direction, and the position and orientation of the line sensor camera are adjusted so that the brightness is uniform in each pixel of the image captured by the line sensor camera at each position. The calibration method according to claim 2 or 3.

5. In the first calculation step, the distance in the Z-axis direction between the line sensor camera and the calibration piece is calculated based on the spacing of the slits in the calibration piece, the spacing of the brightness peaks corresponding to the slits in the image captured by the line sensor camera, and the focal length of the lens of the line sensor camera. The calibration method according to claim 1.

6. The aforementioned slitted calibration piece is provided at the reference coordinates and includes a marking that is detectable on the image, In the second calculation step, based on the distance between the reference coordinate markings obtained from the image of the line sensor camera and the image center, the distance in the Z-axis direction between the line sensor camera and the calibration piece, and the focal length of the lens of the line sensor camera, the distance in the Y-axis direction between the intersection point of the line extending in the Z-axis direction from the image center of the line sensor camera and the calibration piece and the reference coordinate in the Y-axis direction provided on the calibration piece is calculated. The calibration method according to claim 1.

Images