Camera calibration device, camera calibration system, and camera calibration method

The camera calibration device and method address the issue of accumulated errors in multi-camera systems by aligning image data with real-space reference points, enhancing positional accuracy between cameras with non-overlapping fields of view.

JP2024167969APending Publication Date: 2024-12-05KAWASAKI RAILCAR MFG CO LTD
View PDF 1 Cites 0 Cited by

Patent Information

Application Number
JP2023084314
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Filing Date
2023-05-23
Publication Date
2024-12-05

AI Technical Summary

Technical Problem

Existing methods for calibrating multiple cameras result in accumulated errors and increased relative positional errors when the imaging ranges of the cameras are different.

Method used

A camera calibration device and method that inputs image data with distinct reference points, converts this data into a real coordinate system using parameter groups, and adjusts these groups to match the actual positional relationships between reference points in real space, thereby improving the relative positional accuracy between cameras with non-overlapping imaging ranges.

Benefits of technology

The solution effectively reduces relative positional errors between cameras with different imaging ranges, allowing for accurate determination of distances and positional relationships between imaged objects, such as railway tracks.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2024167969000001_ABST
    Figure 2024167969000001_ABST
Patent Text Reader

Abstract

To improve relative positional errors between different imaging ranges.SOLUTION: A camera calibration device includes: an image input unit which receives the input of first image data containing a first reference point and second image data containing a second reference point different from the first reference point; a position information input unit which receives the input of real-space reference position information related to a first real reference point corresponding to the first reference point and a second real reference point corresponding to the second reference point in a real space; and a processing unit which obtains a first parameter group for converting the first image data into data of a real coordinate system and a second parameter group for converting the second image data into data of the real coordinate system. The processing unit determines the first parameter group and the second parameter group so that the positional relationship between the first conversion position where the first reference point is converted into the real coordinate system and the second conversion position where the second reference point is converted into the real coordinate system matches the positional relationship between the first real reference point and the second real reference point in the real space.SELECTED DRAWING: Figure 4
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] This disclosure relates to techniques for calibrating cameras. [Background technology]

[0002] Patent Document 1 discloses a technique for performing calibration by capturing images of a plurality of common calibration points with two cameras. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2018-179981 A Summary of the Invention [Problem to be solved by the invention]

[0004] Here, if calibration is performed separately for a plurality of cameras, errors may accumulate, and the relative position error between the cameras may increase.

[0005] According to the technology described in Patent Document 1, it is possible to perform calibration for two cameras based on a plurality of common calibration points as references. However, this technology cannot be applied when the imaging ranges of the two cameras are different.

[0006] Therefore, an object of the present disclosure is to improve the relative position error between different imaging ranges. [Means for solving the problem]

[0007] In order to solve the above problem, a camera calibration device includes an image input unit to which first image data including a first reference point and second image data including a second reference point different from the first reference point are input, a position information input unit to which real space reference position information regarding a first real reference point corresponding to the first reference point in real space and a second real reference point corresponding to the second reference point in real space is input, and a processing unit that obtains a first parameter group for converting the first image data into data in a real coordinate system and a second parameter group for converting the second image data into data in the real coordinate system, wherein the processing unit determines the first parameter group and the second parameter group based on the first image data, the second image data, and the real space reference position information such that a positional relationship between a first transformed position obtained by transforming the first reference point of the first image data into the real coordinate system and a second transformed position obtained by transforming the second reference point of the second image data into the real coordinate system conforms to a positional relationship between the first real reference point and the second real reference point in real space.

[0008] In addition, in order to solve the above problem, a camera calibration method includes acquiring image data including a first reference point and a second reference point different from the first reference point, acquiring real space reference position information regarding a first real reference point corresponding to the first reference point in real space and a second real reference point corresponding to the second reference point, and determining the first parameter group and the second parameter group so that a positional relationship between a first transformed position obtained by transforming the first reference point of the image data into a real coordinate system using a first parameter group and a second transformed position obtained by transforming the second reference point of the image data into the real coordinate system using a second parameter group conforms to a positional relationship between the first real reference point and the second real reference point in real space. Effect of the Invention

[0009] This calibration device can improve the relative position error between different imaging ranges.

[0010] The calibration method can also improve the relative position error between different imaging ranges. [Brief description of the drawings]

[0011] [Figure 1] FIG. 1 is a block diagram showing a camera calibration system according to the first embodiment. [Diagram 2] FIG. 2 is an explanatory diagram showing the positional relationship between the calibration tool and the first camera. [Diagram 3] FIG. 3 is a block diagram showing the calibration device. [Figure 4] FIG. 4 is a flowchart showing an example of processing by the calibration device. [Diagram 5] FIG. 5 is an explanatory diagram showing the relationship between the actual coordinates of the reference pin and the coordinates of the bright line R in the image data. [Figure 6] FIG. 6 is a block diagram showing a calibration system to which the calibration device according to the second embodiment is applied. [Figure 7] FIG. 7 is a block diagram showing the calibration device. [Figure 8] FIG. 8 is a flowchart showing an example of processing by the calibration device. [Figure 9] FIG. 9 is a diagram showing a display example of the display device. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] {First embodiment} The camera calibration device, camera calibration system, and camera calibration method according to the first embodiment will be described below.

[0013] Fig. 1 is a block diagram showing a camera calibration system, which mainly shows an example of the configuration of a camera that is the subject of calibration.

[0014] The calibration system 20 includes a calibration device 50, a first camera 30A, and a second camera 30B.

[0015] The first camera 30A and the second camera 30B may be supported by a railway vehicle 10. The railway vehicle 10 is a vehicle that travels on a track including a first rail 12A and a second rail 12B. The first rail 12A and the second rail 12B are laid in parallel on the ground or the like via sleepers or the like. The first rail 12A and the second rail 12B may be provided at a position above ground level by an elevated bridge or the like. The track may be provided in a tunnel dug underground.

[0016] The first camera 30A is supported by the railway vehicle 10 in a state in which it can image the first rail 12A. The second camera 30B is supported by the railway vehicle 10 in a state in which it can image the second rail 12B. Therefore, the first camera 30A and the second camera 30B image different imaging ranges. The first image data output from the first camera 30A and the second image data output from the second camera 30B are data that image different imaging ranges. Here, different imaging ranges refer to a case in which the imaging ranges do not match, and therefore includes a case in which the imaging ranges partially overlap and a case in which the imaging ranges do not overlap at all. The calibration according to this embodiment is also applicable to a case in which the imaging ranges do not overlap at all.

[0017] First rail 12A and second rail 12B are located apart below railcar 10. For this reason, by capturing images of first rail 12A and second rail 12B with separate cameras 30A, 30B, it is possible to obtain images with less distortion compared to capturing images of first rail 12A and second rail 12B with a single wide-angle camera.

[0018] The cameras 30A, 30B are cameras including a CCD (Charge Coupled Device) image sensor or a CMOS (Complementary MOS) image sensor. The first camera 30A may be supported under the railcar 10 with its central axis facing the first rail 12A. The second camera 30B may be supported under the railcar 10 with its central axis facing the second rail 12B. By having the central axes of the cameras 30A, 30B facing the rails 12A, 12B, distortion of the rails 12A, 12B in the captured images is reduced.

[0019] The position of the surface shape of the first rail 12A in real space is calculated by performing projective transformation defined by a first preset parameter group on the image of the surface shape of the first rail 12A reflected in the first image data. Similarly, the position of the surface shape of the second rail 12B in real space is calculated by performing projective transformation defined by a second preset parameter group on the image of the surface shape of the second rail 12B reflected in the second image data.

[0020] Based on the position of the surface shape of first rail 12A and the position of the surface shape of second rail 12B in real space, the positional relationship between first rail 12A and second rail 12B can be grasped. For example, the distance between first rail 12A and second rail 12B can be grasped.

[0021] The first and second parameter groups are obtained, for example, by the following calibration process.

[0022] 2 is an explanatory diagram showing the positional relationship between the calibration tool 16A and the first camera 30A during calibration. As shown in Fig. 1 and Fig. 2, the calibration tools 16A and 16B are positioned on the first rail 12A and the second rail 12B, respectively.

[0023] The calibration tool 16A includes a base 17A and a first reference pin 18A. The number of the first reference pins 18A is arbitrary. Eight parameters C11 , C 12 , C 13 , C 21 , C 22 , C 23 , C 31 , C 32 In order to obtain this, the calibration tool 16A should preferably include four or more first reference pins 18A.

[0024] The base 17A supports the first reference pins 18A in a relatively positioned state. Here, the base 17A is formed in a long and narrow plate shape. The shape of the base is not particularly limited, and may be formed in other shapes, for example, a rectangular parallelepiped shape.

[0025] The base 17A includes a positioning recess 17Ah. Here, the positioning recess 17Ah is formed in the lower part of the longitudinal middle part of the base 17A. The positioning recess 17Ah is formed in a concave shape that opens downward. The width of the positioning recess 17Ah is equal to or greater than the width of the upper part of the first rail 12A. The upper part of the first rail 12A can be placed in the positioning recess 17Ah. The upper part of the first rail 12A is placed in the positioning recess 17Ah with the base 17A perpendicular to the extension direction of the first rail 12A. As a result, the top of the first rail 12A contacts the downward surface at the back of the positioning recess 17Ah, and the base 17A is positioned relative to the first rail 12A in the up-down direction. Also, one side edge of the top of the first rail 12A is pressed against one side edge of the positioning recess 17Ah. This allows the positioning of base 17A relative to first rail 12A in the width direction of first rail 12A.

[0026] The base 17A may be positioned on the first rail 12A via another object. For example, the base 17A may be positioned relative to the railcar 10. The railcar 10 contacts the first rail 12A via its wheels, and therefore the base 17A is positioned on the first rail 12A via the railcar 10.

[0027] As described above, the base 17A is directly or indirectly positioned on the first rail 12A, and thereby the multiple first reference pins 18A are also positioned with respect to the first rail 12A.

[0028] The calibration tool 16B, like the calibration tool 16A, includes a base 17B and a second reference pin 18B. The calibration tool 16B may have only one first reference pin 18B, or may have a plurality of first reference pins 18B.

[0029] The position of the first reference pin 18A in the calibration tool 16A and the position of the second reference pin 18B in the calibration tool 16B may be in a mirror image relationship, may have the same positional relationship, or may be in an unrelated positional relationship to each other.

[0030] The calibration tool 16A and the calibration tool 16B may be connected via a connecting portion 16C. If the calibration tool 16A and the calibration tool 16B are connected, the relative positional relationship between the first reference pin 18A and the second reference pin 18B is easily kept constant in real space.

[0031] An example of calibration using the calibration tool 16A will be described.

[0032] Slit light L from slit light source 19 is irradiated onto the multiple first reference pins 18A. Slit light source 19 is composed of a light source such as a light emitting diode or a light bulb, and a light passage limiting section that limits the light from the light source to a slit shape using a slit-shaped hole. When the slit light L is irradiated onto the multiple first reference pins 18A, a bright line R due to the slit light L appears on each of the multiple first reference pins 18A.

[0033] The first camera 30A captures an image of the bright line R caused by the slit light L irradiated onto the first reference pins 18A. The first camera 30A is provided at a position where it can capture an image of the bright line R on the reference pins 18A, 18B from a position different from the slit light source 19.

[0034] The first reference pin 18A is an example of a first real reference point in real space. The first camera 30A captures an image of a first imaging range E1 including at least one of the first reference pins 18A, and outputs first image data. The bright line R irradiated on the first reference pin 18A is also captured in the first image data. The bright line R included in the first image data is an example of a first reference point. Each of the multiple first reference pins 18A is associated with each of the multiple bright lines R on a one-to-one basis. The position of the first reference pin 18A may be expressed by the center coordinates of the first reference pin 18A in the XY plane. Examples of the Z direction, Y direction, and Z direction will be described later. Hereinafter, the position of the first reference pin 18A or the second reference pin 18B may be expressed as coordinates. The center coordinates of the first reference pin 18A may be the geometric center of the projection shape of the first reference pin 18A in the XY plane. Furthermore, the position of the bright line R may be expressed by the central coordinates of the bright line R in the first image data 31A or the second image data 31B. The central coordinates of the bright line R may be the geometric center of the bright line R in the first image data. Hereinafter, the position of the bright line R may be expressed as the coordinates of the bright line R.

[0035] When performing calibration on the premise of using the railway vehicle 10, it can be assumed that the slit light source 19 and the first camera 30A are supported by the railway vehicle 10.

[0036] An example of the positional relationship of the slit light source 19 and the first camera 30A with respect to the first reference pin 18A will be described. For convenience of explanation, the width direction of the rails 12A and 12B may be referred to as the X direction, the height direction of the rails 12A and 12B as the Y direction, and the direction perpendicular to the X direction and the Y direction as the Z direction. The Z direction is the extension direction of the rails 12A and 12B.

[0037] The first camera 30A and the light source 19 are located above the first rail 12A. The first camera 30A and the light source 19 are provided at positions separated from each other in the Z direction.

[0038] Based on the actual coordinates of the multiple first reference pins 18A and the position of the bright line R in the first image data captured by the first camera 30A, a group of parameters can be obtained for projecting and transforming the coordinates on the first image data into the actual coordinates.

[0039] That is, if the slit light source 19 and the first camera 30A are supported at a fixed position and attitude with respect to the base 17A, the positional relationship between the multiple first reference pins 18A supported at a fixed position with respect to the base 17A, the slit light source 19, and the first camera 30A is also fixed. The coordinates of the bright line R in the first image data captured by the first camera 30A are determined by the positional relationship between the first reference pin 18A, the slit light source 19, and the first camera 30A. In addition, when performing calibration, since the first reference pin 18A is positioned on the first rail 12A, the actual coordinates of the first reference pin 18A in the real space can be known information. Therefore, a group of parameters for projectively transforming the coordinates on the first image data into the actual coordinates can be obtained based on the actual coordinates of the multiple first reference pins 18A and the coordinates of the bright line R in the first image data. In this way, the process of irradiating the object with slit light and capturing an image of the slit light to measure the three-dimensional shape of the object is an example of the application of the light section method.

[0040] Similarly, based on the actual coordinates of the multiple second reference pins 18B and the position of the bright line R in the second image data captured by the second camera 30B, a group of parameters can be determined for projecting and transforming the coordinates on the second image data into actual coordinates.

[0041] As described above, the second reference pin 18B is an example of a second real reference point in real space, and the second camera 30B captures an image of the second imaging range E2 including at least one of the second reference pins 18B, and outputs the second image data. The bright line R irradiated onto the second reference pin 18B is also captured in the second image data. The bright line R included in the second image data is an example of a second reference point. Each of the multiple second reference pins 18B is associated with each of the multiple bright lines R in a one-to-one relationship. As described above, the second reference pin 18B is located at a different location from the first reference pin 18A. Therefore, the bright line R in the second image data is different from the bright line R in the second image data.

[0042] Here, it is assumed that the state of the track is monitored based on the positional relationship between the first rail 12A and the second rail 12B. If a group of parameters for projectively transforming the coordinates on the first image data into real coordinates and a group of parameters for projectively transforming the coordinates on the second image data into real coordinates are calculated separately, errors are accumulated, and there is a possibility that the positional relationship between the first rail 12A and the second rail 12B cannot be calculated accurately.

[0043] Therefore, a calibration device 50 and a calibration method described below are used to determine a group of parameters that can more accurately determine the positional relationship between first rail 12A and second rail 12B.

[0044] FIG. 3 is a block diagram showing the calibration device 50.

[0045] The first image data 31A and the second image data 31B obtained by capturing images with the first camera 30A and the second camera 30B are output to a data collection device 34. The data collection device 34 includes a storage device 35 configured with a flash memory, a magnetic storage device, or the like, and the first image data 31A and the second image data 31B are stored in the storage device 35.

[0046] The calibration device 50 includes a calibration processing unit 51 including a processor 52 such as a CPU, a storage device 54, an input / output interface 56, a communication interface 58, etc. The calibration processing unit 51 is formed by, for example, a computer.

[0047] The communication interface 58 is a communication circuit that communicates with the data collecting device 34. The communication interface 58 may perform wireless communication or wired communication with the data collecting device 34. The communication interface 58 is an example of an image input unit to which the first image data 31A and the second image data 31B are input.

[0048] It is not essential that the image input unit is a communication circuit that communicates with the data collecting device 34. For example, the first image data 31A and the second image data 31B stored in the data collecting device 34 may be stored in a portable storage medium. The portable storage medium may be, for example, a small flash memory such as a USB memory or an optical disk. In this case, the image input unit may be a recording medium reading device that reads information from the storage medium.

[0049] The input / output interface 56 is an input / output circuit for inputting and outputting data to and from the outside. The input / output interface 56 may be an example of a position information input unit to which real space reference position information regarding the first reference pin 18A as the first real reference point and the second reference pin 18B as the second real reference point is input. For example, real coordinate data 56D including the XY coordinates of the first reference pin 18A and the XY coordinates of the second reference pin 18B in the real space may be input to the calibration device 50 via the input / output interface 56 as real space reference position information. The real coordinate data 56D may be data prepared in advance in the form of table data, or may be input by a user via an input device such as a keyboard.

[0050] A keyboard 56a having a plurality of switches or a coordinate input device 56b such as a mouse may be connected to the input / output interface 56. By using the keyboard 56a or the coordinate input device 56b, various instructions from a user are accepted. For example, the position of the bright line R in the first image data 31A and the second image data 31B may be specified by the user through the keyboard 56a or the coordinate input device 56b. In addition, the bright line R in the first image data 31A and the second image data 31B may be associated with the first reference pin 18A or the second reference pin 18B by the user.

[0051] The processor 52 includes circuitry.

[0052] The storage device 54 is configured by a non-volatile storage device such as a hard disk drive (HDD) or a solid-state drive (SSD). For example, a program 54a, first image data 31A, second image data 31B, reference point data 54b, actual coordinate data 56D, and a parameter group Pr are stored in the storage device 54. The program 54a describes a process for the processor 52 to realize the function as a processing unit. The reference point data 54b includes coordinates of the bright line R, which is the first reference point and the second reference point, and information that associates each bright line R with the first reference pin 18A and the second reference pin 18B, which are actual reference point data. An example of generating the reference point data 54b will be described later. The parameter group Pr includes the first parameter group and the second parameter group obtained as a result of the process.

[0053] By executing the processing described in the program 54a stored in the memory device 54 or the like, the processor 52 performs the function of a processing unit that determines a first group of parameters for converting the first image data 31A into data in the real coordinate system and a second group of parameters for converting the second image data 31B into data in the real coordinate system.

[0054] More specifically, based on the first image data 31A, the second image data 31B, and real coordinate data 56D, the processor 52 executes a process of determining the first parameter group and the second parameter group so that the positional relationship between a first conversion position obtained by converting the coordinates of the bright line R in the first image data 31A into the real coordinate system and a second conversion position obtained by converting the coordinates of the bright line R in the second image data 31B into the real coordinate system matches the positional relationship between the coordinates of the first reference pin 18A and the coordinates of the second reference pin 18B in real space.

[0055] Here, the positional relationship is a relative positional relationship between a plurality of positions. For example, the relationship between two positions may be defined by a parameter whose value can change if one position is used as a reference and the other position is changed. For example, the relationship may be defined by the distance, elevation difference, inner product, cross product, etc. of the coordinates indicating the two positions. In this embodiment, an example in which the positional relationship is distance will be mainly described later.

[0056] In this embodiment, before determining the first parameter group and the second parameter group, the processor 52 obtains a first initial parameter group so that the first transformed position matches the position of the first reference pin 18A in real space, and obtains a second initial parameter group so that the second transformed position matches the position of the second reference pin 18B in real space. Then, the processor 52 determines the first parameter group and the second parameter group using the first initial parameter group and the second initial parameter group.

[0057] Therefore, the processor 52 has a function as an initial parameter calculation processing unit 52a that determines the first and second initial parameter groups, and a function as a parameter calculation processing unit 52b that determines the first and second parameter groups.

[0058] There may be one or more processors 52. The multiple processors 52 may be incorporated in one computer. The multiple processors 52 may be incorporated in multiple computers, and the multiple computers may perform the processing as a processing unit for determining the above parameters in a distributed manner.

[0059] The calibration device 50 may include a display device 60. The display device 60 may be a liquid crystal display device, an organic EL (Electro-luminescence) display device, or the like. A display device provided in a smartphone, a tablet terminal, a laptop computer, or the like may be used as the display device 60. The display device 60 may be connected to the calibration processing unit 51 via an input / output interface 56.

[0060] The display device 60 may display positional relationship information specified by application of the parameter group. An example of the positional relationship information will be described later. The display device 60 may display the first image data 31A and the second image data 31B. In this case, the user may look at the displayed first image data 31A and the second image data 31B and operate the keyboard 56a or the coordinate input device 56b to input the position of the bright line R in the first image data 31A and the second image data 31B and the correspondence information of the reference pins 18A and 18B.

[0061] An example of processing in the calibration device 50 will be described with reference to the flowchart shown in FIG.

[0062] After the construction process starts, in steps S1a and S1b, the first image data 31A and the second image data 31B are read in. The first image data 31A includes images of multiple bright lines R as multiple first reference points, and the second image data 31B also includes images of multiple bright lines R as multiple second reference points.

[0063] In the next step S2, reference point coordinates in the first image data 31A and the second image data 31B are identified. The coordinates of the reference point are the coordinates of the bright line R. Furthermore, the reference points are associated with actual reference points. The association between the reference points and the actual reference points is achieved by associating each bright line R in the first image data 31A and the second image data 31B with the reference pins 18A and 18B.

[0064] For example, the specification of the reference point coordinates and the association between the reference point and the actual reference point may be performed by receiving an input from a user. For example, the first image data 31A is displayed on the display device 60. The user uses the coordinate input device 56b to match the position of the pointer icon to the position of the bright line R, thereby specifying the position of the bright line R. The coordinates of the specified position are accepted as the coordinates of the bright line R. In conjunction with the position specification of the bright line R, the user inputs association information between the reference point and the actual reference point using the keyboard 56a or the like. For example, it is assumed that an identification code such as P(n) is assigned to the reference pins 18A and 18B. n is a number unique to the reference pins 18A and 18B. In conjunction with the position specification of the bright line R, the user inputs the identification code, thereby inputting association information between the reference point and the actual reference point.

[0065] By repeating the above process for each bright line R and both the first image data 31A and the second image data 31B, the reference point coordinates are specified and the reference points are associated with the actual reference points. The data of the reference point coordinates and the association data between the reference points and the actual reference points are stored in the storage device 54 as the reference point data 54b.

[0066] The above process may be performed automatically by image recognition processing or the like. For example, a pattern matching process may be performed on the first image data 31A and the second image data 31B to recognize the position of the bright line R reflected in the first image data 31A and the second image data 31B. In addition, the position of the bright line R reflected in the first image data 31A and the second image data 31B is considered to be located near an ideal position based on the positions of the reference pins 18A and 18B, the positional relationship of the light source 19 and the cameras 30A and 30B. Therefore, it may be determined which of the reference pins 18A and 18B the position of the bright line R reflected in the first image data 31A and the second image data 31B corresponds to by determining which of the ideal positions of the reflected bright line R the position of the bright line R is closer to.

[0067] Thereafter, in steps S3a and S3b, the coordinate data of the first real reference point and the coordinate data of the second real reference point are read from the actual coordinate data 56D stored in the storage device 54. The coordinate data of the first real reference point and the coordinate data of the second real reference point are, for example, the coordinate data of the first reference pin 18A and the coordinate data of the second reference pin 18B.

[0068] In the next steps S4a and S4b, a calibration process is performed for each of the single camera 30A and the single camera 30B.

[0069] The calibration process for the single camera 30A includes an initial parameter group calculation process.

[0070] The initial parameter group calculation process is a process for obtaining a first initial parameter group and a second initial parameter group before determining a first parameter group for converting the first image data 31A into data in the real coordinate system and a second parameter group for converting the second image data 31B into data in the real coordinate system. The first initial parameter group is obtained so that a first converted position obtained by converting the coordinates of the bright line R of the first image data 31A into the real coordinate system matches the position of the first reference pin 18A in the real space. The second initial parameter group is obtained so that a second converted position obtained by converting the coordinates of the bright line R of the second image data 31B into the real coordinate system matches the position of the second reference pin 18B in the real space.

[0071] A more detailed explanation will be given. Fig. 5 is an explanatory diagram showing the relationship between the actual coordinates of the first reference pin 18A and the second reference pin 18B and the coordinates (x, y) of the bright line R in the first image data 31A or the second image data 31B. The actual coordinates of the first reference pin 18A and the second reference pin 18B are expressed in a world coordinate system having an Xw axis and a Yw axis. The coordinates of the bright line R in each of the first image data 31A and the second image data 31B are expressed in a camera coordinate system having an x ​​axis and a y axis.

[0072] If the actual coordinates of the first reference pin 18A are (X, Z) and the coordinates of the bright line R in the first image data 31A are (x, y), the relationship between the two is expressed by the following equation 1. Equation 1 is an equation showing projective transformation.

[0073]

number

[0074] Here, C 11 , C 12 , C 13 , C 21 , C 22 , C 23 , C 31 , C 32 is a parameter and h is a constant. 11 , C 12 , C 13 , C 21 , C 22 , C 23 , C 31 , C 32 The matrix T, whose matrix components are 1, 2, and 3, is a matrix that performs projective transformation on the coordinates (X, Y) of the first reference pin 18A in the real space to the coordinates (x, y) of the bright line R in the first image data 31A. -1 is a matrix that performs projective transformation of the coordinates (x, y) of the bright line R in the first image data 31A to the coordinates (X, Y) of the first reference pin 18A in real space. 11 , C 12 , C 13 , C 21 , C 22 , C 23 , C 31 , C 32 can be understood as expressing a first parameter group or a first initial parameter group for converting the first image data 31A into data in the real coordinate system.

[0075] And the inverse matrix T -1 The parameter C is set so that the first transformed position by 11 , C 12 , C 13 , C 21 , C 22 , C 23 , C 31 , C 32 is calculated, and the calculation results are determined as a first initial parameter set.

[0076] Specifically, if there are four combinations of coordinates (x, y) of the bright line R in the first image data 31A associated with the coordinates (X, Y) of the first reference pin 18A, eight parameters C 11 , C 12 , C 13 , C 21 , C 22 , C 23 , C 31 , C 32 can be identified.

[0077] When the number of combinations of the coordinates (x, y) of the bright line R in the first image data 31A associated with the coordinates (X, Y) of the first reference pin 18A exceeds four, the eight parameters C are calculated by a least squares method that minimizes the total value of the position error between the first transformed position obtained by projective transformation of the bright line R and the position of the first reference pin 18A in real space. 11 , C 12 , C 13 , C 21 , C 22 , C 23 , C 31 , C 32 may be specified.

[0078] Similarly, the initial parameter group calculation process is performed as a calibration process for the single camera 30B.

[0079] In the next step S5a, the parameter C identified in step S4a is 11 , C 12 , C 13 , C 21 , C 22 , C 23 , C 31 , C 32 are set as a first initial parameter group for the first image data 31A captured by the first camera 30A.

[0080] Similarly, in S5b, the parameter C identified in step S4b is 11 , C 12 , C 13 , C 21 , C22 , C 23 , C 31 , C 32 are set as a second initial parameter group for the second image data 31B captured by the second camera 30B.

[0081] In the next step S6, a multi-camera calibration process is performed.

[0082] That is, in steps S4a and S5a, a first set of initial parameters may be determined mainly based on the characteristics of the first camera 30A, and in steps S4b and S5b, a second set of initial parameters may be determined mainly based on the characteristics of the second camera 30B.

[0083] In step S6, a first initial parameter group and a second parameter group are determined so that the positional relationship between a first conversion position obtained by converting the coordinates of the bright line R in the first image data 31A into the real coordinate system and a second conversion position obtained by converting the coordinates of the bright line R in the second image data 31B into the real coordinate system matches the positional relationship between the first reference pin 18A and the second reference pin 18B in real space.

[0084] In this embodiment, the positional relationship is a distance. More specifically, the first parameter group and the second parameter group are determined so that the distance between the first converted position and the second converted position matches the distance D between the first reference pin 18A and the second reference pin 18B in the real space. In addition, the first parameter group and the second parameter group are determined based on a combination of a plurality of positional relationships.

[0085] More specifically, the bright line R in the first image data 31A is projectively transformed to obtain a first transformed position. The bright line R in the second image data 31B is projectively transformed to obtain a second transformed position. Then, the distance between the first transformed position and the second transformed position is obtained. The distance is calculated using the eight parameters C for the first image data 31A. 11 , C 12 , C 13 , C 21 , C 22 , C 23 , C 31 , C32 and eight parameters C for the second image data 31B 11 , C 12 , C 13 , C 21 , C 22 , C 23 , C 31 , C 32 , that is, it is expressed by 16 parameters.

[0086] By optimizing the above 16 parameters, simultaneous optimization is performed for the first camera 30A and the second camera 30B.

[0087] Specifically, based on the reference point data 54b and the actual coordinate data 56D, the coordinates of the first reference pin 18A associated with the bright line R of the first image data 31A, which is the distance calculation target, and the coordinates of the second reference pin 18B associated with the bright line R of the second image data 31B, which is the distance calculation target, are specified. Then, as shown in Fig. 5, the distance D between the first reference pin 18A and the second reference pin 18B in the real space is calculated. Note that the distance D may be calculated based on the coordinates of the first reference pin 18A and the coordinates of the second reference pin 18B in the real space, or the distance D may be input and stored in advance as information.

[0088] Then, the square of the error between the distance between the first converted position and the second converted position and the distance between the first reference pin 18A and the second reference pin 18B in the real space is calculated.

[0089] The above calculation is performed for multiple combinations of the bright line R in the first image data 31A and the bright line R in the second image data 31B to obtain the sum of squares of the distance error.

[0090] The above 16 parameters are found by the least squares method that minimizes the total value of this position error.

[0091] As an optimization method, for example, the Levenberg-Marquardt method, which is an example of a nonlinear optimization method, may be adopted. Alternatively, the optimization process may be performed by the Gauss-Newton method or the like.

[0092] The first and second initial parameter sets may be used as initial values ​​for optimization, which can prevent, for example, a local optimum solution from being found or a time-consuming calculation from being required.

[0093] Note that it is not essential to perform optimization using the first and second initial parameter sets, and therefore the processes of steps S4a, S4b, S5a, and S5b may be omitted.

[0094] In this embodiment, an example is described assuming two cameras 30A and 30B. Calibration can be performed in the same manner as described above for three or more cameras. In this case, for example, it is advisable to increase the number of combinations of distances between the reference pins.

[0095] In step S7a, of the 16 parameters thus obtained, a parameter group for the first image data 31A is set as a first parameter group, and in step S7b, a parameter group for the second image data 31B is set as a second parameter group.

[0096] This completes the calibration process.

[0097] The first and second parameter groups thus determined can be used to grasp position information in the real space based on image data captured by the cameras 30A and 30B.

[0098] The first parameter group and the second parameter group may be used in the calibration device itself, as in the second embodiment, or may be used in another computer as parameters for calculating the positional relationship between the first measurement object and the second measurement object.

[0099] According to the calibration device 50 and the calibration method configured as above, the calibration processing unit 51 obtains a first conversion position obtained by converting the coordinates of the bright line R, which is the first reference point of the first image data 31A, into the real coordinate system, and obtains a second conversion position obtained by converting the coordinates of the bright line R, which is the second reference point of the second image data 31B, into the real coordinate system. Then, the first parameter group and the second parameter group are determined so that the positional relationship between the first conversion position and the second conversion position matches the positional relationship between the first reference pin 18A, which is the first real reference point, and the second reference pin 18B, which is the second real reference point, in the real space. Therefore, even if the cameras 30A and 30B separately capture the first reference pin 18A and the second reference pin 18B, the calibration can be performed. Therefore, even if the imaging ranges of the multiple cameras 30A and 30B are different from each other, the relative position error between the multiple cameras 30A and 30B can be improved.

[0100] In particular, the first parameter group and the second parameter group are determined so that the distance between the first conversion position and the second conversion position matches the distance between the first reference pin 18A and the second reference pin 18B in real space. Therefore, the first parameter group and the second parameter group can be determined so that the distance between the object reflected in the first image data 31A and the object reflected in the second image data 31B can be obtained more accurately. For example, this is suitable for more accurately detecting the distance between the rails 12A and 12B.

[0101] Moreover, the first image data 31A includes a plurality of bright lines R as first reference points, and the second image data 31B includes a plurality of bright lines R as second reference points. The calibration processing unit 51 determines the first parameter group and the second parameter group based on a plurality of combinations of the positional relationship between the first conversion position and the second conversion position, and the positional relationship between the first reference pin 18A and the second reference pin 18B. This makes it possible to more appropriately determine the first parameter group and the second parameter group.

[0102] Furthermore, the calibration processing unit 51 can determine the first parameter group and the second parameter group by an algorithm that minimizes the error in the positional relationship.

[0103] The calibration processing unit 51 can perform calibration by performing projective transformation to obtain a first transformed position and a second transformed position, and then obtaining parameters used in the projective transformation.

[0104] When applying an algorithm that minimizes the error in the above-mentioned positional relationship, the calibration processing unit 51 can quickly determine a more appropriate set of parameters by using the first and second initial parameter sets obtained by the calibration processing of the standalone cameras 30A and 30B.

[0105] Furthermore, according to the calibration system 20 including the first camera 30A and the second camera 30B, calibration can be performed using the cameras 30A and 30B.

[0106] In this case, when the first camera 30A is supported on the railway vehicle 10 in a state capable of imaging the first rail 12A, and the second camera 30B is supported on the railway vehicle 10 in a state capable of imaging the second rail 12B, a calibration suitable for accurately grasping the positional relationship between the first rail 12A and the second rail 12B on which the railway vehicle 10 runs is performed.

[0107] Furthermore, in this case, the first camera 30A captures an image of the first reference pin 18A positioned on the first rail 12A as a first real reference point, and the second camera 30B captures an image of the second reference pin 18B positioned on the second rail 12B as a second real reference point. This makes it easy to accurately set the first real reference point and the second real reference point for the first rail 12A and the second rail 12B. This allows for a calibration that is suitable for accurately grasping the positional relationship between the first rail 12A and the second rail 12B on which the railway vehicle 10 runs.

[0108] {Second embodiment} A calibration device 150 according to the second embodiment will be described. Fig. 6 is a block diagram showing a calibration system 120 to which the calibration device 150 is applied, and Fig. 7 is a block diagram showing the calibration device 150. In the description of the second embodiment, the same components as those described in the first embodiment are denoted by the same reference numerals and their description will be omitted, and the description will focus on the differences from the first embodiment.

[0109] This calibration device 150 has the configuration of the calibration device 50 described in the first embodiment, and in addition has the function of determining positional relationship data indicating the positional relationship between the first measurement object and the second measurement object based on the determined first parameter group and second parameter group, and displaying positional relationship information based on the positional relationship data.

[0110] That is, the communication interface 58 receives the first measurement object image data 131A in which the first measurement object is captured and the second measurement object image data 131B in which the second measurement object is captured. In this embodiment, the first camera 30A captures the first rail 12A, and the second camera 30B captures the second rail 12B. Since the cameras 30A and 30B capture the rails 12A and 12B from above, it is assumed that the tops of the rails 12A and 12B are captured in the first measurement object image data 131A and the second measurement object image data 131B. As in the above, it is preferable that the slit light from the slit light source 19 is irradiated onto the rails 12A and 12B. In addition, in FIG. 6 and FIG. 7, the first measurement object image data and the second measurement object image data are simply described as the first image data and the second image data, respectively.

[0111] First measurement object image data 131A is output from the first camera 30A, and second measurement object image data 131B is output from the second camera 30B, and these data 131A and 131B are input to a calibration processing unit 151 corresponding to the calibration processing unit 51 via a communication interface 58. The communication interface 58 is an example of a measurement object image input unit to which the first measurement object image data 131A and the second measurement object image data 131B are input.

[0112] The calibration processing unit 151 applies the first parameter group to the first measurement object image data 131A, and also applies the first parameter group to the second measurement object image data 131B to obtain positional relationship data indicating the positional relationship between the first rail 12A, which is the first measurement object, and the second rail 12B, which is the second measurement object. The positional relationship is, for example, the distance between the first rail 12A and the second rail 12B. That is, the processor 52 has a function as a rail positional relationship calculation processing unit 52c.

[0113] The display device 60 displays positional relationship information based on the positional relationship data.

[0114] FIG. 8 is a flowchart showing an example of processing by the calibration processing unit 151 functioning as the rail positional relationship calculation processing unit 52c.

[0115] When the rail gap calculation process is started, the first measurement object image data 131A and the second measurement object image data 131B are read in steps S11a and S11b.

[0116] In the next steps S12a and S12b, the interval reference position is specified in each of the first measurement object image data 131A and the second measurement object image data 131B. For example, in each of the first measurement object image data 131A and the second measurement object image data 131B, a shape portion into which the slit light from the slit light source 19 is reflected is specified based on pixel values. Based on pattern matching or the like, a pixel corresponding to the inner boundary of the head of the rails 12A and 12B is specified among the shape portions into which the slit light is reflected. The pixel corresponding to the inner boundary is specified as the interval reference position.

[0117] In the next steps S13a and S13b, the first and second parameter groups obtained by the above calibration process are read out.

[0118] In the next step S14a, the first parameter group is applied to the coordinates of the interval reference position in the first measurement object image data 131A and converted into real coordinates. In step S14b, the second parameter group is applied to the coordinates of the interval reference position in the second measurement object image data 131B and converted into real coordinates.

[0119] In the next step S15, the distance between the two actual coordinates is calculated based on the position obtained by converting the coordinates of the distance reference position in the first measurement object image data 131A into actual coordinates and the position obtained by converting the coordinates of the distance reference position in the second measurement object image data 131B into actual coordinates. The calculated distance is the distance between the rails 12A and 12B, and indicates the positional relationship between the rails 12A and 12B. The calculated distance is stored in the storage device 54 as rail positional relationship data 154c.

[0120] In the next step S16, the presence or absence of the next first measurement object image data 131A and second measurement object image data 131B is determined. If it is determined that the next measurement object image data 131A, 131B are present, the process returns to steps S11a and S11b and the above process is repeated. By continuously providing the measurement object image data 131A, 131B while changing the position in the extension direction of the rails 12A, 12B, the change in the interval between the rails 12A, 12B in the extension direction of the rails 12A, 12B is obtained.

[0121] If it is determined that there is no next measurement object image data 131A, 131B, the process proceeds to step S17. In step S17, it is determined whether or not there is an instruction to display information on the interval between the rails 12A, 12B as positional relationship information. The instruction to display may be input, for example, by the keyboard 56a or the coordinate input device 56b. If it is determined that there is an instruction to display, the process ends, and if it is determined that there is an instruction to display, the process proceeds to step S18.

[0122] In step S18, the interval information is displayed. Fig. 9 shows an example of the interval information 162 displayed on the display device 60.

[0123] The interval information 162 is a graph showing the change in the interval with respect to kilometers. The kilometers indicate the positions in the extending direction of the rails 12A and 12B, and are displayed on the horizontal axis of the graph. The interval is expressed by the difference with respect to the reference interval of the rails 12A and 12B. The reference interval of the rails 12A and 12B is an ideal interval assumed in design for the rails 12A and 12B. A value obtained by subtracting the reference interval from the interval of the rails 12A and 12B calculated in step S15 is the difference with respect to the reference interval. The difference is displayed on the vertical axis of the graph. As the interval information, information according to the interval of the rails 12A and 12B may be displayed. For example, as the interval information, the value of the interval of the rails 12A and 12B calculated in step S15 may be displayed, or a relative error with respect to the reference interval may be displayed. As the interval information, a frequency distribution table may be displayed, or an average value, a standard deviation, or the like may be displayed.

[0124] The calibration device 150 according to the second embodiment applies the first parameter group and the second initial parameter group to obtain positional relationship data indicating the positional relationship between the first measurement object and the second measurement object, and displays positional relationship information based on the positional relationship data on the display device 60. Therefore, by visually checking the display device 60, it is easy to grasp the positional relationship between the first measurement object image data and the second measurement object image data.

[0125] Furthermore, since the first measurement object is first rail 12A and the second measurement object is second rail 12B, the positional relationship between rails 12A and 12B is grasped as accurately as possible.

[0126] Furthermore, since the distance information between first rail 12A and second rail 12B is displayed on display device 60 as positional relationship information, the distance information can be easily grasped as accurately as possible.

[0127] {Variation} In the above embodiment, the calibration has been described for the cameras 30A and 30B that capture images of the rails 12A and 12B. The cameras to be calibrated may capture images of other objects than rails.

[0128] In the above embodiment, the calibration for two cameras 30A and 30B has been described. The present invention can also be applied to the calibration for one camera. In this case, a parameter group suitable for accurately grasping the positional relationship such as distance is obtained by setting a parameter group so as to minimize the error of the positional relationship such as distance, instead of minimizing the positional error.

[0129] The calibration described in this embodiment can also be applied to calibration for three or more cameras. In this case, for example, it is advisable to increase the number of combinations of positional relationships for obtaining parameters by adding the number of distances between pins.

[0130] The configurations described in the above embodiment and modifications can be combined as appropriate as long as they are not mutually inconsistent.

[0131] The present disclosure discloses the following aspects.

[0132] A first aspect is a camera calibration device comprising: an image input unit to which first image data including a first reference point and second image data including a second reference point different from the first reference point are input; a position information input unit to which real space reference position information regarding a first real reference point corresponding to the first reference point in real space and a second real reference point corresponding to the second reference point in real space is input; and a processing unit that obtains a first parameter group for converting the first image data into data in a real coordinate system and a second parameter group for converting the second image data into data in the real coordinate system, wherein the processing unit determines the first parameter group and the second parameter group based on the first image data, the second image data, and the real space reference position information so that a positional relationship between a first transformed position obtained by transforming the first reference point of the first image data into the real coordinate system and a second transformed position obtained by transforming the second reference point of the second image data into the real coordinate system conforms to a positional relationship between the first real reference point and the second real reference point in real space.

[0133] According to this calibration device, the processing unit determines the first parameter group and the second parameter group so that the positional relationship between the first transformed position obtained by transforming the first reference point of the first image data into the real coordinate system and the second transformed position obtained by transforming the second reference point of the second image data into the real coordinate system matches the positional relationship between the first real reference point and the second real reference point in real space. Therefore, even if multiple cameras separately capture the first real reference point and the second real reference point, calibration can be performed. Therefore, even if the imaging ranges of the multiple cameras are different, the relative position error between the multiple cameras can be improved.

[0134] A second aspect is a camera calibration device related to the first aspect, wherein the processing unit determines the first parameter group and the second parameter group so that the distance between the first transformed position and the second transformed position matches the distance between the first real reference point and the second real reference point in real space.

[0135] In this case, the first parameter group and the second parameter group can be determined so that the distance between an object appearing in the first image data and an object appearing in the second image data can be obtained more accurately.

[0136] A third aspect is a camera calibration device according to the first or second aspect, wherein the first image data includes a plurality of the first reference points, the second image data includes a plurality of the second reference points, and the processing unit determines the first parameter group and the second parameter group based on a plurality of combinations of a positional relationship between the first transformation position and the second transformation position, and a positional relationship between the first real reference point corresponding to the first reference point and the second real reference point corresponding to the second reference point.

[0137] In this case, the first parameter group and the second parameter group are determined based on multiple combinations of the positional relationship between the first converted position and the second converted position and the positional relationship between the first actual reference point and the second actual reference point, so that the first parameter group and the second parameter group can be determined more appropriately.

[0138] A fourth aspect is a camera calibration device relating to any one of the first to third aspects, wherein the processing unit determines the first parameter group and the second parameter group so as to minimize an error between a positional relationship between the first transformed position and the second transformed position, and a positional relationship between the first real reference point and the second real reference point in real space.

[0139] This makes it possible to determine the first parameter group and the second parameter group so as to minimize the position error.

[0140] A fifth aspect is a camera calibration device relating to any one of the first to fourth aspects, wherein the processing unit performs a projective transformation defined by the first group of parameters on the first reference point of the first image data to obtain the first transformed position, and performs a projective transformation defined by the second group of parameters on the second reference point of the second image data to obtain the second transformed position.

[0141] In this way, calibration can be performed by determining the parameters used in the projective transformation.

[0142] A sixth aspect is a camera calibration device according to any one of the first to fifth aspects, wherein, before determining the first parameter group and the second parameter group, the processing unit determines a first initial parameter group so that the first transformed position matches the position of the first real reference point in real space, and determines a second initial parameter group so that the second transformed position matches the position of the second real reference point in real space, and determines the first parameter group and the second parameter group using the first initial parameter group and the second initial parameter group so that the positional relationship between the first transformed position and the second transformed position matches the positional relationship between the first real reference point and the second real reference point in real space.

[0143] In this way, a first initial parameter group is obtained so that the first transformed position matches the position of the first real reference point in real space, and a second initial parameter group is obtained so that the second transformed position matches the position of the second real reference point in real space. Then, the first parameter group and the second parameter group are determined using the first initial parameter group and the second initial parameter group, so that more appropriate parameter groups can be determined.

[0144] A seventh aspect is a camera calibration device relating to any one of the first to sixth aspects, wherein the first image data and the second image data are data captured in different imaging ranges.

[0145] In this manner, calibration can be performed when the first image data and the second image data capture different imaging ranges.

[0146] An eighth aspect is a camera calibration device relating to any one of the first to seventh aspects, further comprising a measurement object image input unit to which first measurement object image data in which a first measurement object is captured and second measurement object image data in which a second measurement object is captured are input, and the processing unit applies the first parameter group to the first measurement object image data and also applies the first parameter group to the second measurement object image data to obtain positional relationship data indicative of a positional relationship between the first measurement object and the second measurement object, and further comprises a display device that displays positional relationship information based on the positional relationship data.

[0147] This makes it easy to grasp the positional relationship between the first measurement object image data and the second measurement object image data by visually checking the display device.

[0148] A 9th aspect is a camera calibration device related to the 8th aspect, wherein the first measurement object is a first rail and the second measurement object is a second rail parallel to the first rail, and the processing unit may obtain data indicating a positional relationship between the first rail and the second rail as the positional relationship data.

[0149] This makes it possible to grasp the positional relationship between the first rail and the second rail as accurately as possible.

[0150] A tenth aspect is a camera calibration device related to the ninth aspect, wherein the processing unit obtains data indicating the distance between the first rail and the second rail as the positional relationship data, and the display device may display distance information between the first rail and the second rail as the positional relationship information.

[0151] This makes it possible to more accurately determine the distance between the first rail and the second rail and to display the distance.

[0152] A camera calibration system according to an eleventh aspect includes a camera calibration device according to any one of the first to tenth aspects, a first camera that images a first imaging range including the first real reference point and outputs the first image data, and a second camera that images a second imaging range including the second real reference point and outputs the second image data.

[0153] In this case, the range imaged by the first camera and the range imaged by the second camera can be calibrated.

[0154] A twelfth aspect is a camera calibration system relating to the eleventh aspect, wherein the first camera is supported on a railway vehicle in a state in which it can image a first rail, and the second camera is supported on the railway vehicle in a state in which it can image a second rail parallel to the first rail.

[0155] This makes it easier to accurately grasp the positional relationship between the first rail and the second rail on which the railway vehicle runs.

[0156] A thirteenth aspect is a camera calibration system according to the twelfth aspect, wherein the first camera images a first reference pin positioned on the first rail as the first real reference point, and the second camera images a second reference pin positioned on the second rail as the second real reference point.

[0157] In this way, by using the first and second reference pins, it is easy to accurately set the first and second actual reference points for the first and second rails, making it easy to accurately grasp the positional relationship between the first and second rails on which the railway vehicle runs.

[0158] A camera calibration method according to a fourteenth aspect includes acquiring image data including a first reference point and a second reference point different from the first reference point, acquiring real space reference position information regarding a first real reference point corresponding to the first reference point in real space and a second real reference point corresponding to the second reference point, and determining the first parameter group and the second parameter group so that a positional relationship between a first transformed position obtained by transforming the first reference point of the image data into a real coordinate system using a first parameter group and a second transformed position obtained by transforming the second reference point of the image data into the real coordinate system using a second parameter group conforms to a positional relationship between the first real reference point and the second real reference point in real space.

[0159] According to this calibration method, when the area near the first reference point and the area near the second reference point are different, that is, between the different ranges, the relative position error can be improved.

[0160] A 15th aspect is a camera calibration method relating to the 14th aspect, in which the image data obtained are first image data including the first reference point and second image data including the second reference point different from the first reference point.

[0161] This makes it possible to improve the relative position error between the imaging range of the first camera and the imaging range of the second camera.

[0162] The above description is illustrative in all respects, and the present invention is not limited thereto. It is understood that countless variations not illustrated can be assumed without departing from the scope of the present invention. [Explanation of symbols]

[0163] 10. Railroad cars 12A 1st rail 12B 2nd rail 18A 1st reference pin 18B Second reference pin 20, 120 Calibration System 30A 1st Camera 30B Second Camera 31A First image data 31B Second image data 50, 150 Calibration device 51, 151 Calibration processing section 52 processors 52a Initial parameter calculation processing section 52b Parameter calculation processing section 52c Rail positional relationship calculation processing unit 54 Storage device 54a Program 54b Reference point data 56D Actual coordinate data 56a Keyboard 56b Coordinate input device 58 Communication Interface 60 Display device 131A First measurement object image data 131B Second measurement object image data 154c Rail position data 162 Spacing Information D Distance E1 First imaging area E2 Second imaging range Pr Parameter Group R emission line

Claims

1. an image input unit to which first image data including a first reference point and second image data including a second reference point different from the first reference point are input; a position information input unit to which real space reference position information regarding a first real reference point corresponding to the first reference point in a real space and a second real reference point corresponding to the second reference point in a real space is input; a processing unit that obtains a first parameter group for converting the first image data into data in a real coordinate system, and a second parameter group for converting the second image data into data in the real coordinate system; Equipped with The processing unit determines the first parameter group and the second parameter group based on the first image data, the second image data, and the real space reference position information so that a positional relationship between a first transformed position obtained by transforming the first reference point of the first image data into the real coordinate system and a second transformed position obtained by transforming the second reference point of the second image data into the real coordinate system conforms to a positional relationship between the first real reference point and the second real reference point in real space.

2. 2. The camera calibration device according to claim 1, The processing unit determines the first group of parameters and the second group of parameters so that the distance between the first transformed position and the second transformed position matches the distance between the first real reference point and the second real reference point in real space.

3. 3. The camera calibration device according to claim 1, further comprising: The first image data includes a plurality of the first reference points; The second image data includes a plurality of the second reference points; The processing unit determines the first group of parameters and the second group of parameters based on multiple combinations of the positional relationship between the first transformation position and the second transformation position, and the positional relationship between the first real reference point corresponding to the first reference point and the second real reference point corresponding to the second reference point.

4. 3. The camera calibration device according to claim 1, further comprising: The processing unit determines the first parameter group and the second parameter group so as to minimize an error between a positional relationship between the first transformed position and the second transformed position and a positional relationship between the first real reference point and the second real reference point in real space.

5. 3. The camera calibration device according to claim 1, further comprising: The processing unit performs a projective transformation defined by the first group of parameters on the first reference point of the first image data to obtain the first transformed position, and performs a projective transformation defined by the second group of parameters on the second reference point of the second image data to obtain the second transformed position.

6. 3. The camera calibration device according to claim 1, further comprising: The processing unit includes: Before determining the first parameter set and the second parameter set, a first initial parameter set is obtained so that the first transformed position matches the position of the first real reference point in real space, and a second initial parameter set is obtained so that the second transformed position matches the position of the second real reference point in real space; A camera calibration device that determines the first parameter group and the second parameter group using the first initial parameter group and the second initial parameter group so that the positional relationship between the first transformed position and the second transformed position matches the positional relationship between the first real reference point and the second real reference point in real space.

7. 3. The camera calibration device according to claim 1, further comprising: A camera calibration device, wherein the first image data and the second image data are data captured in different imaging ranges.

8. 3. The camera calibration device according to claim 1, further comprising: A measurement object image input unit is further provided to which a first measurement object image data in which the first measurement object is captured and a second measurement object image data in which the second measurement object is captured are input, the processing unit applies the first parameter group to the first measurement object image data, and also applies the first parameter group to the second measurement object image data to obtain positional relationship data indicating a positional relationship between the first measurement object and the second measurement object; The camera calibration device further comprises a display device that displays positional relationship information based on the positional relationship data.

9. 9. The camera calibration device according to claim 8, the first measurement object is a first rail, the second measurement object is a second rail parallel to the first rail, The processing unit obtains data indicating the positional relationship between the first rail and the second rail as the positional relationship data.

10. 10. The camera calibration device according to claim 9, the processing unit obtains data indicating a distance between the first rail and the second rail as the positional relationship data, The display device displays distance information between the first rail and the second rail as the positional relationship information.

11. The camera calibration device according to claim 1 or 2, a first camera that captures an image of a first imaging range including the first real reference point and outputs the first image data; a second camera that captures an image of a second imaging range including the second real reference point and outputs the second image data; A camera calibration system comprising:

12. 12. A camera calibration system according to claim 11, comprising: the first camera is supported on a railway vehicle in a state in which the first camera can capture an image of a first rail; A camera calibration system, wherein the second camera is supported on the railway vehicle in a state in which it can image a second rail that is parallel to the first rail.

13. 13. A camera calibration system according to claim 12, comprising: the first camera captures an image of a first reference pin positioned on the first rail as the first real reference point; A camera calibration system, wherein the second camera images a second reference pin positioned on the second rail as the second real reference point.

14. acquiring image data including a first reference point and a second reference point different from the first reference point; acquiring real space reference position information relating to a first real reference point corresponding to the first reference point in a real space and a second real reference point corresponding to the second reference point; A camera calibration method comprising: determining the first parameter group and the second parameter group such that a positional relationship between a first transformed position obtained by transforming the first reference point of the image data into a real coordinate system using a first parameter group, and a second transformed position obtained by transforming the second reference point of the image data into the real coordinate system using a second parameter group, conforms to a positional relationship between the first real reference point and the second real reference point in real space.

15. A camera calibration method according to claim 14, comprising the steps of: A camera calibration method, comprising: acquiring, as the image data, first image data including the first reference point; and second image data including the second reference point different from the first reference point.

Citation Information

Patent Citations

  • JP2018‐179981A