Sensor calibration device and sensor calibration method

Abstract

To provide a technique for simply calibrating a position and an attitude of a sensor in a sensor calibration device.SOLUTION: A sensor calibration device includes: a reference line setting unit; a reference point setting unit which sets a reference point being a center of staying of a moving object on the reference line; a position measurement unit for measuring a position of the object; a storage unit which stores a position measurement result of the object moving on the reference line at each time point as a measurement point set; a staying point calculation unit which calculates estimated staying points of the object on the basis of a density distribution on the measurement points in the measurement point set; a line approximation unit for creating an approximate line of the measurement point set; and a parameter calibration unit which calibrates a position and attitude parameter by using the reference line, the reference point, the approximate line, and the staying point. The parameter calibration unit calibrates the position parameter by translating the approximate line so as to allow the estimated staying point to approach the reference point, and calibrates the attitude parameter by rotating the approximate line so as to allow the approximate line to approach the reference line.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a sensor calibration device and a sensor calibration method. [Background technology]

[0002] Conventionally, sensor calibration devices that calibrate the attitude and position of a sensor have been known (for example, see Patent Document 1). The sensor calibration device sets a reference within the detection range of the sensor and measures the position of an object relative to the reference, thereby calibrating the position and attitude parameters of the sensor. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent No. 5311365 Summary of the Invention [Problem to be solved by the invention]

[0004] However, depending on the installation environment of the sensor, it may be difficult to measure the position of an object relative to a reference. In such cases, it is necessary to acquire a large amount of information about the object's position in order to calibrate the position and orientation parameters of the sensor, which takes time.

[0005] The present invention has been made to solve the above-mentioned problems, and an object of the present invention is to provide a technique that can easily calibrate the position and orientation of a sensor in a sensor calibration device. [Means for solving the problem]

[0006] The present invention has been made to solve at least part of the above-mentioned problems, and can be realized in the following aspects.

[0007] (1) According to one aspect of the present invention, there is provided a sensor calibration device used to calibrate the position and orientation of a sensor, the sensor calibration device including: a reference line setting unit that sets a reference line; a reference point setting unit that sets a reference point on the reference line that will be the center of a position of an object moving in three-dimensional space; a position measurement unit that measures the position of the object using position and orientation parameters that are preset as parameters representing the temporary position and orientation of the sensor and an output from the sensor that detects the object in three-dimensional space; a storage unit that stores, as a set of measurement points, position measurement results of the object moving on the reference line at each time, measured by the position measurement unit; and a storage unit that stores the density of the measurement points in the set of measurement points. The system includes a stay point calculation unit that calculates an estimated stay point based on the distribution; a straight line approximation unit that creates an approximation line of the set of measurement points using the set of measurement points stored in the storage unit; and a parameter calibration unit that calibrates the position and orientation parameters using the reference line, the reference point, the approximation line, and the estimated stay point, wherein the parameter calibration unit calibrates the position parameters by translating the approximation line so that the estimated stay point approaches the reference point, and calibrates the orientation parameters by rotating the approximation line so that the approximation line approaches the reference line.

[0008] According to this configuration, an approximate line is created based on position measurement results at each time of an object moving on the reference line, for each of the reference line and a reference point set on the reference line, and an estimated stay point is calculated. The parameter calibration unit can calibrate the orientation parameters of the sensor by rotating the approximate line so that it approaches the reference line. The estimated stay point is calculated based on the density distribution of measurement points in a measurement point set that combines the position measurement results of the object at each time, and is the center of stay of the object moving using the reference point as a landmark. As a result, the parameter calibration unit can calibrate the position parameters by translating the approximate line so that the calculated estimated stay point approaches the reference point. As a result, the position and orientation parameters of the sensor can be calibrated by the object moving on the reference line within the detection range of the sensor, with the reference point as the center of stay. Therefore, the position and orientation parameters of the sensor can be calibrated even if the position measurement unit has few position measurement results of the object, allowing the position and orientation parameters to be calibrated in a short time.

[0009] (2) In the sensor calibration device of the above aspect, the reference point setting unit may set a plurality of the reference points on the reference line and set the reference point where the object stays the most among the plurality of set reference points as a representative reference point; the stay point calculation unit may calculate the plurality of estimated stay points based on a density distribution of the measurement points in the set of measurement points and set the estimated stay point with the highest density distribution of the measurement points in the set of measurement points as an estimated longest stay point among the plurality of estimated stay points; and the parameter calibration unit may calibrate the position parameters by translating the approximation line so that the estimated longest stay point approaches the representative reference point. According to this configuration, when setting a plurality of reference points on the reference line, the reference point setting unit sets the reference point where the object stays the most as the representative reference point. When calculating the plurality of estimated stay points based on the density distribution of the measurement points in the set of measurement points, the stay point calculation unit sets the estimated stay point with the highest density distribution of the measurement points as the estimated longest stay point. This allows the sensor position parameters to be calibrated by setting a representative reference point and an estimated longest stay point, even if multiple reference points can be set, and translating the approximate straight line so that the representative reference point and the estimated longest stay point approach each other.

[0010] (3) In the sensor calibration device of the above aspect, the line approximation unit may create an approximate line of the set of measurement points by performing principal component analysis on the estimated longest stay point. According to this configuration, the approximate line is created by principal component analysis performed on the estimated longest stay point. As a result, the approximate line passes through the estimated longest stay point, so that by translating the approximate line, the representative reference point and the estimated longest stay point can be superimposed, and by rotating the approximate line, the approximate line can be superimposed on the reference line. Therefore, the position and orientation parameters of the sensor can be reliably calibrated.

[0011] (4) In the sensor calibration device of the above aspect, the reference point setting unit may set a plurality of the reference points on the reference line, the stay point calculation unit may calculate the same number of estimated stay points as the number of the reference points set by the reference point setting unit, and the parameter calibration unit may calibrate the position parameters by translating the approximation line so that each of the plurality of estimated stay points approaches each of the corresponding plurality of reference points. According to this configuration, the reference point setting unit sets a plurality of reference points on the reference line. The stay point calculation unit calculates the plurality of estimated stay points based on a density distribution of the measurement points in the set of measurement points. The parameter calibration unit translates the approximation line so that each of the plurality of estimated stay points approaches each of the plurality of reference points, thereby calibrating the position parameters. This can improve the calibration accuracy of the position and orientation parameters of the sensor compared to a case in which the approximation line is translated so that one estimated stay point approaches one reference point.

[0012] (5) In the sensor calibration device of the above aspect, the parameter calibration unit may calibrate the position parameters by translating the approximation line so as to minimize the sum of absolute values ​​of distances between the plurality of reference points and the plurality of estimated stay points corresponding to each of the plurality of reference points. With this configuration, the parameter calibration unit calibrates the position parameters using a least absolute value method, which is less susceptible to the influence of outliers. This can further improve the calibration accuracy of the sensor's position parameters.

[0013] (6) In the sensor calibration device of the above aspect, the sensor calibration device may be connected to two sensors installed in different locations, the reference line setting unit sets intersecting reference lines for each of the two sensors, the reference point setting unit sets a reference point on each of the two reference lines, and the line approximation unit acquires information regarding the distance between the reference point and the intersection point on a reference broken line formed by the two intersecting reference lines and the intersection point of the two reference lines, and creates an approximated line for the set of measurement points using the information regarding the distance between the reference point and the intersection point. According to this configuration, the line approximation unit sets a reference broken line formed by the two reference lines set for each of the two sensors. For each of the two reference lines, the line approximation unit acquires the distance from the intersection point of the two reference lines to each reference point on the two reference lines, and calibrates the position and orientation parameters of the sensor using the position of the reference point and the distance from the reference point to the intersection point. This allows the approximated line to be determined by two points, the reference point and the intersection point, thereby improving the accuracy of the approximated line. Therefore, the accuracy of calibration of the position and orientation parameters of the sensor can be improved.

[0014] (7) According to another aspect of the present invention, there is provided a sensor calibration method used to calibrate the position and orientation of a sensor. a step of calibrating the position and orientation parameters using the reference line, the reference point, the reference point being the center of stay of an object moving in three-dimensional space; a step of measuring the position of the object using position and orientation parameters that are preset as parameters representing the temporary position and orientation of the sensor and an output from the sensor that detects the object in three-dimensional space; a step of storing position measurement results of the object moving on the reference line at each time by the position measurement unit as a measurement point set; a step of calculating an estimated stay point based on a density distribution of measurement points in the measurement point set; a step of creating an approximation line of the measurement point set using the stored measurement point set; and a calibration step of calibrating the position and orientation parameters using the reference line, the reference point, the approximation line, and the estimated stay point. In the calibration step, the position parameters are calibrated by translating the approximation line so that the estimated stay point approaches the reference point, and the orientation parameters are calibrated by rotating the approximation line so that the approximation line approaches the reference line. According to this configuration, the sensor calibration method creates an approximate line from position measurement results at each time of an object moving on a reference line with respect to the reference line. The sensor calibration method calculates an estimated stay point based on the density distribution of measurement points in a measurement point set that combines position measurement results of the object with respect to the reference point with respect to the reference point. This allows the position and orientation parameters of the sensor to be calibrated simply by the object moving on the reference line within the sensor's detection range, with the reference point as the center of stay, thereby enabling the position and orientation parameters to be calibrated in a short time.

[0015] The present invention can be realized in various forms, for example, in the form of a system including a sensor calibration device, a control method for these devices and systems, a computer program for causing these devices and systems to execute the sensor calibration method, a server device for distributing the computer program, a non-transitory storage medium on which the computer program is stored, etc. [Brief explanation of the drawings]

[0016] [Figure 1] 1 is a schematic diagram showing a schematic configuration of a sensor calibration device according to a first embodiment. [Figure 2] 1 is a flowchart illustrating a sensor calibration method. [Figure 3] FIG. 10 is a diagram illustrating a reference setting process. [Figure 4] FIG. 10 is a diagram illustrating a stay point calculation process. [Figure 5] FIG. 1 is a first diagram illustrating a parameter calibration process. [Figure 6] FIG. 2 is a second diagram illustrating the parameter calibration process. [Figure 7] FIG. 10 is a third diagram illustrating the parameter calibration process. [Figure 8] 10A and 10B are diagrams illustrating how to overlap a reference line and an approximation line. [Figure 9] FIG. 10 is a diagram illustrating a method for creating an approximate straight line in a first comparative example. [Figure 10] FIG. 10 is a first diagram illustrating a method for creating an approximate straight line in a second comparative example. [Figure 11] FIG. 10 is a second diagram illustrating a method for creating an approximate straight line in the second comparative example. [Figure 12] FIG. 3 is a diagram illustrating a method for creating an approximate straight line according to the first embodiment. [Figure 13] 10A to 10C are diagrams illustrating a sensor calibration method according to a second embodiment. [Figure 14] FIG. 10 is a schematic diagram showing a schematic configuration of a sensor calibration device according to a third embodiment. [Figure 15] FIG. 2 is a diagram illustrating the positional relationship between two monocular cameras. DETAILED DESCRIPTION OF THE INVENTION

[0017] First Embodiment FIG. 1 is a schematic diagram showing the overall configuration of a sensor calibration device 1 according to a first embodiment. The sensor calibration device 1 according to this embodiment is a device that calibrates a sensor that is fixed in one location for use. The sensor calibration device 1 is used, for example, when correcting position parameters representing the position and attitude parameters representing the attitude of a sensor that is temporarily installed in conjunction with the establishment of an event. The sensor calibration device 1 according to this embodiment calibrates one monocular camera 5 as a sensor. The monocular camera 5 is installed, for example, near a road on which vehicles travel, continuously captures images of vehicles traveling on the road and pedestrians 6 (see FIG. 1) on the side of the road, and outputs the captured images (multidimensional information). The sensor calibration device 1 includes a measuring instrument 10 and a calibration processor 20. Note that, in this embodiment, the sensor is a monocular camera 5; however, the type of sensor is not limited to this and may be a stereo camera, radar, LiDAR (laser radar), sonar, or the like.

[0018] The measuring instrument 10 includes a position measurement unit 11 and a reference setting unit 12. The measuring instrument 10 is electrically connected to the monocular camera 5, and an image captured by the monocular camera 5 is input.

[0019] The position measurement unit 11 measures the position of an object (pedestrian 6) included in an image captured by the monocular camera 5. Provisional position and orientation parameters including both position parameters and orientation parameters that are set in advance as parameters representing the provisional position and orientation of the monocular camera 5 are input to the position measurement unit 11. The position measurement unit 11 measures the position of the moving pedestrian 6 from the position and orientation parameters and the image captured by the monocular camera 5. The position measurement unit 11 outputs the measurement results and the provisional position and orientation parameters to the calibration processor 20, which is electrically connected to the position measurement unit 11.

[0020] The reference setting unit 12 sets a reference line and a reference point on the captured image, which is the output information of the monocular camera 5. The reference line is a line that can be set arbitrarily, and specifically, is set on the image based on an object arranged in a line, such as a row of guiding blocks for the visually impaired on a sidewalk or a white line dividing a sidewalk into a pedestrian lane and a bicycle lane, that is included in the image captured by the monocular camera 5. The reference point is a point set on the reference line in the image captured by the monocular camera 5, and indicates the position of an object that is preset as the center of pedestrian 6's stay. Specifically, the reference point is set on the image corresponding to a location in the image captured by the monocular camera 5 where an object that is easily recognizable from the pedestrian's perspective is installed, such as a fixed bus stop sign or a warning block among guiding blocks for the visually impaired. Note that the objects that serve as the reference for setting the reference line and the objects that are preset for setting the reference point are not limited to these. For example, the object used as the reference for setting the reference line may be a white line that distinguishes lanes on a road, or a curb that distinguishes the roadway from the sidewalk, and the object that is set in advance as the reference point may be a bench or a trash can on the sidewalk.

[0021] The calibration processor 20 includes a storage unit 21 and a CPU 22. The calibration processor 20 calibrates the position and orientation parameters of the monocular camera 5 using the position measurement results of the object measured by the position measurement unit 11, the provisional position and orientation parameters in the measurement, and the reference line set by the reference setting unit 12. The calibrated position and orientation parameters are output by the calibration processor 20 to the position measurement unit 11 of the measuring instrument 10, which is electrically connected to the calibration processor 20.

[0022] The storage unit 21 is configured with a hard disk, a flash memory, a memory card, etc. The storage unit 21 stores the position measurement results of an object moving on a reference straight line at each time measured by the position measurement unit 11 as a set of measurement points. The storage unit 21 also stores various calculation results in the CPU 22. The storage unit 21 outputs this stored information to the CPU 22 as appropriate.

[0023] The CPU 22 controls each part of the sensor calibration device 1, including the measuring instrument 10, by loading a computer program stored in the ROM into the RAM and executing it. In addition, the CPU 22 functions as a stay point calculation unit 22a, a straight line approximation unit 22b, and a parameter calibration unit 22c, and calculates an estimated stay point and creates an approximated line from the position measurement results of a moving object measured by the position measurement unit 11, and calibrates position and orientation parameters from the relationship between the estimated stay point and the reference point, and the relationship between the approximated line and the reference line.

[0024] The stay point calculation unit 22a calculates an estimated stay point based on the density distribution of the measurement points in the set of measurement points stored in the storage unit 21. Specifically, the stay point calculation unit 22a calculates a point in the set of measurement points stored in the storage unit 21 where the density distribution of the measurement points is relatively high, and sets this point as the estimated stay point. The detailed function of the stay point calculation unit 22a will be described later.

[0025] The straight line approximation unit 22b creates an approximate straight line of the set of measurement points using the set of measurement points of the object position measurement results stored in the storage unit 21. The detailed function of the straight line approximation unit 22b will be described later.

[0026] The parameter calibration unit 22c uses the approximate line created by the line approximation unit 22b, the reference line, and the provisional position and orientation parameters to calibrate the position and orientation parameters of the monocular camera 5. The detailed functions of the parameter calibration unit 22c will be described later.

[0027] FIG. 2 is a flowchart illustrating a sensor calibration method. Next, the sensor calibration method of this embodiment will be described. The sensor calibration method of this embodiment is performed by the sensor calibration device 1, for example, when the monocular camera 5 is installed in a predetermined location. The sensor calibration method is performed as a work (on-site correction work) to check the installation position of the monocular camera 5 and the orientation of the monocular camera 5, which affect the direction of the optical axis of the monocular camera 5, and correct these during installation of the monocular camera 5. Such on-site correction work is performed to reduce false detections (determining that a measurement target object is present within the detectable range of the monocular camera 5 when it is not) and non-detections (determining that a measurement target object is not present within the detectable range of the monocular camera 5 when it is not). Therefore, on-site calibration is required after installation of the monocular camera 5. Note that measuring the position and orientation of the monocular camera 5 itself or calibration using a calibration board cannot immediately accommodate this on-site correction work. Furthermore, position and orientation estimation using other sensors, such as positioning satellite reception or a gyroscope, is not suitable for the monocular camera 5 because it cannot properly estimate the orientation when the camera is stationary.

[0028] First, a reference line and a reference point are set (reference setting process: step S1). In step S1, the reference setting unit 12 sets objects arranged in a line at the site as a reference line in the coordinate system of the position measurement unit 11, and sets a reference point on the set reference line. In this embodiment, the reference setting unit 12 sets a row 7 of guiding blocks for visually impaired people on the sidewalk as the reference line (see FIG. 1). The reference point set by the reference setting unit 12 is preferably a landmark for pedestrians 6 moving along the row 7 of guiding blocks for visually impaired people. In this embodiment, the reference points are multiple points (not shown) including a location 81 (see FIG. 1) where a sign 8 installed along the row 7 of guiding blocks for visually impaired people is fixed, and the location 81 where the sign 8 is fixed (hereinafter referred to as the "location of the sign") is set as a representative reference point. Note that the objects that serve as the reference line and the reference points are not limited to these.

[0029] Fig. 3 is a diagram illustrating the reference setting step. Fig. 3 shows a portion of an image captured by the monocular camera 5. In the reference setting step, the reference setting unit 12 sets a reference line Ls on the x-axis of the coordinate system of the image shown in Fig. 3, and sets a representative reference point Ps0 on the reference line Ls that corresponds to the location 81 of the sign.

[0030] Next, the position of pedestrian 6 is measured (position measurement process: step S2). In step S2, position measurement unit 11 measures the position of pedestrian 6 moving on row 7 of guiding blocks for visually impaired persons, using pre-set provisional position and orientation parameters of monocular camera 5 and the output of monocular camera 5. In this embodiment, when pedestrian 6 walks on row 7 of guiding blocks for visually impaired persons, he moves so that the time spent at location 81 of sign serving as a representative reference point is longer than the time spent at other locations.

[0031] Here, the provisional position and orientation parameters of the monocular camera 5 in the sensor calibration method of this embodiment are the coordinate values ​​s t (vector notation) and azimuth angle a t and elevation angle e t That is, the position measurement unit 11 calculates the provisional position parameter s t (vector notation) and the attitude parameter e t , a t The position of the pedestrian 6 output by the monocular camera 5 with the above settings set is measured. The position measurement unit 11 outputs the position measurement result of the pedestrian 6 to the storage unit 21. Note that when a character shown in the text is a vector, "(vector notation)" is written after the target character.

[0032] Next, the position measurement results at each time of the pedestrian 6 moving on the row 7 of guiding blocks for visually impaired persons, measured in step S2, are stored as a set of measurement points (storage process: step S3). The memory unit 21 stores the position measurement results of the pedestrian 6 output by the position measurement unit 11 as a set of measurement points in three-dimensional space. The memory unit 21 also stores the position information of the row 7 of guiding blocks for visually impaired persons set by the reference setting unit 12 as position information of the reference straight line.

[0033] Next, an estimated stay point is calculated from the position measurement result of the pedestrian 6 stored in step S3 (stay point calculation step: step S4). Using the measurement point set stored in the storage unit 21, the stay point calculation unit 22a calculates a position where the pedestrian 6 stayed for a relatively long time as an estimated stay point.

[0034] FIG. 4 is a diagram illustrating the staying point calculation step and shows an example of a measurement point set. FIG. 4 shows a two-dimensional plane (xy plane) on which the position measurement results of the pedestrian 6 stored in the memory unit 21 are plotted. Each of the multiple circles on the xy plane shown in FIG. 4 indicates the position measurement results of the pedestrian 6 at each time, and indicates the position (measurement point M) of the pedestrian 6 measured at regular time intervals within a predetermined period. In other words, the memory unit 21 stores a set of position measurement results of the pedestrian 6 at each time (measurement point set Gm). For ease of explanation, FIG. 4 shows the measurement point set on a two-dimensional plane, and also shows the reference line Ls and the representative reference point Ps0 set in the reference setting step.

[0035] Here, the method for calculating the estimated stay points in step S4 will be described. In this embodiment, the stay point calculation unit 22a calculates the estimated stay points using a known hierarchical clustering shortest distance method, which aggregates the measurement points M included in the measurement point set Gm into groups called "clusters" in order of distance. The stay point calculation unit 22a terminates clustering when the maximum distance within a cluster reaches a threshold value that takes into account the variance of position measurements, for example, 0.1 m, and selects clusters in order of the number of points within the cluster until the number of points within the cluster becomes the same as the reference point. In this embodiment, for each cluster, the median value of the positions of each point within the cluster is set as the estimated stay point Pe, and the coordinate point with the highest distribution density of measurement points M among the multiple clusters selected in the measurement point set Gm is set as the estimated longest stay point LPe.

[0036] Next, in step S5, an approximation line of the set of measurement points is created using the set of measurement points stored in the storage unit 21 (straight line approximation step: step S5). The straight line approximation unit 22b uses the set of measurement points of the position measurement results of the pedestrian 6 stored in the storage unit 21 to calculate the position parameter s of the monocular camera 5. t (vector notation) and the attitude parameter e t , a t Create an approximate line to calibrate the

[0037] In the straight line approximation step, the straight line approximation unit 22b first creates an approximated straight line candidate using a method that combines the principal component analysis method and the RANSAC method. In this embodiment, the straight line approximation unit 22b performs principal component analysis using the coordinate values ​​of the estimated longest stay point, instead of the usual principal component analysis that uses the average of the coordinate values ​​of multiple measurement points, so that the approximated straight line always passes through the estimated longest stay point, and creates an approximated straight line candidate. Specifically, the coordinate value matrix of each of the multiple measurement points is defined as matrix P, and the coordinate values ​​of the estimated longest stay point are defined as coordinate values ​​p0 t (vector notation) and the number of measurement points is n, the following formula (1) is subjected to eigenvalue decomposition to find the maximum eigenvalue and the first eigenvector fev (vector notation) corresponding to the maximum eigenvalue. Note that T in formula (1) is a transpose operator.

number

[0038] In the straight-line approximation step, the straight-line approximation unit 22b determines whether to adopt a combination of the maximum eigenvalue and the first eigenvector fev (vector representation) calculated by the principal component analysis described above, based on the square root of the maximum eigenvalue. The straight-line approximation unit 22b adopts the created approximated straight-line candidate if the square root of the maximum eigenvalue is equal to or greater than a threshold value set based on the length of a line segment included in the position-measurable range of the reference straight line, and rejects the created approximated straight-line candidate if the square root is less than the threshold value. Specifically, assuming that the length of a line segment included in the position-measurable range of the reference straight line is 4 m and the threshold is 1.8 m, half the measurable range of 3.6 m that takes detection omissions into account, the straight-line approximation unit 22b adopts the created approximated straight-line candidate if the length is 1.8 m or greater, and rejects the created approximated straight-line candidate if the length is less than 1.8 m. The straight-line approximation unit 22b repeatedly calculates the combination of the maximum eigenvalue and the first eigenvector fev (vector representation) calculated by the principal component analysis described above and makes a determination based on the square root of the maximum eigenvalue until the number of times that the combination has not been rejected reaches a predetermined number. When the number of times that the eigenvalues ​​have not been rejected reaches a preset number, the straight line approximation unit 22b selects, as an approximate straight line, a straight line that has the smallest approximation error from among the combinations of the maximum eigenvalue that has not been rejected and the first eigenvector fev (vector notation). The straight line approximation unit 22b stores the maximum eigenvalue and the first eigenvector fev (vector notation) of the selected approximate straight line in the storage unit 21.

[0039] 2, after step S5, the position parameters and attitude parameters of the monocular camera 5 are calibrated using the reference line set in step S1 and the approximate line selected in step S5 (parameter calibration step: step S6). In this embodiment, the parameter calibration unit 22c calibrates the attitude parameters a and e, and then calibrates the position parameter s.

[0040] FIG. 5 is a first diagram illustrating the parameter calibration process. Here, a method for calibrating the position and orientation parameters in the parameter calibration process will be described. FIG. 5(a) shows an xy plane in a three-dimensional space, and shows projection points Mxy obtained by projecting each of a plurality of measurement points M onto the xy plane, and a projected approximate line Laxy obtained by projecting the approximate line La onto the xy plane. FIG. 5(a) also shows a provisional position parameter s of the monocular camera 5 on the xy plane. t (vector notation) and the azimuth angle a t Here, the provisional azimuth angle is, for example, azimuth angle a t = 15 degrees. Figure 5(b) shows the xz plane in three-dimensional space, and shows the projection points Mxz obtained by projecting each of the multiple measurement points M onto the xz plane, and the projected approximate line Laxz obtained by projecting the approximate line La onto the xz plane. Figure 5(b) also shows the provisional position parameters s of the monocular camera 5 on the xz plane. t (vector notation) and the elevation angle e, which represents the provisional attitude t Here, the provisional elevation angle is, for example, the elevation angle e t Assume =0 degrees.

[0041] In step S1, the reference line Ls is set on the x-axis (see FIG. 5). At this time, each of the multiple measurement points M is a point in a coordinate system centered on the monocular camera 5 (a coordinate system in which the monocular camera 5 is at the origin of the three-dimensional space and the axis of the front of the monocular camera 5 is the x-axis, and the x-axis is the optical axis of the monocular camera 5). t , then the azimuth angle a t After rotating with , the position of the monocular camera 5 is set to the coordinate value s t (Vector notation) is obtained by translating it.

[0042] In step S5, the coordinate value p0 of the estimated longest stay point is t (vector notation) and the first eigenvector fev (vector notation) of the approximate line La are stored in the storage unit 21. The parameter calibration unit 22c calculates these pieces of information and the two coordinate values ​​p0 t(vector notation) and coordinate value p1 t (vector notation) and the calibrated elevation angle e c and azimuth angle a c Here, the coordinate value p1 t is the coordinate value p0 t (vector notation) and the first eigenvector fev (vector notation) of the approximate line La, it is expressed by the following equation (2).

number

number

[0043] The azimuth angle a is defined as equation (4) by rotating the x-axis counterclockwise as viewed from above around the z-axis (axis from bottom to top).

number

[0044] The elevation angle e after calibration that you want to derive c and azimuth angle a c are the elevation and azimuth angles at which the projection approximation lines Laxy and Laxz become parallel to the x-axis, which is the reference line Ls, when the influence of the provisional position and orientation parameters is cancelled out and the image is rotated again by the elevation and azimuth angles. t The coordinate value p0 after calibration is calculated by calibrating only the attitude parameters (vector notation). p (vector notation) and coordinate value p1 t (Vector notation) After calibration, only the attitude parameters are calibrated. p(vector notation) and the difference between the y and z components will be 0 simultaneously. p (Vector notation) is expressed by equation (5), and the coordinate value p1 after calibration p (Vector notation) is expressed by equation (6).

number

[0045] Elevation angle after calibration e c and azimuth angle a c To derive this, we use the simultaneous equations (7) and (8) below to calculate the elevation angle difference Δe and the azimuth angle a c Equation (7) is the equation for the y component, and equation (8) is the equation for the z component. The elevation angle difference Δe is (e c -e t ) and the elevation angle e t is a known value as the provisional elevation angle, so once the elevation angle difference Δe is found, the elevation angle e t can be obtained.

number

[0046] By solving the simultaneous equations consisting of equations (7) and (8), the azimuth angle a is obtained as shown in equations (9) and (10). c Using these equations, the azimuth angle a after calibration is calculated as c and elevation angle e c Calculate.

number

[0047] 6A and 6B are second diagrams illustrating the parameter calibration process. The diagrams shown in Fig. 6A and 6B correspond to Fig. 5A and Fig. 5B, respectively, and show the azimuth angle a after calibration calculated using Equation (9) and Equation (10). c and elevation angle e cThe projected approximate lines Laxy and Lxxz are corrected using the above equation. As shown in Figure 6, each of the projected approximate lines Laxy and Laxz is parallel to the reference line Ls. In concrete terms, the azimuth angle a is calibrated to be between 15 degrees and 6.1 degrees, and the elevation angle e is calibrated to be between 0 degrees and -3 degrees.

[0048] 7 is a third diagram illustrating the parameter calibration process. Next, the parameter calibration unit 22c calibrates the position parameters. In the calibration of the position parameters, the coordinate value p0 after calibration in which only the attitude parameters are calibrated is calculated. p The monocular camera 5 is translated so that the y-component and the z-component of the monocular camera 5 become 0. For example, in the case of FIG. 5(a) where only the azimuth angle a is calibrated, the projected approximate line Laxy is moved in the negative direction of the y-axis to coincide with the reference line Ls (see FIG. 5(a)). Also, in the case of FIG. 6(b) where only the elevation angle e is configured, the projected approximate line Laxz is moved in the positive direction of the z-axis to coincide with the reference line Ls (see FIG. 7(a)). As a result, the approximate line La coincides with the x-axis, which is the reference line Ls. In other words, the position s of the monocular camera 5 after translation is c can be expressed by equation (11).

number

[0049] FIG. 8 is a diagram illustrating how to superimpose the reference line and the approximate line in the sensor calibration method of this embodiment. Here, the calibration of the position and orientation parameters in the above-mentioned sensor calibration method will be explained using the positional relationship between the reference line Ls, the representative reference point Ps0, the estimated longest stay point LPe, and the approximate line La0. In the sensor calibration method of this embodiment, first, the line approximation unit 22b creates the approximate line La0 that passes through the estimated longest stay point LPe. Next, the parameter calibration unit 22c calculates the position and orientation parameters e t , a t8, thereby rotating the approximate line La0 (arrow R1 shown in FIG. 8), and creating a virtual line La1 parallel to the reference line Ls. Next, the parameter calibration unit 22c translates the virtual line La1 so that the virtual line La1 and the reference line Ls approach each other, creating a virtual line La2 (arrow T1 shown in FIG. 8). Finally, the parameter calibration unit 22c translates the virtual line La2 so that the estimated longest stay point LPe on the virtual line La2 approaches the representative reference point Ps0 (arrow T2 shown in FIG. 8). In this way, the parameter calibration unit 22c rotates the approximate line La0 so that the approximate line La0 is parallel to the reference line Ls, and translates the approximate line La0 so that the estimated longest stay point LPe approaches the representative reference point Ps0, thereby calibrating the attitude parameters and the position parameters. Note that "approaching" here also means bringing the estimated longest stay point LPe closer to the representative reference point Ps0, thereby overlapping the estimated longest stay point LPe and the representative reference point Ps0.

[0050] As described above, in step S6, the parameter calibration unit 22c calibrates the orientation parameters and then the position parameters. The parameter calibration unit 22c outputs the calibrated position and orientation parameters to the measuring instrument 10. The measuring instrument 10 uses the input calibrated position and orientation parameters to perform subsequent position measurements of the object.

[0051] Here, the effect of the sensor calibration device 1 of this embodiment, particularly with regard to the accuracy of the approximated line created by the linear approximation unit, will be explained by comparing it with several comparative examples. The first comparative example is an example of creating an approximated line using principal component analysis, and the second comparative example is an example of creating an approximated line using a method that combines principal component analysis and RANSAC.

[0052] 9 is a diagram illustrating a method for creating an approximate line in a first comparative example. The circles in FIG. 9 represent multiple measurement points M. In the principal component analysis method, an approximate line L01 is created using all of the measurement points M. For this reason, the approximate line L01 is subject to the influence of outliers such as the measurement point Mx within the ellipse R10, and therefore tends to have a large slope and deviate from the reference line Ls.

[0053] FIG. 10 is a first diagram illustrating a method for creating an approximate line in a second comparative example. In the second comparative example, first, in the first step, some of all measurement points M are randomly selected. For example, as shown in FIG. 10, multiple thick circles M2 are selected. A provisional approximate line L02 is obtained for these selected measurement points M2 using principal component analysis. Next, in the second step, of the measurement points M that were not selected, points whose distance from the provisional approximate line L02 exceeds a threshold are considered outliers, and the approximate line and approximation error are obtained for points other than the outliers.

[0054] FIG. 11 is a second diagram illustrating a method for creating an approximate line in the second comparative example. In the second comparative example, the first and second steps are repeated as a third step, and the line that minimizes the approximation error is taken as the approximate line L12. The plotted point "x" shown in FIG. 11 is the measurement point M that was deemed an outlier in the second step. The approximate line L12 has a steeper slope than the first comparative example, and also deviates more from the reference line Ls.

[0055] FIG. 12 is a diagram illustrating a method for creating an approximate line in the first embodiment. FIG. 14 shows an approximate line L03 created by the above-described method for creating an approximate line using the same set of measurement points as those used in the first and second comparative examples. As shown in FIG. 12, the approximate line L03 created by this embodiment has a smaller slope than the approximate line L01 of the first comparative example, and it is clear that a reasonable result close to the reference line Ls can be obtained. Note that in FIGS. 9, 11, and 13, the solid lines indicated by the symbols L01, L12, and L03 represent line segments representing portions of the approximate lines obtained by the respective methods, and the lengths of the line segments are proportional to the standard deviations obtained from the orthogonal projections of each measurement point M onto the approximate line.

[0056] According to the sensor calibration device 1 of the present embodiment described above, an approximate line La is created from the position measurement results at each time of a pedestrian 6 moving on the reference line Ls, for each of the reference line Ls and the representative reference point Ps0 set on the reference line Ls, and an estimated longest stay point LPe is calculated. The parameter calibration unit 22c can calibrate the attitude parameters a and e of the monocular camera 5 by rotating the approximate line La so that it approaches the reference line Ls. Furthermore, the estimated longest stay point LPe is calculated based on the density distribution of the measurement points M in the measurement point set Gm, which combines the position measurement results of the pedestrian 6 at each time, and is the center of stay of the pedestrian 6 moving using the representative reference point Ps0 as a landmark. As a result, the parameter calibration unit 22c can calibrate the position parameter s t (vector notation) can be calibrated. As a result, the position and orientation parameters of the monocular camera 5 can be calibrated by the pedestrian 6 moving on the reference line Ls within the detection range of the monocular camera 5, with the representative reference point Ps0 as the center of movement. Therefore, even if the position measurement unit 11 has only a few position measurement results for the pedestrian 6, the position and orientation parameters can be calibrated in a short time.

[0057] Furthermore, according to the sensor calibration device 1 of this embodiment, when setting multiple reference points on the reference line Ls, the reference setting unit 12 sets the representative reference point Ps0 where the pedestrian 6 stays the most to the sign location 81. When calculating multiple estimated stay points Pe based on the density distribution of the measurement points in the measurement point set Gm, the stay point calculation unit 22a sets the estimated stay point Pe with the highest density distribution of the measurement points to the estimated longest stay point LPe. As a result, even when multiple reference points can be set, by setting the sign location 81 and the estimated longest stay point LPe, it is possible to calibrate the position parameters of the monocular camera 5 by translating the approximation line La so that the sign location 81 and the estimated longest stay point LPe approach each other.

[0058] Furthermore, according to the sensor calibration device 1 of this embodiment, the approximate line La is created by principal component analysis performed on the estimated longest stay point LPe. As a result, the approximate line La passes through the estimated longest stay point LPe, so that by translating the approximate line La, the location 81 of the sign and the estimated longest stay point LPe can be superimposed, and by rotating the approximate line La, the approximate line La can be superimposed on the reference line Ls. Therefore, the position and orientation parameters of the monocular camera 5 can be reliably calibrated.

[0059] Furthermore, according to the sensor calibration method of this embodiment, an approximate line La is created from the position measurement results at each time of the pedestrian 6 moving on the reference line Ls with respect to the reference line Ls. The sensor calibration method calculates an estimated stay point Pe based on the density distribution of measurement points in a measurement point set Gm that combines the position measurement results of the pedestrian 6 at each time with respect to the representative reference point Ps0. As a result, the position and orientation parameters of the monocular camera 5 can be calibrated simply by the pedestrian 6 moving on the reference line Ls within the detection range of the monocular camera 5, with the representative reference point Ps0 as the center of stay, thereby enabling the position and orientation parameters to be calibrated in a short time.

[0060] Second Embodiment 13 is a diagram illustrating a sensor calibration method according to the second embodiment. The sensor calibration method according to the sensor calibration device of the second embodiment differs from the sensor calibration method according to the first embodiment (FIG. 1) in that the position and orientation parameters of the sensor are calibrated using a plurality of estimated stay points selected in the stay point calculation step.

[0061] In the sensor calibration method using the sensor calibration device of the second embodiment, a reference setting step sets multiple reference points Ps1, Ps2, Ps3, and Ps4 on a reference line Ls, as shown in Fig. 13. As in the first embodiment, the multiple reference points Ps1, Ps2, Ps3, and Ps4 are preferably landmarks for a pedestrian 6 moving along the row 7 of guiding blocks for the visually impaired, and may be warning blocks included in the row 7 of guiding blocks for the visually impaired or manholes on the road. In the position measurement step, a pedestrian 6 moving along the row 7 of guiding blocks for the visually impaired moves so that the time spent at each of the multiple reference points Ps1, Ps2, Ps3, and Ps4 is longer than the time spent at other positions.

[0062] In the stay point calculation step following the position measurement step, the stay point calculation unit 22a calculates positions where the pedestrian 6 stayed for a relatively long time as estimated stay points using the set of measurement points stored in the storage unit 21. In this embodiment, the same number of clusters as the number of reference points set in the reference setting step are selected using a hierarchical clustering shortest distance method. In this embodiment, for each cluster, the median of the cluster is set as the estimated stay points Pe1, Pe2, Pe3, and Pe4.

[0063] In the parameter calibration process, similar to the first embodiment, the posture parameters are calibrated, and then the approximate line Lp is translated so that the estimated staying points Pe1, Pe2, Pe3, and Pe4 approach each of the multiple reference points Ps1, Ps2, Ps3, and Ps4, respectively, to calibrate the position parameters. In this embodiment, the position parameters are calibrated using the least absolute value method, which is more resistant to outliers than the least squares method. Specifically, the approximate line Lp is translated so that the sum of the absolute values ​​of the distances between the estimated staying points Pe1, Pe2, Pe3, and Pe4 corresponding to each of the reference points Ps1, Ps2, Ps3, and Ps4 and the reference points Ps1, Ps2, Ps3, and Ps4 is minimized.

[0064] According to the sensor calibration device of this embodiment described above, the reference setting unit 12 sets multiple reference points Ps1, Ps2, Ps3, and Ps4 on the reference line Ls. The stay point calculation unit 22a calculates multiple estimated stay points Pe1, Pe2, Pe3, and Pe4 based on the density distribution of measurement points M in the measurement point set Gm. The parameter calibration unit 22c translates the approximate line La to calibrate the position parameters so that each of the multiple estimated stay points Pe1, Pe2, Pe3, and Pe4 approaches each of the multiple reference points Ps1, Ps2, Ps3, and Ps4. This improves the calibration accuracy of the position and orientation parameters of the monocular camera 5 compared to translating the approximate line La so that one estimated stay point approaches and overlaps with one reference point.

[0065] Furthermore, according to the sensor calibration device of this embodiment, the parameter calibration unit 22c calibrates the position parameters using the least absolute value method, which is less susceptible to the influence of outliers, thereby further improving the calibration accuracy of the position parameters of the monocular camera 5.

[0066] <Third embodiment> 14 is a schematic diagram showing the general configuration of a sensor calibration device according to the third embodiment. The sensor calibration device according to the third embodiment differs from the sensor calibration device according to the first embodiment (FIG. 1) in that it includes two measuring instruments and calibrates the position and orientation parameters of two sensors.

[0067] The sensor calibration device 3 of this embodiment includes two measuring instruments 10a and 10b and a calibration processor 30. Each of the two measuring instruments 10a and 10b of this embodiment is connected to a monocular camera 5a and 5b, respectively. The two measuring instruments 10a and 10b and the calibration processor 30 are connected, for example, by wireless communication. The calibration processor 30 receives the measurement details and setting details of each of the two measuring instruments 10a and 10b, and can output calibrated position and orientation parameters to each of the two measuring instruments 10a and 10b.

[0068] Measuring instrument 10a includes a position measurement unit 11a and a reference setting unit 12a. Measuring instrument 10a is electrically connected to monocular camera 5a and receives as input an image captured by monocular camera 5a. Position measurement unit 11a measures the position of pedestrian 6a. Reference setting unit 12a sets a reference line 7a and a reference point 81a on reference line 7a on the image captured by monocular camera 5a. Measuring instrument 10b includes a position measurement unit 11b and a reference setting unit 12b. Measuring instrument 10b is electrically connected to monocular camera 5b and receives as input an image captured by monocular camera 5b. Position measurement unit 11b measures the position of pedestrian 6b. Reference setting unit 12b sets a reference line 7b and a reference point 81b on reference line 7b on the image captured by monocular camera 5b.

[0069] FIG. 15 is a diagram illustrating the positional relationship between two monocular cameras 5a and 5b. FIG. 15 shows a map of an urban area 9 as an example. The two monocular cameras 5a and 5b configured with the sensor calibration device 3 are installed at separate locations, for example, in one block 9a in the urban area 9. In this embodiment, two reference setting units 12a and 12b corresponding to the two monocular cameras 5a and 5b respectively set reference lines 7a and 7b so that they intersect. Here, the intersection of the reference lines 7a and 7b is referred to as intersection point 7c (see FIG. 15).

[0070] The calibration processor 30 includes a storage unit 31 and a CPU 32. The calibration processor 30 calibrates the position and orientation parameters of the monocular cameras 5a and 5b using the position measurement results of the two pedestrians 6a and 6b measured by the measuring instruments 10a and 10b, respectively, the provisional position and orientation parameters, and the reference lines 7a and 7b and reference points 81a and 81b set by the reference setting unit 12.

[0071] The storage unit 31 stores, as a set of measurement points, the position measurement results of the pedestrians 6a and 6b moving on the reference lines 7a and 7b measured by the two position measurement units 11a and 11b at each time. The storage unit 31 also stores various calculation results of the CPU 32 and outputs the stored information to the CPU 32 as appropriate.

[0072] The CPU 32 functions as a stay point calculation unit 32a, a straight line approximation unit 32b, and a parameter calibration unit 32c, and controls each unit of the sensor calibration device 3, including the two measuring instruments 10a and 10b. The CPU 32 calculates estimated stay points from the position measurement results of the pedestrians 6a and 6b measured by the two position measurement units 11a and 11b, creates approximate lines, and calibrates position and orientation parameters from the relationship between the estimated stay points and the reference point and the relationship between the approximate line and the reference line.

[0073] The stay point calculation unit 32a has the same function as the stay point calculation unit 22a in the first embodiment. The stay point calculation unit 32a calculates estimated stay points using two sets of measurement points that are the position measurement results by the two position measurement units 11a and 11b.

[0074] The straight line approximation unit 32b includes a candidate creation unit 321b, a determination unit 322b, and a selection unit 323b. The candidate creation unit 321b performs principal component analysis using the coordinate values ​​of the reference points of each of the two measurement point sets to create approximated line candidates. The determination unit 322b determines whether to adopt the combination of approximated line candidates created by the candidate creation unit 321b. The selection unit 323b selects a combination of approximated lines to be used for calibrating the position and orientation parameters from the combination of approximated line candidates adopted by the determination unit 322b. The detailed functions of the straight line approximation unit 32b will be described later.

[0075] The parameter calibration unit 32c has the same function as the parameter calibration unit 22c in the first embodiment. The parameter calibration unit 32e calibrates the position and orientation parameters using the approximate line selected by the selection unit 32d, the reference line, and the provisional position and orientation parameters.

[0076] Next, the sensor calibration method of this embodiment will be described. In the sensor calibration method of this embodiment, first, the reference setting units 12a and 12b set, for each of the two monocular cameras 5a and 5b, reference lines 7a and 7b, a location 81a on the reference line 7a where a marker 8a is fixed as a reference point 81a, and a location 81b on the reference line 7b where a marker 8b is fixed as a reference point 81b. In this embodiment, the reference lines 7a and 7b are set so as to intersect, as shown in FIG. 15. The provisional position and orientation parameters of the monocular camera 5a are set in a three-dimensional space with the reference line 7a as the x-axis and the reference point 81a as the origin. The provisional position and orientation parameters of the monocular camera 5b are set in a three-dimensional space with the reference line 7b as the x-axis and the reference point 81b as the origin. In other words, the reference lines 7a and 7b and the reference points 81a and 81b are set in different coordinate systems.

[0077] Next, the position measurement units 11a and 11b measure the positions of the pedestrians 6a and 6b moving on the reference lines 7a and 7b, respectively, and the memory unit 31 stores the position measurement results of the pedestrians 6a and 6b at each time as a set of measurement points. Next, the stay point calculation unit 32a calculates the estimated stay points of the pedestrians 6a and 6b from the position measurement results stored in the memory unit 31 using the same method as in the first embodiment.

[0078] Next, the candidate creating unit 321b creates approximate line candidates for the sets of measurement points for each of the sets of measurement points for the two monocular cameras 5a and 5b using a method that combines principal component analysis and the RANSAC algorithm for the coordinate values ​​of the reference points, as in the first embodiment. In this embodiment, the candidate creating unit 321b calculates approximate line candidates until the number of times that they have not been rejected reaches a preset number, as in the line approximation process of the first embodiment. The candidate creating unit 321b creates a combination of approximate line candidates by combining the approximate line candidates created for each of the two monocular cameras 5a and 5b.

[0079] Next, the determination unit 322b first converts the reference lines 7a, 7b and the reference points 81a, 81b into a common coordinate system, for example, the coordinate system of a Geospatial Information Authority of Japan map, and sets a reference broken line 9c formed by the two reference lines 7a, 7b (see FIG. 15). When setting the reference broken line 9c, the determination unit 322b acquires information on the distances Dma, Dmb (hereinafter referred to as "measured distances") from each of the two reference points 81a, 81b on the reference broken line 9c to the intersection 7c of the reference lines 7a, 7b (see FIG. 15). The measured distances Dma, Dmb may be distances actually measured using, for example, a roller distance meter, or may be distances calculated from a map.

[0080] Next, the determination unit 322b converts the approximate line candidates for each of the two monocular cameras 5a and 5b created by the candidate creation unit 321b into a coordinate system common to the reference broken line 9c, and calculates the distance from the converted estimated stay point to the intersection of the two approximate line candidates (hereinafter referred to as the "estimated distance"). The determination unit 322b compares the absolute value of the difference between the measured distance and the estimated distance with an error (hereinafter referred to as the "intersection rejection threshold") that takes into account the procedures for distance measurement and setting the reference broken line. If the error is equal to or less than the intersection rejection threshold, the determination unit 322b accepts the combination of the two approximate line candidates, and if the error is greater than the intersection rejection threshold, the determination unit 322b rejects the combination of the two approximate line candidates.

[0081] For example, suppose the candidate creation unit 321b creates a combination of an approximate line candidate Lpa1 (estimated stay point Pea1) for the monocular camera 5a and an approximate line candidate Lpb1 (estimated stay point Peb1) for the monocular camera 5b. For this combination of approximate line candidates (Lpa1, Lpb1), the distance from the intersection of the approximate line candidate Lpa1 and the approximate line candidate Lpb1 to the estimated stay point Pea1 is defined as an estimated distance Dea, and the estimated distance from the intersection of the approximate line candidate Lpa1 and the approximate line candidate Lpb1 to the estimated stay point Peb1 is defined as an estimated distance Deb. In this case, if both the difference between the estimated distance Dea and the measured distance Dma and the difference between the estimated distance Deb and the measured distance Dmb are equal to or less than the intersection rejection threshold, the combination of approximate line candidates (Lpa1, Lpb1) is adopted. On the other hand, even if the estimated distance Dea is approximately equal to the measured distance Dma, if the difference between the estimated distance Deb and the measured distance Dmb is greater than the intersection rejection threshold, the combination of approximate line candidates (Lap1, Lpb1) is rejected. Note that the intersection rejection threshold may be set to a uniform value if the two measured distances are approximately the same. The intersection rejection threshold may also be set as a ratio to the measured distance. In this way, the determination unit 322b determines whether to accept or reject the combination of multiple approximate line candidates created by the candidate creation unit 321b.

[0082] Next, the selection unit 323b selects one combination from the combinations of approximate line candidates adopted by the determination unit 322b. In this embodiment, the selection unit 323b selects, from the adopted combinations of approximate line candidates, the combination that minimizes the aggregated value of the approximation errors of the two approximate line candidates. The aggregated value of the errors of the approximate line candidates may be a simple average or a weighted average based on the number of measurement points.

[0083] According to the sensor calibration device 3 of the present embodiment described above, the line approximation unit 32b sets a reference broken line 9c formed from two reference lines 7a, 7b set for each of the two monocular cameras 5a, 5b. For each of the two reference lines 7a, 7b, the line approximation unit 32b acquires a measured distance from the intersection 7c of the two reference lines 7a, 7b to the reference points 81a, 81b on the two reference lines 7a, 7b. The line approximation unit 32b also acquires an estimated distance from the estimated stay point to the intersection of the two approximate line candidates, and calibrates the position and orientation parameters of the monocular cameras 5a, 5b using the positions of the reference points 81a, 81b, the measured distance, and the estimated distance. This allows the reference lines 7a, 7b to be determined by two points: the reference point and the intersection, thereby improving the accuracy of the approximate line. This improves the calibration accuracy of the position and orientation parameters of the sensor.

[0084] <Modification of this embodiment> The present invention is not limited to the above-described embodiment, and can be embodied in various forms without departing from the spirit of the invention. For example, the following modifications are also possible.

[0085] [Variation 1] In the above-described embodiment, the approximate line candidate is generated using a method that combines the principal component analysis method and the RANSAC method in the line approximation process. However, the method for generating the approximate line candidate is not limited to this. The approximate line candidate may be generated using only the principal component analysis method.

[0086] [Variation 2] In the first embodiment, the parameter calibration unit 22c calibrates the orientation parameters a and e, and then calibrates the position parameter s. However, the order in which the position and orientation parameters of the monocular camera 5 are calibrated is not limited to this. The orientation parameter may be calibrated by translating the approximate line so that the estimated longest stay point LPe overlaps with the reference point Ps, and then by rotating the approximate line so that the approximate line La overlaps with the reference line Ls.

[0087] [Variation 3] In the first and second embodiments, the sensor calibration device 1 calibrates one monocular camera 5. However, the "sensor" does not have to be a monocular camera 5, and as described above, may be a stereo camera, radar, LiDAR (laser radar), sonar, etc. The same applies to the third embodiment in which two monocular cameras 5 are configured.

[0088] [Variation 4] In the second embodiment, the parameter calibration unit calibrates the location parameters by moving a plurality of estimated stay points closer to a plurality of reference points using the least absolute value method, which is less susceptible to the influence of outliers. The method of calibrating the location parameters by the parameter calibration unit is not limited to this. For example, the least squares method may be used.

[0089] [Variation 5] In the third embodiment, the selection unit 323b selects, from among the combinations of adopted approximate line candidates, a combination in which the aggregated value of the approximation errors of the two approximate line candidates is the smallest. However, the method of selecting a combination by the selection unit 323b is not limited to this. For example, one approximate line candidate is calculated for each set of measurement points, and if no combination is rejected, the combination is retained. When the number of retained combinations reaches a predetermined number, the combination in which the aggregated value of the approximation errors is the smallest may be selected. This method can improve the accuracy of the approximate line depending on the rejection conditions, thereby reducing the effort required to calculate approximate line candidates.

[0090] [Variation 6] In the first and second embodiments, the sensor calibration device 1 is configured from two devices: a measuring device 10 and a calibration processor 20. However, the sensor calibration device may be configured such that the measuring device and the calibration processor are integrated into one device, or may be installed in different locations as in the third embodiment.

[0091] This aspect has been described above based on embodiments and modifications. However, the above-described embodiments are intended to facilitate understanding of this aspect and are not intended to limit this aspect. This aspect may be modified or improved without departing from the spirit and scope of the claims, and equivalents thereof are included in this aspect. Furthermore, if a technical feature is not described as essential in this specification, it may be deleted as appropriate. [Explanation of symbols]

[0092] 1,3...Sensor calibration device 5, 5a, 5b...Monocular camera 6, 6a, 6b...Pedestrians 10, 10a, 10b...Measuring instruments 11, 11a, 11b... Position measurement unit 12, 12a, 12b...Standard setting section 20,30...Calibration processor 21,31...Storage section 22a, 32a... Stay point calculation unit 22b,32b…Line approximation part 22c, 32c...Parameter calibration section La…Approximate straight line Ls…Reference straight line LPe: Estimated longest stay point Ps0…Representative reference point Ps1,Ps2,Ps3,Ps4…Reference point Pe, Pe1, Pe2, Pe3, Pe4, Pea1, Peb1...Estimated stay point a,a c , a t ...Azimuth angle (attitude parameter) e,e c , e t ...Elevation angle (attitude parameter) s,s c , s t ...coordinate values ​​(position parameters)

Claims

1. A sensor calibration device used to calibrate the position and orientation of a sensor, comprising: a reference line setting unit that sets a reference line; a reference point setting unit that sets a reference point on the reference line that will be the center of a position of an object moving in a three-dimensional space; a position measurement unit that measures the position of the object using position and orientation parameters that are set in advance as parameters that represent the temporary position and orientation of the sensor and an output from the sensor that detects the object in three-dimensional space; a storage unit that stores, as a set of measurement points, position measurement results at each time of the object moving on the reference straight line obtained by the position measurement unit; a stay point calculation unit that calculates an estimated stay point of the object based on a density distribution of the measurement points in the measurement point set; a straight line approximation unit that creates an approximation straight line of the set of measurement points using the set of measurement points stored in the storage unit; a parameter calibration unit that calibrates the position and orientation parameters using the reference line, the reference point, the approximate line, and the estimated stay point; The parameter calibration unit calibrating position parameters by translating the approximation line so that the estimated stay point approaches the reference point; calibrating the attitude parameters by rotating the approximated line so that the approximated line approaches the reference line; Sensor calibration equipment.

2. 2. The sensor calibration device of claim 1, The reference point setting unit A plurality of the reference points are set on the reference line; Among the plurality of set reference points, the reference point where the object stays most often is set as a representative reference point; The staying point calculation unit calculating the plurality of estimated stay points based on a density distribution of the measurement points in the set of measurement points; Among the plurality of estimated stay points, an estimated stay point having the highest density distribution of measurement points in the set of measurement points is set as an estimated longest stay point; the parameter calibration unit calibrates the position parameters by translating the approximation line so that the estimated longest stay point approaches the representative reference point; Sensor calibration equipment.

3. 3. The sensor calibration device of claim 2, the straight line approximation unit performs principal component analysis on the estimated longest stay point to create an approximate straight line of the set of measurement points. Sensor calibration equipment.

4. 2. The sensor calibration device of claim 1, the reference point setting unit sets a plurality of the reference points on the reference straight line; the stay point calculation unit calculates the same number of estimated stay points as the number of reference points set by the reference point setting unit, the parameter calibration unit calibrates the position parameters by translating the approximation line so that each of the plurality of estimated stay points approaches each of the corresponding plurality of reference points; Sensor calibration equipment.

5. 5. The sensor calibration device of claim 4, the parameter calibration unit calibrates the position parameters by translating the approximation line so as to minimize a sum of absolute values ​​of distances between a plurality of the reference points and a plurality of the estimated stay points corresponding to the plurality of the reference points, respectively. Sensor calibration equipment.

6. 6. A sensor calibration device according to claim 1, comprising: the sensor calibration device is connected to each of two sensors installed at different locations; the reference line setting unit sets reference lines that intersect with each other for each of the two sensors; the reference point setting unit sets a reference point on each of the two reference straight lines; The linear approximation unit acquire information about the distance between the reference point and the intersection point on a reference broken line formed by the two intersecting reference straight lines and the intersection point of the two reference straight lines; creating an approximation line of the set of measurement points using information about the distance between the reference point and the intersection point; Sensor calibration equipment.

7. A sensor calibration method for calibrating the position and orientation of a sensor using a sensor calibration device, comprising: A step of setting a reference line; setting a reference point on the reference line as a center of stay of an object moving in three-dimensional space; measuring the position of the object using position and orientation parameters that are preset as parameters representing a temporary position and orientation of the sensor and an output from the sensor that detects the object in three-dimensional space; a step of storing, as a set of measurement points, position measurement results at each time of the object moving on the reference straight line measured in the step of measuring the position of the object; calculating an estimated stay point of the object based on a density distribution of the measurement points in the set of measurement points; creating an approximation line of the set of measurement points using the set of measurement points stored; a calibration step of calibrating the position and orientation parameters using the reference line, the reference point, the approximate line, and the estimated stay point, In the calibration step, calibrating position parameters by translating the approximation line so that the estimated stay point approaches the reference point; calibrating the attitude parameters by rotating the approximated line so that the approximated line approaches the reference line; Sensor calibration methods.

Images