Mobile robot, movement amount estimation method, and program

Abstract

To provide a mobile robot capable of estimating the position of the mobile robot with high accuracy even in the case that a camera for imaging a road surface does not face the road surface, a movement amount estimation method, and a program.SOLUTION: A mobile robot 100 traveling on a road surface includes a three-dimensional camera 2 for capturing an image of a road surface 5 to measure a distance to the road surface 5, an extraction unit 11 for extracting a feature point included in the image picked up by the three-dimensional camera 2, a correction unit 12 for correcting the coordinates of the feature point of two frame images included in the image on the basis of a distance (depth d) from the three-dimensional camera 2 to the road surface 5, and a calculation unit 13 for calculating a movement amount of the mobile robot 100 on the basis of a movement vector showing a movement amount of the corrected feature point.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to a mobile robot, a movement amount estimation method, and a program. [Background technology]

[0002] Mobile robots equipped with sensors and capable of moving by remote control are used for tasks such as inspecting infrastructure facilities. Non-Patent Document 1 discloses a robot equipped with a holonomic omnidirectional mobile mechanism that can move in any direction. Such mobile robots are required to measure their position on a two-dimensional plane with high accuracy.

[0003] Non-Patent Document 2 discloses that a mobile robot moving indoors is imaged from above using a camera installed on the ceiling, and the position of the mobile robot is estimated based on the image.

[0004] Non-Patent Document 3 discloses that the position of a mobile robot is estimated by calculating the distance from a transmitter to the mobile robot using Wi-Fi radio waves used for indoor communication.

[0005] Non-Patent Document 4 discloses the use of visual odometry, which captures images of the ground on which a mobile robot travels and estimates the mobile robot's own position based on the captured images and the detection values of axle encoders.

[0006] Non-Patent Document 5 discloses that in a visual odometry algorithm, when a pair of feature vectors present in two frame images that are close to each other are matched, the accuracy of position estimation is improved by using the MSAC algorithm to remove outliers from the matching. [Prior art documents] [Non-patent literature]

[0007] [Non-Patent Document 1] K. Tadakuma,R. Tadakuma and J. Berengeres,”Development of Holonomic Omnidirectional Vehicle with“Omini-Ball”:Spherical Wheels,Proceedings of the 2007 IEEE / RSJInternational Conference on Intelligent Robotsand Systems, San Diego, CA, USA 2007 ).

Outdoor Tool2

Outdoor Tools3

Outdoor Tools 4

Direct Environment 5

[0008] However, the method disclosed in the above-mentioned Non-Patent Document 2 is intended to estimate the position of a mobile robot traveling indoors. When this method is applied to a mobile robot traveling outdoors, a large-scale device such as a crane is required to capture images of the entire field in which the mobile robot travels from above.

[0009] In the method disclosed in Non-Patent Document 3, if there is an electromagnetic obstruction outdoors, Wi-Fi radio waves may not reach the area, and the movement amount of the mobile robot cannot be estimated with high accuracy.

[0010] The method disclosed in Non-Patent Document 4 requires that ground image information be acquired in advance. Also, in larger-scale systems, the map database capacity increases, which causes the problem of longer search times.

[0011] The method disclosed in Non-Patent Document 5 can improve the accuracy of position estimation using visual odometry, but has the problem that highly accurate position estimation cannot be performed when the imaging surface of the camera that captures the road surface is not positioned parallel to the road surface, i.e., when the camera is not directly facing the road surface.

[0012] The present disclosure has been made in consideration of the above circumstances, and its purpose is to provide a mobile robot, a movement amount estimation method, and a program that can estimate the position of the mobile robot with high accuracy even when the camera capturing the road surface is not directly facing the road surface. [Means for solving the problem]

[0013] A mobile robot according to one embodiment of the present invention is a mobile robot that travels on a road surface, and is equipped with a three-dimensional camera that captures an image of the road surface and measures the distance to the road surface, an extraction unit that extracts feature points contained in the image captured by the three-dimensional camera, a correction unit that corrects the coordinates of the feature points of two frame images contained in the image based on the distance from the three-dimensional camera to the road surface, and a calculation unit that calculates the amount of movement of the mobile robot based on a movement vector that indicates the amount of movement of the feature points after the correction.

[0014] A movement amount estimation method according to one embodiment of the present invention is a movement amount estimation method for estimating the movement amount of a mobile robot traveling on a road surface, which involves capturing an image of the road surface using a 3D camera, measuring the distance to the road surface, extracting feature points contained in the image, correcting the coordinates of the feature points of two frame images contained in the image based on the distance from the 3D camera to the road surface, and calculating the movement amount of the mobile robot based on a movement vector indicating the movement amount of the feature points after the correction.

[0015] One aspect of the present invention is a program for causing a computer to execute control of the mobile robot. [Effects of the Invention]

[0016] According to the present invention, it is possible to estimate the position of a mobile robot with high accuracy even when the camera is not directly facing the road surface. [Brief explanation of the drawings]

[0017] [Figure 1] FIG. 1 is a block diagram showing the configuration of a mobile robot according to an embodiment. [Figure 2] FIG. 2 is an explanatory diagram showing the relationship between the camera coordinate system set in the three-dimensional camera, the image coordinate system, and the image capture area of the road surface. [Figure 3] FIG. 3 is an explanatory diagram showing a camera coordinate system, an image coordinate system, and an imaging area of a road surface, and shows a case where the imaging surface of the camera is inclined with respect to the road surface. [Figure 4]FIG. 4 is an explanatory diagram showing an image capture area on a road surface captured by a three-dimensional camera when the image capture surface of the three-dimensional camera is inclined at an angle α with respect to the road surface. [Figure 5] FIG. 5 is an explanatory diagram that schematically shows an image in the image coordinate system when the imaging surface of the three-dimensional camera is installed at an angle α with respect to the road surface. [Figure 6A] FIG. 6A is an explanatory diagram showing a mobile robot moving linearly in the lateral direction. [Figure 6B] FIG. 6B is a graph showing the path of the mobile robot when it is moving in a straight line. [Figure 7A] FIG. 7A is a flowchart showing the procedure of the position estimation process performed by the mobile robot. [Figure 7B] FIG. 7B is a flowchart showing the detailed processing procedure of step S3 shown in FIG. 7A. [Figure 8A] FIG. 8A is an explanatory diagram showing a first frame image and a second frame image included in an image captured by a three-dimensional camera. [Figure 8B] FIG. 8B is an explanatory diagram showing feature points extracted from the first frame image and the second frame image. [Figure 8C] FIG. 8C is an explanatory diagram showing feature vectors of feature points included in the first frame image and the second frame image. [Figure 8D] FIG. 8D is an explanatory diagram showing matching of feature points included in the first frame image and the second frame image. [Figure 8E] FIG. 8E is an explanatory diagram showing a movement vector calculated based on the pixel coordinates of each matched feature point. [Figure 9A] FIG. 9A is a graph showing a travel trajectory of a mobile robot estimated by visual odometry when errors due to tilt of a three-dimensional camera are not corrected. [Figure 9B] FIG. 9B is a graph showing the relationship between the number of frames and the angle when errors due to the tilt of the three-dimensional camera are not corrected. [Figure 10A]FIG. 10A is a graph showing the estimated travel path of a mobile robot using visual odometry when errors due to tilt of a three-dimensional camera are not corrected and when the errors are corrected. [Figure 10B] FIG. 10B is a graph showing the relationship between the number of frames and the angle when the error due to the tilt of the three-dimensional camera is not corrected and when the error is corrected. [Figure 11] FIG. 11 is a block diagram showing the hardware configuration of this embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0018] Hereinafter, an embodiment will be described with reference to the drawings. Fig. 1 is a block diagram showing the configuration of a mobile robot 100 according to an embodiment. As shown in Fig. 1, the mobile robot 100 includes a measuring device 1, a three-dimensional camera 2 (hereinafter abbreviated as "camera 2"), and a cart 3.

[0019] In this embodiment, the actual dimensions (hereinafter abbreviated as "actual dimensions") of each pixel included in the image captured by camera 2 are calculated, the number of pixels by which the feature point moves between each frame image is calculated, and the actual dimensions of the movement amount of the feature point are calculated based on the actual dimensions of each pixel, thereby improving the accuracy of estimating the movement amount of mobile robot 100.

[0020] When the camera 2 and the road surface 5 (see Figure 2 described later) are directly facing each other, the actual size of the pixels included in the image captured by the camera 2 can be calculated by dividing the size of the area captured by the camera 2 by the number of pixels. For example, in one dimension, if the distance of the entire image capture range is "L" and the number of pixels is "n", then one pixel corresponds to the distance "L / n" in real space.

[0021] The actual size per pixel can be calculated based on the focal length, imaging distance (distance from camera 2 to road surface 5, depth d described later), and sensor size, which are parameters of camera 2. Even if the imaging surface of camera 2 is tilted with respect to road surface 5, the actual size of each pixel can be calculated if the imaging distance (depth d) for each pixel is known.

[0022] To determine the actual size of each pixel, it is necessary to measure the imaging distance of each pixel. In this embodiment, the depth measured by the 3D camera 2 is used as the imaging distance. Note that instead of a 3D camera, a 2D camera and a distance sensor may be used in combination.

[0023] The cart 3 shown in FIG. 1 is equipped with a plurality of wheels 4 (for example, four wheels) and travels on a road surface 5. The road surface 5 is, for example, an outdoor asphalt road. The wheels 4 are, for example, omni-wheels. The cart 3 can move in any direction on the two-dimensional plane of the road surface 5 by operating the omni-wheels.

[0024] The camera 2 is installed on the underside of the dolly 3 with its imaging surface facing (downward) towards the road surface 5. The camera 2 is, for example, an RGB-d camera, which captures a color image of the road surface and measures the imaging distance (hereinafter referred to as "depth") from the camera 2 to the road surface 5.

[0025] FIG. 2 is an explanatory diagram showing the image coordinate system of the image captured by camera 2, the camera coordinate system set in camera 2, and its positional relationship with the road surface 5. As shown in FIG. 2, camera 2 is set with a camera coordinate system P1 that indicates the actual size of the object being imaged, such as road surface 5, and an image coordinate system P2 that indicates the image of a predetermined area of road surface 5 in pixel units. Camera 2 measures the depth d from a reference point p0 in camera coordinate system P1 to an arbitrary point on road surface 5. The camera coordinate system P1 is represented by three-dimensional coordinates X, Y, and Z. Specifically, the imaging surface of camera 2 is the XY plane, and the direction perpendicular to this XY plane is the Z direction. The image coordinate system P2 is represented by three-dimensional coordinates U, V, and D. Specifically, the imaging surface of camera 2 is the UV plane, and the direction perpendicular to this UV plane is the D direction. The image data and depth data captured by camera 2 are output to measurement device 1. Note that instead of the three-dimensional camera 2, a two-dimensional camera and a distance sensor may be used in combination.

[0026] As shown in Fig. 1, the measuring device 1 is mounted on a cart 3. The measuring device 1 includes an extraction unit 11, a correction unit 12, a calculation unit 13, an estimation unit 14, a map generation unit 15, and a recording unit 16.

[0027] The extraction unit 11 acquires an image of the road surface 5 (e.g., asphalt) captured by the camera 2, i.e., an image in pixel units shown in the image coordinate system P2 of FIG. 2. The extraction unit 11 extracts feature points from the image of the road surface 5. The feature points are, for example, SURF feature points. The extraction unit 11 acquires the feature points included in each frame image captured by the camera 2 while the cart 3 is moving on the road surface 5.

[0028] The correction unit 12 corrects the feature points in each frame image extracted by the extraction unit 11. When the camera 2 shown in FIG. 1 does not face the road surface 5, that is, when the imaging surface of the camera 2 is not parallel to the road surface 5, a tilt occurs with respect to the road surface 5 centered on at least one of the X direction and Y direction shown in FIG. 2. The correction unit 12 corrects errors in the feature point positions that occur due to this tilt. The detailed processing method by the correction unit 12 will be described later.

[0029] The calculation unit 13 calculates a movement vector of the feature point based on the feature point included in the two frame images (the feature point corrected by the correction unit 12). For example, if point a1, which is a candidate feature point, exists on the road surface 5 shown in FIG. 2, point b1 corresponding to point a1 is included in the first frame image (image in the image coordinate system P2) captured by the camera 2. This point b1 is set as the feature point b1 by the extraction unit 11. When the mobile robot 100 is moving, the feature point b1 moves in the second frame image relative to its position in the first frame image. The calculation unit 13 calculates the movement amount of the mobile robot 100 based on the movement vector of feature point b1 included in the first and second frame images.

[0030] The estimation unit 14 estimates the current position of the mobile robot 100 based on the amount of movement of the mobile robot 100 calculated by the calculation unit 13. That is, by repeatedly calculating the movement vector between two frame images, the amount of movement of the mobile robot 100 for each frame image is calculated, and by accumulating the amount of movement, the position of the mobile robot 100 on a two-dimensional plane is estimated.

[0031] The map generating unit 15 generates a map showing the movement route of the mobile robot 100 based on the position information of the mobile robot 100 estimated by the estimating unit 14. The generated map is displayed on, for example, a display (not shown).

[0032] The recording unit 16 records the map data generated by the map generating unit 15.

[0033] Next, detailed processing by the correction unit 12 will be described. Fig. 3 is an explanatory diagram showing a camera coordinate system P1, an image coordinate system P2, and an imaging area of a road surface 5. Fig. 3 shows a case where the imaging surface of the camera 2 does not face the road surface 5 directly, but is tilted at a predetermined angle with respect to the road surface 5 around the X axis. That is, in the imaging area of the road surface 5 shown in Fig. 3, the N2 side (lower side in the drawing) of the arrow is closer to the camera 2 than the N1 side (upper side in the drawing).

[0034] Therefore, the depth d11 from the reference point p0 of the camera coordinate system P1 shown in FIG. 3 to the road surface 5 on the N1 side and the depth d12 to the road surface 5 on the N2 side do not match. That is, d11 > d12, and the N1 side is farther from the reference point p0 than the N2 side.

[0035] When the imaging of a predetermined area on the road surface 5 by the camera 2 is started and the mobile robot 100 moves in the X-axis direction, the first frame image of the image captured by the camera 2 includes feature points p11 and q11 corresponding to points p1 and q1 on the road surface 5. The second frame image includes feature points p12 and q12 corresponding to points p2 and q2 on the road surface 5.

[0036] Since the mobile robot 100 is moving in the X-axis direction, the distance L1 from point p1 to p2 on the road surface 5 and the distance L2 from point q1 to q2 are the same distance, that is, L1 = L2. However, due to the inclination of the imaging surface of the camera 2 with respect to the road surface 5, in the image coordinate system P2 which is the image captured by the camera 2, the distance L11 from the feature point p11 to p12 is shorter than the distance L12 from the feature point q11 to q12. That is, L11 < L12. This is due to the fact that the depth d11 is larger than the depth d12. Therefore, when the movement vector (the movement distance and direction of the feature point for each frame image) is extracted based on the image captured by the camera 2 and the position of the mobile robot 100 is estimated by visual odometry, a large error will occur in the position estimation.

[0037] The correction unit 12 reduces the error in the position estimation by visual odometry by correcting the error of the movement vector caused by the inclination of the imaging surface of the camera 2 with respect to the road surface 5.

[0038] In three-dimensional space, the mounting angle error of the camera 2 exists in each of the three axes, X, Y, and Z. Of these, rotation around the Z axis only results in a deviation on the XY plane and does not cause a measurement error due to the inclination of the imaging surface of the camera 2 and the road surface 5. Therefore, the angle errors around the X and Y axes affect the error in estimating the movement amount of the cart 3. Here, we will explain the case where the cart is inclined by α degrees around the X axis as an example.

[0039] 4 is an explanatory diagram showing the image capture area on the road surface 5 captured by the camera 2 when the imaging surface of the camera 2 is inclined at an angle α with respect to the road surface 5. When the imaging surface of the camera 2 is installed parallel to the road surface 5, the distance between the camera 2 and the road surface 5 is defined as depth d0, and the angle of view of the camera 2 is defined as θ. The depths d1 and d2, which are the distances from the imaging surface of the camera 2 to the image edge points, can be expressed by the following equations (1) and (2). d1=d0sin(π-θ) / 2) / cos(θ / 2-α) …(1) d2=d0sin(π-θ) / 2) / cos(θ / 2+α) …(2)

[0040] Fig. 5 is an explanatory diagram that schematically shows an image in image coordinate system P2 when the imaging surface of camera 2 is installed at an angle α with respect to road surface 5, as shown in Fig. 4. The N3 side (upper side in the figure) of the arrow shown in Fig. 5 indicates a position close to road surface 5, and the N4 side (lower side in the figure) indicates a position far from road surface 5. In other words, the N3 side is at a depth d1 from the imaging surface of camera 2, and the N4 side is at a depth d2 (d2>d1) from the imaging surface of camera 2.

[0041] As explained above with reference to Fig. 3, the movement amount of the feature point is measured as a larger value as the depth d decreases. Therefore, even if the mobile robot 100 moves linearly, the movement vector of the feature point on the N3 side is relatively larger than that on the N4 side, as shown in Fig. 5, due to the inclination of the imaging surface of the camera 2. Therefore, even though the mobile robot 100 is moving linearly, it is measured as if it is rotating in the direction of arrow Y1 shown in Fig. 5.

[0042] For example, when calculating an angle from a movement vector, under the conditions of distance d0 from the imaging surface of camera 2 to road surface 5 = 332 mm, number of feature points: 150, image range: 160 pixel x 160 pixel, 123 mm x 123 mm, movement speed: 250 mm / sec, and total movement distance: 1000 mm, as the tilt of camera 2 increases, the rotation angle of mobile robot 100 due to error increases linearly, and when the tilt of camera 2 is 1 degree, the rotation angle when moving 1000 mm is 0.002 [rad].

[0043] FIG. 6A is an explanatory diagram showing the mobile robot 100 moving linearly laterally. FIG. 6B is a graph showing the travel trajectory of the mobile robot 100 moving linearly in the linear direction. As shown in FIG. 6A, when the mobile robot 100 moves back and forth linearly along the arrow Y2, if the camera 2 is not tilted, i.e., if the camera 2 is directly facing the road surface 5, a trajectory of a linear return to point r1 is measured, as indicated by reference symbol h1 in FIG. 6B. However, because the camera 2 is installed at an angle, the amount of movement in the rotational direction is detected as indicated by the arrow Y1 in FIG. 5, and the trajectory of the mobile robot 100 reaching point r2 by drawing an arc is measured, as indicated by reference symbol h2 in FIG. 6B.

[0044] The correction unit 12 corrects errors caused by the camera 2 being installed at an angle. As shown in Fig. 3, when the imaging surface of the camera 2 and the road surface 5 are not parallel, the relationship between the distance L11 (movement vector) and the distance L12 (movement vector) measured in the image coordinate system P2 is L12 > L11, as described above. The correction unit 12 corrects errors caused by the tilt of the camera 2 by converting the coordinates of the feature points in the image coordinate system P2 in pixel units into the camera coordinate system P1 in actual size units according to the ratio between the focal length f and the depth d of the camera 2.

[0045] Specifically, the X coordinate, Y coordinate, and Z coordinate of the image captured by the camera 2 are corrected using the following equations (3) to (5). x={(u-ppx) / fx}*d …(3) y={(v-ppy) / fy}*d …(4) z=d …(5)

[0046] In equations (3) to (5), "ppx" is the offset value (in pixels) between the camera coordinate system P1 and the image coordinate system P2 in the X direction, and "ppy" is the offset value (in pixels) between the camera coordinate system P1 and the image coordinate system P2 in the Y direction. "u" and "v" are pixel positions in the image coordinate system P2. "d" is the actual depth in the D direction, which can be obtained from the depth information of the 3D camera 2. "fx" is the focal length in the horizontal direction (X-axis direction; first direction), and "fy" is the focal length in the vertical direction (Y-axis direction; second direction), which are respectively expressed by equations (6) and (7) below. fx = {(number of horizontal light receiving elements) / (horizontal length of camera 2)}*f ... (6) fy = {(number of vertical light receiving elements) / (vertical length of camera 2)}*f ... (7)

[0047] "f" in equations (6) and (7) is the focal length of the 3D camera 2 in real size. The pixel position of the feature point in the (u, v) image coordinate system is calculated from the frame image acquired by the camera. The conversion to real size units using equations (3) to (5) above can be considered as a projection conversion onto the road surface 5.

[0048] Next, the position estimation process of the mobile robot 100 according to this embodiment configured as described above will be described. Fig. 7A is a flowchart showing the processing procedure of the position estimation process by the mobile robot 100, and Fig. 7B is a flowchart showing the detailed processing procedure of step S3 shown in Fig. 7A. In this embodiment, an example will be described in which "SURF features" are used as feature points.

[0049] When the mobile robot 100 starts traveling on the road surface 5 and the camera 2 starts capturing an image of the road surface 5, in step S1 of Fig. 7A, the extraction unit 11 sets two frame images in the 3D image captured by the camera 2. For example, a first frame image D1 shown in Fig. 8A(a) and a second frame image D2 shown in Fig. 8A(b) are set. Each of the frame images D1 and D2 is an image on the image coordinate system P2 and is expressed in pixel units.

[0050] In step S2, the extraction unit 11 acquires the depth d from the imaging surface of the camera 2 to the road surface 5 based on the three-dimensional image captured by the camera 2.

[0051] In step S3, the extraction unit 11, the correction unit 12, and the calculation unit 13 extract feature points contained in the two frame images D1 and D2, and further calculate the movement amount of the mobile robot 100 based on these feature points. The detailed processing procedure of step S3 will be described below with reference to Fig. 7B.

[0052] In step S31 of FIG. 7B, the extraction unit 11 extracts feature points contained in two frame images D1 and D2. For example, as shown in FIGS. 8B(a) and (b), feature points p21 and q21 are extracted from the first frame image D1 and the second frame image D2. A feature point is a point specified in pixel units in an area on an image that has a distinctive shading pattern. It is recommended to set several tens to several hundreds of feature points. Note that, to avoid complicating the explanation, the number of feature points is set to two.

[0053] In step S32, the extraction unit 11 calculates the coordinates (UV coordinates) of the feature points p21 and q21 included in the first frame image D1 and the feature points p22 and q22 included in the second frame image D2 in pixel units.

[0054] In step S33, the extraction unit 11 extracts feature vectors of each of the feature points p21, q21, p22, and q22. For example, as shown in FIGS. 8C(a) and 8C(b), the extraction unit 11 extracts feature vectors Fi = [Fi1, Fi2, Fi64] and F'j = [F'j1, F'j2, F'j64] of the feature points p21 and p22.

[0055] In step S34, the correction unit 12 converts the feature vectors extracted in step S33 into feature vectors in the camera coordinate system P1 expressed in actual size units based on the depth d of the two frame images D1 and D2 expressed in pixel units. Specifically, the conversion is performed using the above-mentioned equations (3) to (5).

[0056] In step S35, the calculation unit 13 matches pairs of feature vectors that are close in distance between the two frame images D1' and D2' converted into the camera coordinate system P1 (actual size units). Specifically, as shown in Figures 8D(a) and 8D(b), in the first frame image D1' and second frame image D2' converted into the camera coordinate system P1, feature points p21 and p22 are matched, and feature points q21 and q22 are matched.

[0057] In step S36, the calculation unit 13 excludes, from among the feature points included in each of the frame images D1' and D2', the feature points that were not matched in the process of step S35.

[0058] In step S37, the calculation unit 13 calculates the displacement of the mobile robot 100 based on the pixel coordinates of each matched feature point. Specifically, as shown in Fig. 8E, the calculation unit 13 calculates the displacement of the mobile robot 100 based on the movement vector vi of the feature point. Thereafter, the process proceeds to step S4 in Fig. 7A.

[0059] In step S4, the estimation unit 14 accumulates the amount of displacement relative to the start position where the mobile robot 100 started moving, and estimates the self-position of the mobile robot 100. Thereafter, the map generation unit 15 creates a map of the movement trajectory of the mobile robot 100, and records the created map in the recording unit 16.

[0060] Next, the results of visual odometry measurements when the mobile robot 100 according to this embodiment moves back and forth in a straight line and when a conventional mobile robot moves back and forth in a straight line will be compared and explained.

[0061] FIG. 9A is a graph showing the estimated travel trajectory of the mobile robot 100 by visual odometry when the error due to the tilt of the camera 2 is not corrected, and FIG. 9B is a graph showing the relationship between the number of frames and the angle.

[0062] 9A, when the imaging surface of camera 2 is not parallel to road surface 5, the estimated travel trajectory obtained when mobile robot 100 moves back and forth along the X-axis direction is a curved one with lateral displacement on the order of meters, as shown by curve s11. Also, as shown by curve s14 in FIG. 9B, the estimated result indicates that mobile robot 100 is moving at an angle, even though it is actually moving in a straight line.

[0063] FIG. 10A is a graph showing the estimated travel path of a mobile robot using visual odometry when the error due to the tilt of camera 2 is not corrected and when the error due to the tilt of camera 2 is corrected using the method of this embodiment, and FIG. 10B is a graph showing the relationship between the number of frames and the angle.

[0064] Curve s12 in Fig. 10A shows the estimated driving trajectory when error is not corrected, and curve s13 shows the estimated driving trajectory when error is corrected. Curve s14 in Fig. 10B shows the relationship between the number of frames and the angle when error is not corrected, and curve s13 shows the relationship between the number of frames and the angle when error is corrected.

[0065] As shown by the curve s13 in Fig. 10A, by employing the method of this embodiment to correct the error due to the tilt of the camera 2, it is understood that an estimated travel trajectory is obtained almost faithfully when the mobile robot 100 is caused to move back and forth in a straight line. Furthermore, as shown by the curve s15 in Fig. 10B, the result shows that the mobile robot 100 moves in a straight line in the direction at an angle of approximately 0 degrees. In other words, it was confirmed that the accuracy of the position estimation of the mobile robot 100 is improved by performing the correction process according to this embodiment.

[0066] As described above, the mobile robot 100 according to this embodiment is a mobile robot 100 that travels on a road surface, and is equipped with a 3D camera 2 that captures an image of the road surface 5 and measures the distance to the road surface 5, an extraction unit 11 that extracts feature points contained in the image captured by the 3D camera 2, a correction unit 12 that corrects the coordinates of the feature points of two frame images contained in the image based on the distance (depth d) from the 3D camera 2 to the road surface 5, and a calculation unit 13 that calculates the amount of movement of the mobile robot 100 based on a movement vector that indicates the amount of movement of the feature points after correction.

[0067] In this embodiment, even if the imaging surface of the camera 2 mounted on the mobile robot 100 is tilted, it is possible to correct errors caused by this tilt. This reduces the effort required for fine adjustments to make the imaging surface of the camera 2 parallel to the road surface 5. Therefore, there is no need to use a high-precision laser displacement sensor to finely adjust the mounting position of the camera 2. It is also possible to lower the precision required for the mounting position. As a result, it is possible to simplify the configuration of the mobile robot 100 and reduce costs.

[0068] In this embodiment, the depth d from the camera 2 to an arbitrary point on the road surface 5 is measured, and the coordinates of the feature point in the image coordinate system P2 are corrected based on the measured depth d, so that the movement vector of the feature point can be calculated with high accuracy even when the imaging surface of the camera 2 does not directly face the road surface 5. Consequently, it becomes possible to estimate the self-position of the mobile robot 100 with high accuracy using visual odometry.

[0069] In this embodiment, the image captured by the camera 2 in pixel units (image coordinate system P2) is converted into an image in actual size units (camera coordinate system P1), so that the movement amount of the feature point can be measured with high precision.

[0070] In this embodiment, the coordinates of the feature points in the two frame images are converted using a conversion formula set based on the focal length f and depth d of the camera 2, i.e., the above-mentioned formulas (3) to (5), so that the movement amount of the feature points can be measured with high accuracy.

[0071] Furthermore, since the focal length f is the horizontal focal length fx and the vertical focal length fy, it becomes possible to measure the movement amount of the feature point with even higher accuracy.

[0072] By using a two-dimensional camera and a distance sensor instead of the three-dimensional camera 2 described above, it is possible to achieve an inexpensive configuration.

[0073] The measuring device 1 mounted on the mobile robot 100 of the present embodiment described above can be, for example, a general-purpose computer system including a CPU (Central Processing Unit, processor) 901, a memory 902, a storage 903 (HDD: Hard Disk Drive, SSD: Solid State Drive), a communication device 904, an input device 905, and an output device 906, as shown in Fig. 11. The memory 902 and the storage 903 are storage devices. In this computer system, the CPU 901 executes a predetermined program loaded on the memory 902, thereby realizing each function of the measuring device 1.

[0074] The measurement device 1 may be implemented in one computer or in multiple computers, or may be a virtual machine implemented in a computer.

[0075] The program for the measuring device 1 can be stored in a computer-readable recording medium such as a HDD, SSD, USB (Universal Serial Bus) memory, CD (Compact Disc), or DVD (Digital Versatile Disc), or can be distributed via a network. The computer-readable recording medium is, for example, a non-transitory recording medium.

[0076] The present disclosure is not limited to the above-described embodiments, and various modifications are possible within the scope of the present disclosure. [Explanation of symbols]

[0077] 1. Measuring equipment 2. 3D camera (camera) 3 carts 4 wheels 5 Road surface 11 Extraction part 12 Correction unit 13 Calculation section 14 Estimation part 15 Map generation section 16 Recording section 100 Mobile Robots d Depth (distance) f focal length P1 camera coordinate system P2 image coordinate system

Claims

1. A mobile robot that travels on a road surface, a three-dimensional camera that captures an image of the road surface and measures the distance to the road surface; an extraction unit that extracts feature points included in the image captured by the three-dimensional camera; a correction unit that corrects coordinates of feature points of two frame images included in the image based on a distance from the three-dimensional camera to the road surface; a calculation unit that calculates a movement amount of the mobile robot based on a movement vector that indicates the movement amount of the feature point after correction; A mobile robot equipped with

2. The correction unit converts coordinates of feature points of the two frame images using a conversion formula set based on a focal length of the three-dimensional camera and a distance to a road surface measured by the three-dimensional camera. The mobile robot of claim 1 .

3. As the focal length, a focal length in a first direction on the image and a focal length in a second direction intersecting the first direction are used. The mobile robot according to claim 2 .

4. Instead of the three-dimensional camera, a two-dimensional camera for capturing an image of the road surface and a distance sensor for measuring the distance to the road surface are provided. The mobile robot of claim 1 .

5. A movement amount estimation method for estimating a movement amount of a mobile robot traveling on a road surface, comprising: capturing an image of the road surface with a three-dimensional camera and measuring the distance to the road surface; extracting feature points contained in the image; correcting the coordinates of feature points of the two frame images included in the image based on the distance from the three-dimensional camera to the road surface; Calculating the movement amount of the mobile robot based on the movement vector indicating the movement amount of the feature point after correction Movement amount estimation method.

6. A program that causes a computer to function as the mobile robot according to claim 1 or 2.

Images