Mobile robot, method for estimating movement distance, and mobile robot control program

Abstract

To provide a moving robot for which a movement amount can be accurately estimated even if using a visual odometry, a method of estimating a movement amount and a program for controlling a moving robot.SOLUTION: A moving robot comprises: a camera 32 that takes a moving image of a traveling-road surface; an image setting section 11 that sets up a base image; a transition-area setting section 13 that sets up a transition area R1 in the base image; a displacement-amount calculating section 12 that calculates a displacement amount of a feature point between images based on feature points included in the base image and a target image; a determining section 15 that determines whether or not a moving robot 1 is inside the transition area R1 based on the displacement amount; an image update section 16 that, if the moving robot 1 is inside the transition area R1, does not change the base image and, if the moving robot 1 is outside the transition area R1, sets a target image obtained from a precedent frame where the displacement amount is outside the transition area to a new base image; and a movement-amount calculating section 17 that calculates a movement amount based on the feature points of the base image and target image.SELECTED DRAWING: Figure 2

Description

Technical Field

[0001] The present invention relates to a mobile robot, a method for estimating a moving amount, and a mobile robot control program.

Background Art

[0002] When inspecting infrastructure facilities, etc., a mobile robot equipped with sensors and movable by remote control is used. Non-Patent Document 1 discloses a robot equipped with a holonomic omnidirectional moving mechanism that can move in any direction. Such a mobile robot is required to accurately measure its position on a two-dimensional plane.

[0003] Non-Patent Document 2 discloses that a mobile robot moving indoors is imaged from above using a camera installed on the ceiling, and the position of the mobile robot is estimated based on the captured image.

[0004] Non-Patent Document 3 discloses that the position of a mobile robot is estimated by calculating the distance from a transmitter to the mobile robot using Wi-Fi radio waves used for indoor communication.

[0005] Non-Patent Document 4 discloses that an image of the ground on which a mobile robot travels is captured, and the self-position of the mobile robot is estimated based on the captured image and the detection value of an axle encoder.

Prior Art Documents

Non-Patent Documents

[0006]

Non-Patent Document 1

[0007] However, the method disclosed in Non-Patent Document 2 mentioned above is intended for estimating the position of a mobile robot traveling indoors. Applying this to a mobile robot moving outdoors would require large-scale equipment such as a crane to image the entire field in which the mobile robot is moving from above.

[0008] In the method disclosed in Non-Patent Document 3, Wi-Fi signals may not reach the device if there are electromagnetic shielding objects outdoors, making it impossible to estimate the movement of the mobile robot with high accuracy.

[0009] The method disclosed in Non-Patent Document 4 has several drawbacks, including the need to acquire ground image information in advance and the fact that, in larger-scale systems, the map database becomes larger, resulting in longer search times.

[0010] Furthermore, a method is known to estimate the two-dimensional position of a mobile robot by using visual odometry, which involves capturing moving images of the road surface while the mobile robot is moving and measuring the amount of image displacement between frames contained in the captured moving images to calculate the amount of movement the mobile robot has made in which direction and by how much.

[0011] However, methods using visual odometry introduce displacement errors between frames, and these errors accumulate as the mobile robot travels. Consequently, as the distance the mobile robot travels increases, the accumulated error grows, leading to a problem where the mobile robot's position cannot be estimated with high accuracy.

[0012] The present invention has been made in view of the above circumstances, and its object is to provide a mobile robot, a method for estimating the amount of movement, and a mobile robot control program that can estimate the amount of movement with high accuracy even when using visual odometry. [Means for solving the problem]

[0013] A mobile robot according to one aspect of the present invention includes: an imaging unit that captures a moving image of a road surface; an image setting unit that sets an arbitrary frame of the moving image as a base image; a transition region setting unit that sets a transition region based on a reference position in the base image; a displacement amount calculation unit that calculates the amount of displacement of the feature points between images based on the feature points included in the base image and a target image obtained from a frame after the base image; a determination unit that determines whether the reference position in the target image is inside the transition region based on the displacement amount; an image update unit that, if the reference position in the target image is inside the transition region, does not change the base image, and if it is outside, sets a target image obtained from a frame before the reference position in the target image becomes outside the transition region as the new base image; and a movement amount calculation unit that calculates the amount of movement of the mobile robot based on the displacement amounts of the feature points included in the base image and the feature points included in the target image.

[0014] A method for estimating the amount of movement according to one aspect of the present invention comprises the steps of: capturing a moving image of a road surface; setting an arbitrary frame of the moving image as a base image; setting a transition region based on a reference position in the base image; calculating the amount of displacement of the feature points between images based on the feature points included in the base image and a target image obtained from a frame after the base image; determining whether the reference position in the target image is inside the transition region based on the amount of displacement; if the reference position in the target image is inside the transition region, not changing the base image; if it is outside, setting a target image obtained from a frame before the reference position in the target image becomes outside the transition region as a new base image; and calculating the amount of movement of a mobile robot based on the amount of displacement of the feature points included in the base image and the feature points included in the target image.

[0015] One aspect of the present invention is a mobile robot control program for causing a computer to function as the mobile robot described above.

Advantages of the Invention

[0016] According to the present invention, even when visual odometry is used, it becomes possible to accurately estimate the amount of movement.

Brief Description of the Drawings

[0017] [Figure 1] FIG. 1 is a perspective view schematically showing the configuration of a mobile robot according to an embodiment. [Figure 2] FIG. 2 is a block diagram showing the detailed configuration of a measuring device. [Figure 3] FIG. 3 is a flowchart showing displacement amount calculation processing. [Figure 4] FIG. 4 is an explanatory diagram showing a base image and a transition region set in the base image. [Figure 5] FIG. 5 is an explanatory diagram showing the displacement amount of feature points included in each frame of a moving image when the mobile robot moves linearly. [Figure 6] FIG. 6 is an explanatory diagram showing an operation in which the base image is updated as the mobile robot moves. [Figure 7] FIG. 7 is a flowchart showing the operation of a mobile robot according to an embodiment. [Figure 8] FIG. 8 is an explanatory diagram showing the displacement of the center points of the base image and the target image as the mobile robot moves. [Figure 9A] FIG. 9A is a graph showing the results according to Example 1. [Figure 9B] FIG. 9B is a diagram showing the numerical values of the results according to Example 1. [Figure 10A] FIG. 10A is a graph showing the results according to Example 2. [Figure 10B] FIG. 10B is a diagram showing the numerical values of the results according to Example 2. [Figure 11] FIG. 11 is a block diagram showing the hardware configuration of the present embodiment.

Modes for Carrying Out the Invention

[0018] Embodiments of the present invention will be described below with reference to the drawings. Figure 1 is a perspective view showing the configuration of a mobile robot 1 according to an embodiment. In this embodiment, a four-wheeled mobile robot is used as an example for explanation, but the present invention is not limited to this and can be applied to mobile robots other than four-wheeled robots.

[0019] As shown in Figure 1, the mobile robot 1 is equipped with a rectangular parallelepiped mobile platform 31, four wheels V1, V2, V3, and V4, and a measuring device 10. The wheels V1 to V4 are, for example, omni-wheels. A camera 32 is mounted on the front of the mobile platform 31.

[0020] The mobile robot 1 travels on a travel surface 4. By controlling the rotation of each wheel V1 to V4, the mobile robot 1 can move in a desired direction on the travel surface 4. The travel surface 4 may be indoors or outdoors. The travel surface 4 is, for example, an outdoor road surface. The travel surface 4 is not necessarily flat and may have irregularities.

[0021] The camera 32 is installed so that its imaging direction faces the road surface 4 (i.e., downwards), and it captures moving images of the road surface 4 at a predetermined frame rate (for example, 30 fps). In other words, the camera 32 functions as an imaging unit that captures moving images of the road surface 4.

[0022] Figure 2 is a block diagram showing the configuration of the measuring device 10 and camera 32 mounted on the mobile robot 1.

[0023] As shown in Figure 2, the measuring device 10 includes an image setting unit 11, a displacement amount calculation unit 12, a transition area setting unit 13, a determination unit 15, an image update unit 16, a movement amount calculation unit 17, an estimation unit 18, a map creation unit 19, a storage unit 20, and a GPS receiving unit 21.

[0024] The image setting unit 11 sets an image from any frame included in the video captured by the camera 32 as the base image used to measure the amount of movement of the mobile robot 1. The arbitrary frame can be, for example, the frame at the start of video capture. For example, the image from the frame captured by the camera 32 at the initial position of the mobile robot 1 can be used as the base image.

[0025] Specifically, the image setting unit 11 sets an image from any frame of the video captured by the camera 32 as the base image. The image setting unit 11 sets a frame image after the base image as the target image to be displaced relative to the base image. Alternatively, a target image may be set for each frame of the video.

[0026] The displacement calculation unit 12 acquires a moving image of the road surface 4 captured by the camera 32. The displacement calculation unit 12 extracts feature points contained in each frame of the moving image of the road surface 4 in a time series at predetermined time intervals. For example, if the frame rate of the image captured by the camera 32 is 30 [fps], the predetermined time interval is 1 frame (0.03 seconds). Feature points include, for example, characters, symbols, patterns, dirt, shadows of protrusions, etc., painted on the road surface 4 included in the image. Note that the time interval for detecting feature points is not limited to 1 frame, but may be a predetermined number of frames. Also, the time interval for detecting feature points is not limited to a predetermined time interval, but may be any time interval.

[0027] The displacement calculation unit 12 extracts feature points for each frame of the moving image captured by the camera 32 and calculates the displacement of the extracted feature points. The displacement calculation unit 12 calculates the displacement of the feature points between each frame (for example, the number of pixels that have moved) using optical flow. The displacement calculation unit 12 outputs the displacement of the feature points to the determination unit 15 and the movement amount calculation unit 17. In other words, the displacement calculation unit 12 calculates the displacement of feature points included in the moving image accompanying the movement of the mobile robot 1 in a time series, based on the moving image of the road surface 4.

[0028] The following describes how to calculate the displacement between frames. Figure 3 is a flowchart showing the procedure for calculating the displacement between frames using the well-known visual odometry technique. As a preprocessing step for the process shown in Figure 3 (hereinafter referred to as the "displacement calculation process"), feature point detection is performed for each frame image. The displacement calculation process shown in Figure 3 is performed by the displacement calculation unit 12. Figure 3 illustrates an example of calculating the displacement of feature points between two frames (referred to as the first frame and the second frame).

[0029] In step S11 of Figure 3, the displacement calculation unit 12 extracts a feature vector from a feature point in the first frame to a feature point in the second frame. Feature vector extraction may be performed for multiple feature points.

[0030] In step S12, the displacement calculation unit 12 matches pairs of feature vectors extracted in the processing of S11 that are close in distance.

[0031] In step S13, the displacement calculation unit 12 excludes feature vectors that are not matched in the processing of S12. Furthermore, it calculates the rotation angle of the feature vectors.

[0032] In step S14, the displacement calculation unit 12 calculates the displacement amount for each feature point based on the distance between the feature point and the camera 32. That is, as shown in Figure 1, since the distance between the imaging plane of the camera 32 and the road surface 4 is approximately constant, it is possible to calculate the displacement amount of feature points between frames of the moving image captured by the camera 32.

[0033] In step S15, the average value of the feature displacement is calculated, and this average value is taken as the displacement between two frames, i.e., between the first frame and the second frame.

[0034] The "displacement calculation process" described above allows us to calculate the position of the second frame (the destination position) with respect to the first frame as the reference position. Specifically, the reference position and the destination position can be shown by equation (1) below.

[0035]

number

[0036] In equation (1), (x,y) are the displacement vectors of the base image and the target image to which the image is moved, and θ is the rotation angle.

[0037] Returning to Figure 2, the transition region setting unit 13 sets a reference position in the base image set by the image setting unit 11, and sets a transition region based on this reference position. In this embodiment, as an example, the center point of the base image is set as the reference position. Also in this embodiment, as an example, the region within a predetermined distance from the reference position is set as the transition region. That is, the transition region setting unit 13 sets a transition region R1 based on the reference position (for example, the center point) in the base image. The transition region R1 is, for example, a region within a radius r from the reference position.

[0038] The "transition region" is a threshold region used to determine whether or not to update the base image when the center point of the target image, obtained from a frame after the base image, moves farther from the center point of the base image. Specifically, if the center point of the target image is inside the transition region, the target image is not far enough from the base image, so the base image is not updated. On the other hand, if the center point of the target image is outside the transition region, the target image is far enough from the base image, so the base image is updated. Further details will be described later.

[0039] Figure 4 is an explanatory diagram showing the base image. For example, the image of the frame at the start of video capture by camera 32 can be used as the base image D1. As shown in Figure 4, the center point C1 of the base image D1 is set as the reference position, and the region within a predetermined distance from this center point C1 is set as the transition region R1. In Figure 4, the region with radius r from the center point C1 is set as the transition region R1. Note that the transition region does not have to be a region equidistant from the center point C1. For example, it may be an ellipse, rectangle, square, etc., centered on the center point C1. Also, the reference position does not have to be the center point C1.

[0040] The determination unit 15 determines, based on the displacement amount calculated by the displacement amount calculation unit 12, whether the center point of the target image is inside or outside the transition region R1 set in the base image D1 (see Figure 4). In other words, the determination unit 15 determines, based on the displacement amount, whether the reference position in the target image is inside the transition region R1.

[0041] When the determination unit 15 determines that the center point of the target image is outside the transition region R1, the image update unit 16 sets the image of the frame immediately preceding (immediately before) the target image as the new base image. The reason for updating the base image when the center point of the target image is determined to be outside the transition region R1 will be explained below.

[0042] Figure 5 is an explanatory diagram showing the displacement of feature points contained in each frame of a video when a mobile robot moves in one dimension. In Figure 5, it is assumed that images are acquired in the order of "Frame0", "Frame1", "Frame2", ... after the start of video capture.

[0043] Figure 5(b) shows the actual locations of feature points in each frame, "Frame0" to "Frame4".

[0044] Figure 5(a) shows an example (conventional method) of calculating the displacement of each frame relative to "Frame0" by sequentially accumulating the displacement between each frame, i.e., the displacement between adjacent frames. Specifically, displacement d0 is calculated from "Frame0" and "Frame1", and displacement d1 is calculated from "Frame1" and "Frame2". In other words, Figure 5(a) calculates the displacement by changing the reference position for each frame.

[0045] On the other hand, Figure 5(c) shows an example of calculating the displacement amounts for "Frame 1" to "Frame 4" using "Frame 0," the frame immediately after the start of imaging, as the reference point. Specifically, displacement d'0 is calculated from "Frame 0" and "Frame 1," displacement d'1 is calculated from "Frame 0" and "Frame 2," and displacement d'2 is calculated from "Frame 0" and "Frame 3." In other words, Figure 5(c) calculates the displacement amount without changing the reference position for each frame.

[0046] As can be understood by comparing Figures 5(b) and (a), when the reference position is changed for each frame and the amount of displacement is calculated, an error Δd occurs between each frame, and this error Δd is added as the number of frames increases. Specifically, the error in "Frame 3" is "3 * Δd".

[0047] On the other hand, as can be seen by comparing Figures 5(b) and (c), when the displacement is calculated without changing the reference position, the error Δd is almost constant in each frame. Specifically, the error in "Frame 3" is Δd, which is smaller than in the case of Figure 5(a).

[0048] In this embodiment, the displacement of each target image (corresponding to "Frame1" to "Frame4") is calculated based on the base image (corresponding to "Frame0"). When the displacement exceeds the radius r shown in Figure 4, the base image is updated, and the displacement of each target image is calculated using the updated base image as the new reference.

[0049] Specifically, in "Frame 4" shown in Figure 5(c), the displacement d'3 exceeds the radius r. Therefore, "Frame 3," which is the image immediately preceding "Frame 4," is set as the new base image and the displacement is calculated. As a result, in "Frame 4" shown in Figure 5(c), the displacement is the sum of the displacement d'2 and the displacement d''3. In this case, the error in the displacement is 2*Δd, which is smaller than the error of 4*Δd in "Frame 4" shown in Figure 5(a).

[0050] As described above, if the displacement of feature points is accumulated between each frame, an error will accumulate with each accumulation, and the error may increase as the number of frames increases. In this embodiment, the displacement of each target image is calculated based on the base image, and when the center point of the target image falls outside the transition region R1 (exceeds the radius r), the base image is updated by the image update unit 16 to reduce the error.

[0051] In other words, the image update unit 16 does not change the base image if the reference position in the target image is inside the transition region R1, and if it is outside, it sets the target image obtained from the frame before the reference position in the target image became outside the transition region R1 as the new base image.

[0052] In this embodiment, when it is determined that the center point of the target image is outside the transition region R1, an example is shown in which the image of the frame immediately preceding the target image is set as the new base image. However, an image from two or more frames prior may also be set as the new base image.

[0053] The movement amount calculation unit 17 shown in Figure 2 calculates the movement amount (movement distance, movement direction) of the mobile robot 1 based on the displacement amounts of feature points included in the base image and feature points included in the target image.

[0054] The method for calculating the amount of movement will be explained below with reference to Figure 6. Figure 6 is a schematic diagram illustrating the operation in which the base image is updated as the mobile robot 1 moves.

[0055] Figure 6(a) shows the base image D1, and Figures 6(b) and (c) show the target image D2 in the frame following the base image D1. Figure 6(d) shows the target image D3 in the frame following the target image D2, and Figure 6(e) shows the target image D4 in the frame following the target image D3.

[0056] As shown in Figure 6(a), the base image D1 contains feature points p1 and p2. Based on the displacement of feature points p1 and p2 in the target image D2 obtained after the base image D1, the displacement from the center point C1 of the base image D1 to the center point C2 of the target image D2 can be calculated.

[0057] Specifically, by comparing the feature points p1 and p2 contained in the base image D1 in Figure 6(a) with the feature points p1 and p2 contained in the target image D2 in Figure 6(b), and calculating the displacement, the displacement from the center point C1 of the base image D1 to the center point C2 of the target image D2 (indicated by arrow Y1) can be calculated, as shown in Figure 6(c).

[0058] As shown in Figures 6(b) and (c), the center point C2 of the target image D2 is inside the transition region R1, so the base image is not updated. The displacement from center point C1 to C2 becomes the movement amount of the mobile robot 1.

[0059] Similarly, as shown in Figure 6(d), when the target image D3 is obtained, the center point C3 of the target image D3 is inside the transition region R1, and the base image is not updated. The displacement is calculated based on the positions of feature points p1 and p2 included in the target image D3, and consequently, the displacement from the center point C1 to C3 (indicated by arrow Y2) becomes the movement amount of the mobile robot 1.

[0060] Next, as shown in Figure 6(e), once the target image D4 is obtained, the displacement is calculated based on the positions of feature points p1 and p2 included in the target image D4. This, in turn, calculates the displacement from the center point C1 of the base image D1 to the center point C4 of the target image D4 (indicated by arrow Y3). The displacement from center point C1 to C4 becomes the movement amount of the mobile robot 1.

[0061] In this case, since the center point C4 is outside the transition region R1, the image update unit 16 updates the base image. Specifically, the target image D3, which is one step prior to the target image D4, is set as the new base image. The transition region R1 is then set again on the updated base image, and the amount of movement of the mobile robot 1 is calculated by repeating the process shown in Figure 6(a) and subsequent steps.

[0062] Specifically, the movement amount calculation unit 17 uses the center point set in the base image (for example, C1 in Figure 6(a)) as a reference and calculates the movement amount of the mobile robot 1 based on the displacement of the center point of the target image obtained after the base image. Furthermore, if the center point of the target image is outside the transition region R1, the base image is updated, and the movement amount of the mobile robot 1 is calculated using the updated base image as a reference in the same process as described above.

[0063] Specifically, the movement amount calculation unit 17 calculates the movement amount of the mobile robot 1 based on the displacement amounts of feature points included in the base image and feature points included in the target image obtained from a frame after the base image.

[0064] Returning to Figure 2, the map creation unit 19 creates a map showing the path traveled by the mobile robot 1 based on the amount of movement of the mobile robot 1 calculated by the movement amount calculation unit 17.

[0065] The memory unit 20 stores the map created by the map creation unit 19.

[0066] The GPS receiver 21 communicates with GPS satellites and receives the position information of the mobile robot 1. The GPS receiver 21 outputs the position information of the mobile robot 1 to the estimation unit 18.

[0067] The estimation unit 18 receives initial position information of the mobile robot 1 from the GPS receiver 21. That is, the estimation unit 18 can acquire two-dimensional position information of the mobile robot 1 before it starts moving. Based on the amount of movement of the mobile robot 1 calculated by the movement amount calculation unit 17 and the two-dimensional position information of the mobile robot 1 before it starts moving, the estimation unit 18 estimates the two-dimensional position of the mobile robot 1 when it moves on the road surface 4. The estimation unit 18 outputs the estimated two-dimensional position information to an external device.

[0068] In other words, the estimation unit 18 estimates the two-dimensional position of the mobile robot 1 based on its initial position and the amount of movement.

[0069] Next, the operation of the mobile robot according to this embodiment, configured as described above, will be explained with reference to the flowchart shown in Figure 7 and the displacement diagrams of each frame image shown in Figure 8. The process shown in Figure 7 is performed by the measuring device 10 shown in Figure 2.

[0070] First, in step S31, the user sets the size of the transition region to be set in the base image. Specifically, the radius r shown in Figures 3 and 8 is set.

[0071] In step S32, when the camera 32 starts capturing video, the image setting unit 11 reads the first frame included in the video and sets it as the base image. For example, as shown in Figure 8, the initial frame of the video is set as the base image D1. Furthermore, a region with radius r centered on the center point C1 is set as the transition region R1.

[0072] Furthermore, the coordinates of the center point C1 of the base image D1 are set as the origin (0,0), the left-right direction of the mobile robot 1 is defined as the X-axis, and the front-back direction is defined as the Y-axis.

[0073] In step S33, the displacement calculation unit 12 detects feature points included in the base image set by the image setting unit 11. The feature points are as shown in Figure 6.

[0074] In step S34, the image setting unit 11 sets the next frame of the base image as the target image. Specifically, it sets the target image D2 shown in Figure 8.

[0075] In step S35, the displacement calculation unit 12 detects feature points included in the target image.

[0076] In step S36, the displacement calculation unit 12 performs the "displacement calculation process" shown in Figure 3 above to calculate the displacement amount d[dX,dY] and rotation angle [dθ] of the target image D2 relative to the base image D1.

[0077] For example, if the target image D2 shown in Figure 8 is set, the displacement d[dX,dY] and rotation angle [dθ] from the center point C1 of the base image D1 to the center point C2 of the target image D2 are calculated.

[0078] In step S37, the determination unit 15 determines whether the displacement amount d is less than or equal to the radius r. That is, it determines whether "(displacement amount) ≤ radius r". If the displacement amount d is less than or equal to the radius r (S37; YES), the process proceeds to step S41; otherwise (S37; NO), the process proceeds to step S38. Specifically, the center points C2 and C3 shown in Figure 8 are inside the transition region R1 of radius r, so the determination in S37 is YES, and the center point C4 is outside the transition region R1 of radius r, so the determination in S37 is NO.

[0079] In step S41, the movement amount calculation unit 17 calculates the movement amount (including movement distance and rotation angle) of the mobile robot 1 based on the center point (initially C1) of the base image (initially D1) and the center point of the target image. Specifically, it calculates the movement amount of the mobile robot 1 when it moves from the base image D1 to the target image D2 based on the displacement amounts of the center points C1 and C2 shown in Figure 8. It also calculates the movement amount of the mobile robot 1 when it moves from the base image D1 to the target image D3 based on the displacement amounts of the center points C1 and C3.

[0080] Furthermore, the estimation unit 18 estimates the current two-dimensional position of the mobile robot 1 based on the initial two-dimensional position of the mobile robot 1 received by the GPS receiver unit 21 and the amount of movement calculated by the movement amount calculation unit 17, i.e., the distance traveled and the azimuth angle. In addition, the map creation unit 19 creates a map showing the movement path of the mobile robot 1 and stores it in the storage unit 20.

[0081] In step S42, the image setting unit 11 determines whether or not it is the final frame of the moving image captured by the camera 32. If it is not the final frame (S42; NO), the process returns to step S34; otherwise (S42; YES), the process ends.

[0082] On the other hand, in the process of step S37 described above, if it is determined that the displacement amount d exceeds the radius r (S37; NO), then in step S38, a transition vector is set with respect to the center point C1. Specifically, the center point C4 shown in Figure 8 (the target image corresponding to C4 is not shown) is located outside the transition region R1. In this case, the transition vector M(MX,MY,Mθ) is set as a vector from the center point C1 to the center point C3 of the target image D3, which is the frame immediately preceding the target image.

[0083] In step S39, the image update unit 16 updates the base image. Specifically, as shown in Figure 6 above, when the center of the target image falls outside the transition region set for the base image, the image from the frame immediately preceding this frame, i.e., the target image D3, is set as the new base image. Furthermore, for the new base image (D3), a new transition region is set for a circle with radius r extending from the center point.

[0084] In step S40, the displacement calculation unit 12 sets feature points for the new base image and returns to step S36. Since the feature points included in the previous frame have already been detected, these feature points may be copied, or feature point detection may be performed again. In this way, it becomes possible to measure the amount of movement of the mobile robot while updating the base image as appropriate. As shown in Figure 8, the images move sequentially from C4 to C5 to... to C16, reaching the target image D16. That is, by updating the base image and calculating the amount of displacement up to each target image, the two-dimensional position coordinates of the mobile robot 1 can be estimated.

[0085] In other words, as shown in Figure 8, when the target image D3 is set, the movement of the mobile robot 1 is not taken as the sum of the displacement of center point C2 relative to center point C1 and the displacement of center point C3 relative to center point C2, but rather as the displacement of center point C3 relative to center point C1. Therefore, as explained in Figure 5 above, it is possible to reduce the error included in the movement.

[0086] Next, we will describe the results of experiments conducted in which the mobile robot 1 was actually driven. The inventors conducted the experiments shown in the first and second embodiments below. The details will be explained below with reference to Figures 9A, 9B, 10A, and 10B.

[0087] (First embodiment) As the first embodiment, a floor imaging camera (corresponding to camera 32 shown in Figure 1) for capturing floor motion images was installed on a hand truck (hereinafter simply referred to as "cart") that simulated a mobile robot 1, and an experiment was conducted in which two-dimensional self-position estimation was performed using floor motion images. Figure 9A is a graph showing the results of self-position estimation. Figure 9B is a diagram showing the numerical values ​​of the above experimental results.

[0088] In the first embodiment, an external camera capable of capturing images of the entire trolley from above was used to capture the path the trolley traveled, and the two-dimensional position of the trolley was calculated based on this image. Furthermore, this was compared with the result of estimating the two-dimensional position of the trolley using a floor-surface imaging camera. In Figure 9A, the solid curve q1 shows the measurement result when using the external camera, and the dashed curve q2 shows the result when two-dimensional position self-position estimation was performed using the floor-surface imaging camera.

[0089] In measurements using an external camera, the distance from the external camera's imaging surface to the floor is compared with the pixel size of the image sensor mounted on the external camera and the actual size of the image being captured. As a result, the actual size of the image per pixel of the image sensor, i.e., the resolution, can be calculated. The difference between the calculated resolution and the estimated displacement result calculated without setting the transition region R1 is defined as the error in the displacement estimation result.

[0090] The estimation of the trolley's movement using moving images captured by a floor-surface camera employs the process shown in the flowchart of Figure 7. However, in the first embodiment, the radius r representing the transition region R1 is set to "0". That is, in Figure 4, the radius r=0, which is equivalent to not setting the transition region R1. The experimental procedure is as shown in (a) to (c) below. (a) Video recording (a1) Move the cart forward diagonally to the left in an arc.

[0091] (a2) The trolley is temporarily stopped at the point where its angle reaches 90 degrees relative to its initial position (stopping point).

[0092] (a3) The cart is reversed to retrace the path it took to the point where it stopped, and then moved back to the starting point (final point). In other words, the cart is moved back and forth from the starting point → stopping point → final point.

[0093] (a4) At this time, video is captured by both an external camera installed above the trolley and a floor-mounted camera installed on the trolley. (b) Displacement calculation using an external camera The amount of movement of the trolley is calculated based on consecutive frames of images captured by an external camera. (c) Displacement calculation using road surface motion images The displacement calculation process shown in Figure 3 is used to calculate the amount of movement of the trolley.

[0094] In the experiments described in (a) to (c) above, the cart was moved manually by an operator at an average speed of approximately 10 cm / sec. Furthermore, the resolution of the external camera was measured and found to be 3 mm.

[0095] From the curves q1 and q2 shown in Figure 9A, it was shown that when the trolley was moved over an area of ​​approximately (1500 mm) × (1200 mm), there was a certain error between the two-dimensional position estimation result from the external camera and the two-dimensional position estimation result from the floor imaging camera. In other words, when the radius r is set to "0" and the transition region R1 is not set, the amount of movement is calculated by accumulating the displacement amounts between images of adjacent frames. As shown in Figure 9B, an error of 55.7 mm occurred at the stopping point (end of the outward journey), and an error of 43.8 mm occurred at the final point (end of the return journey).

[0096] (Second example) In the second embodiment, a transition region R1 was set on the base image D1 shown in Figure 4, and the base image was updated only when the center point of the target image moved outside the transition region R1. The same experiment as in the first embodiment described above was then conducted using this algorithm.

[0097] Specifically, in the displacement calculation process shown in (c) above, the amount of trolley movement was calculated by setting the transition region R1 to a radius r=5mm. Also, similar to the first embodiment described above, the amount of trolley movement was calculated without setting the transition region R1 (i.e., with the radius of the transition region R1 set to r=0).

[0098] In Figure 10A, the solid curve q11 shows the two-dimensional position of the trolley calculated from images captured by the upper camera. The dashed curve q12 shows the two-dimensional position of the trolley calculated using moving images of the road surface captured by a floor-mounted camera installed on the trolley, without setting a transition region R1. The dashed line curve q13 shows the two-dimensional position of the trolley calculated using moving images of the road surface captured by a floor-mounted camera installed on the trolley, with a transition region R1 of radius 5 mm set.

[0099] From the curves q11 and q12 shown in Figure 10A, it was shown that the self-position estimation result of the trolley calculated without setting the transition region R1 had a certain error compared to the self-position estimation result of the trolley calculated based on video footage captured by an external camera. In contrast, curves q11 and q13 were almost identical, and the result obtained was that the self-position estimation result of the trolley calculated with the transition region R1 set was almost identical to the self-position estimation result of the trolley calculated based on video footage captured by an external camera.

[0100] Specifically, as shown in Figure 10B, at the end point (stopping point) of the outward journey, an error of 58.2 mm occurred when the transition region R1 was not set, whereas the error was 8.1 mm when the transition region R1 was set (indicated as "radius 5 mm"). Similarly, at the end point (final point) of the return journey, the error was 43.9 mm when the transition region R1 was not set, compared to 0.5 mm when the transition region R1 was set.

[0101] By using an algorithm that sets a transition region in the base image and updates the base image only when the center point of the target image moves outside the transition region, we obtained results that allow for highly accurate estimation of the two-dimensional position of the trolley.

[0102] As described above, the mobile robot according to this embodiment includes a camera 32 (imaging unit) that captures moving images of the road surface 4, an image setting unit 11 that sets an arbitrary frame of the moving images (for example, the initial frame) as the base image, a transition region setting unit 13 that sets a transition region R1 based on a reference position in the base image, a displacement amount calculation unit 12 that calculates the amount of displacement of feature points between images based on feature points included in the base image and the target image obtained from a frame after the base image, a determination unit 15 that determines whether or not the reference position in the target image is inside the transition region R1 based on the amount of displacement, an image update unit 16 that, if the reference position in the target image is inside the transition region R1, does not change the base image, and if it is outside, sets the target image obtained from a frame before the reference position in the target image becomes outside the transition region R1 as the new base image, and a movement amount calculation unit 17 that calculates the amount of movement of the mobile robot 1 based on the amount of displacement of feature points included in the base image and feature points included in the target image.

[0103] In this embodiment, when the center point of the target image lies inside the transition region R1, the displacement of the target image is calculated based on the feature points or center point of the base image, thus enabling highly accurate calculation of the movement of the mobile robot 1. Therefore, it becomes possible to estimate the movement with high accuracy even when using visual odometry.

[0104] Furthermore, in this embodiment, the image of the frame immediately preceding the point where the center point of the target image falls outside the transition region R1 set in the base image is set as the new base image, making it possible to calculate the amount of movement of the mobile robot 1 with higher accuracy.

[0105] In other words, as shown in Figure 5(a), in the method of accumulating the displacement amount for each frame included in the moving image, the error Δd is accumulated, resulting in a large overall error. However, in this embodiment, as shown in Figure 5(c), the displacement amount is calculated based on one base image until the base image is updated, making it possible to reduce the error.

[0106] In this embodiment, a target image is set for each frame included in the moving image captured by the camera 32, and the displacement amount relative to the base image is calculated, making it possible to calculate the amount of movement of the mobile robot 1 with higher accuracy.

[0107] In this embodiment, the center point of the transition region R1 is set to the center of the base image, making it possible to calculate the amount of movement of the mobile robot 1 with higher accuracy.

[0108] In this embodiment, a reference position (e.g., a center point) is set in the base image, and the range of radius r from this reference position (i.e., an area within a predetermined distance) is set as the transition region R1. This makes it possible to more accurately recognize whether the center of the target image is inside or outside the transition region and calculate the amount of movement of the mobile robot 1.

[0109] In this embodiment, since no special equipment such as maps or surrounding environment information is required, the mobile robot 1 can be constructed at a low cost. Furthermore, by enabling the reduction of errors, it becomes possible to handle situations where the mobile robot 1 travels long distances.

[0110] As shown in Figure 11, the mobile robot 1 of this embodiment can use a general-purpose computer system that includes, for example, a CPU (Central Processing Unit, processor) 901, memory 902, storage 903 (HDD: Hard Disk Drive, SSD: Solid State Drive), communication device 904, input device 905, and output device 906. The memory 902 and storage 903 are storage devices. In this computer system, the various functions of the mobile robot 1 are realized by the CPU 901 executing a predetermined program loaded onto the memory 902.

[0111] The mobile robot 1 may be implemented on a single computer, or on multiple computers. Furthermore, the mobile robot 1 may also be a virtual machine implemented on a computer.

[0112] The program for mobile robot 1 can be stored on computer-readable storage media such as HDDs, SSDs, USB (Universal Serial Bus) memory, CDs (Compact Discs), and DVDs (Digital Versatile Discs), or it can be distributed via a network.

[0113] It should be noted that the present invention is not limited to the embodiments described above, and numerous modifications are possible within the scope of its essence. [Explanation of Symbols]

[0114] 1 Mobile robot 4. Driving surface 10 Measuring device 11 Image Settings Section 12 Displacement Amount Calculation Unit 13 Transition area setting section 15 Judgment section 16 Image Update Section 17 Movement Amount Calculation Unit 18 Estimation part 19 Map Creation Department 20 Memory section 21 GPS receiver 31 Mobile cart 32 cameras R1 transition region

Claims

1. An imaging unit that captures moving images of the road surface, An image setting unit sets any frame of the aforementioned video as the base image, A transition region setting unit sets a transition region based on a reference position in the base image, A displacement calculation unit calculates the amount of displacement of feature points between images based on the feature points contained in the base image and the target image obtained from a frame after the base image. A determination unit that determines whether the reference position in the target image is inside the transition region based on the displacement amount, If the reference position in the target image is inside the transition region, the base image is not changed; if it is outside, the image update unit sets the target image obtained from the frame immediately preceding the frame in which the reference position in the target image is outside the transition region, or from a frame earlier than that frame, as the new base image. A movement amount calculation unit calculates the amount of movement of a mobile robot based on the displacement amounts of the feature points included in the base image and the feature points included in the target image. A mobile robot equipped with [a specific feature / equipment].

2. Estimation unit that estimates the two-dimensional position of the mobile robot based on the initial position of the mobile robot and the amount of movement. The mobile robot according to claim 1, further comprising the above.

3. If the reference position in the target image is outside the transition region, the image update unit sets the target image obtained from the frame immediately preceding the frame in which the reference position in the target image is outside the transition region as the new base image. The mobile robot according to claim 1 or 2.

4. The target image includes each frame image contained in the video. The mobile robot according to claim 1.

5. The reference position in the base image includes the center point of the base image. The mobile robot according to claim 1.

6. The transition region includes a region within a predetermined distance from the reference position in the base image. The mobile robot according to claim 1.

7. The steps include capturing moving images of the road surface, The steps include setting any frame of the aforementioned video as the base image, The steps include setting a transition region based on a reference position in the base image, A step of calculating the amount of displacement of the feature points between images based on the feature points contained in the base image and the target image obtained from the frame after the base image, The steps include determining whether the reference position in the target image is inside the transition region based on the displacement amount, If the reference position in the target image is inside the transition region, the base image is not changed; if it is outside, the target image obtained from the frame immediately preceding the frame in which the reference position in the target image is outside the transition region, or from a frame earlier than that frame, is set as the new base image. A step of calculating the amount of movement of a mobile robot based on the displacement amounts of the feature points included in the base image and the feature points included in the target image. A method for estimating the amount of movement, comprising the following features.

8. A mobile robot control program that causes a computer to function as a mobile robot according to claim 1 or 2.

Images