Moving body control device, moving body control method, and non-transitory computer-readable medium

Abstract

The purpose of the present disclosure is to provide a moving body control device, a moving body control method, and a program that improve the accuracy of recognizing a target object using a sensor mounted on a moving body, without controlling the sensing range of the sensor. A moving body control device (10) according to the present disclosure comprises: a sensor information acquisition unit (11) that acquires environmental information from a sensor mounted on a moving body; a target object position and orientation estimation unit (12) that estimates position and orientation information of a target object in response to the acquisition of the environmental information; a determination unit (13) that determines whether or not the estimation accuracy of the position and orientation information of the target object is low; and a movement mode switching unit (14) that, if the estimation accuracy is low, switches the movement mode of the moving body to an approach mode in which the moving body approaches the target object while the sensor is still capturing the target object.

Description

MOBILE BODY CONTROL DEVICE, MOBILE BODY CONTROL METHOD, AND NON-TRANSITORY COMPUTER-READABLE MEDIUM

[0001] The present disclosure relates to a mobile control device, a mobile control method, and a non-transitory computer-readable medium.

[0002] Patent Document 1 discloses a technique for manipulating the camera's imaging range so that, when the recognition accuracy of a feature in the vehicle's environment is low, the feature remains within the camera's imaging range.

[0003] Japanese Patent Application Laid-Open No. 2022-151680

[0004] If the sensing range of a sensor mounted on a moving object cannot be controlled, it becomes difficult to improve the accuracy of object recognition.

[0005] In view of the above-mentioned problems, one of the objectives of the present disclosure is to provide a mobile body control device, a mobile body control method, and a program that improve the accuracy of object recognition by a sensor mounted on the mobile body without manipulating the sensing range of the sensor.

[0006] A mobile body control device according to a first aspect of the present disclosure includes: a sensor information acquisition means for acquiring environmental information from a sensor mounted on a mobile body; an object position and orientation estimation means for estimating position and orientation information of an object in accordance with the acquired environmental information; a determination means for determining whether the estimation accuracy of the position and orientation information of the object is low; and a movement mode switching means for, if the estimation accuracy is low, switching the movement mode of the mobile body to an approach mode in which the mobile body approaches the object while the sensor keeps capturing the object.

[0007] A mobile body control method according to a second aspect of the present disclosure includes acquiring environmental information from a sensor mounted on the mobile body, estimating position and orientation information of an object in accordance with the acquired environmental information, determining whether the estimation accuracy of the position and orientation information of the object is low, and if the estimation accuracy is low, switching the movement mode of the mobile body to an approach mode in which the mobile body approaches the object while the sensor keeps capturing the object.

[0008] A program according to a third aspect of the present disclosure causes a computer to acquire environmental information from a sensor mounted on a moving body, estimate position and orientation information of an object in accordance with the acquired environmental information, determine whether the estimation accuracy of the position and orientation information of the object is low, and if the estimation accuracy is low, switch the movement mode of the moving body to an approach mode in which the moving body approaches the object while the sensor keeps capturing the object.

[0009] The present disclosure makes it possible to provide a mobile body control device, a mobile body control method, and a program that improve the accuracy of object recognition by a sensor mounted on the mobile body without manipulating the sensing range of the sensor.

[0010] Fig. 1 is a flowchart of a control method according to a comparative example. Fig. 2 is a block diagram of a mobile body control device according to the present disclosure. Fig. 3 is a flowchart of a mobile body control method according to the present disclosure. Fig. 4 is a block diagram of a mobile body control device according to the present disclosure. Fig. 5 is a diagram explaining processing according to the present disclosure. Fig. 6 is a flowchart of a mobility control method according to the present disclosure. Fig. 7 is a block diagram explaining the hardware configuration of a mobile body control device according to the present disclosure.

[0011] Background to the embodiment An autonomously moving forklift needs to move to a position directly opposite the pallet before transporting the pallet. The forklift first sets the approximate location of the pallet as a goal position and then autonomously moves to the vicinity of the goal position. The forklift captures an image of the pallet with a camera while autonomously moving. The camera is mounted on the forklift. The forklift then estimates the position and orientation information of the pallet based on the captured image, and sets a goal position directly opposite the pallet based on the estimation result. The forklift then autonomously moves toward the goal position.

[0012] When the distance from the forklift to the pallet is long, the estimation accuracy of the pallet's position and orientation information is low. Also, if the pallet is not within the camera's shooting range while the forklift is moving autonomously, the forklift cannot update the estimation results of the pallet's position and orientation information, and the estimation accuracy decreases. The estimation accuracy of the pallet's position and orientation information also decreases when the forklift is moving fast.

[0013] Generally, cameras with long recognition distances have narrow viewing angles, while cameras with wide viewing angles have short recognition distances. When a camera with a long recognition distance is used, the pallet is likely to go outside the camera's range of view. When a camera with a wide viewing angle is used, the distance at which the pallet can be recognized is short, making it difficult for the forklift to move autonomously toward the goal position.

[0014] 1 is a flowchart showing a control method for a forklift according to a comparative example. First, the forklift sets a rough goal position (step S101). Next, the forklift autonomously moves toward the goal position (step S102). Next, the forklift determines whether a camera mounted on the forklift has recognized a pallet (step S103). If a pallet is not recognized, step S107 is executed.

[0015] If the pallet is recognized, the forklift updates the estimated position and orientation of the pallet based on the latest pallet recognition result and previous recognition results (step S104). The forklift then determines whether the distance from the current position of the forklift to the goal position is equal to or greater than a predetermined distance (e.g., one meter) (step S105). If the distance is less than the predetermined distance, step S107 is executed. If the distance is equal to or greater than the predetermined distance, the forklift sets a goal position directly facing the pallet based on the updated estimated position and orientation of the pallet (step S106). The forklift then determines whether the forklift has reached the goal position (step S107). If the forklift has not reached the goal position, step S102 is executed. If the forklift has reached the goal position, the process ends.

[0016] The forklift must face the pallet within an allowable range of approximately 6 cm to the left and right of the center of the pallet. Therefore, in step S107, a determination is made within a similarly small allowable range. However, the goal position may fluctuate due to fluctuations in the estimated positions and attitudes of the forklift and pallet. If the goal position continues to fluctuate while the forklift is near the goal position, there is a risk that the forklift will not be able to reach the goal position.

[0017] The inventors of the present application discovered a problem in that the forklift in the comparative example cannot update the goal position to the correct position, which may cause the forklift to be unable to face the pallet. If the pallet's position estimated in advance by the forklift differs from the actual position of the pallet, the accuracy of the pallet's position estimation may be low, and the forklift may be unable to face the pallet. For example, if the forklift estimates the pallet's position at a point far from the pallet, the accuracy of the pallet's position estimation may be low. If the pallet moves out of the camera's viewing range as the forklift moves toward the goal position, the forklift cannot update the estimated pallet's position. When the forklift arrives near the goal position, the pallet enters the camera's viewing range. In this case, because the forklift's position is too close to the goal position, the forklift cannot update the goal position and arrives at a position that is deviated from the position directly facing the pallet. In other words, if the estimated position and estimated orientation of the pallet fluctuate significantly, the forklift's position may shift significantly laterally from the position directly facing the pallet, making it difficult for the forklift to reach the goal position.

[0018] Therefore, the present disclosure provides a mobile object control device that improves the estimation accuracy of the position and orientation of a pallet.

[0019] Embodiment 1 Hereinafter, an embodiment of the present disclosure will be described with reference to the drawings. First, a configuration example of a mobile body control device 10 according to the present disclosure will be described using FIG. 2. The mobile body control device 10 may be a computer device that operates when a processor executes a program stored in a memory. The mobile body control device 10 may be an information processing device, for example, a server device. The mobile body control device 10 may also be composed of multiple computer devices. In this case, the components or functions that make up the mobile body control device 10 may be distributed and arranged across the multiple computer devices. The multiple computers may be connected via a network or directly via a cable or the like.

[0020] The mobile object control device 10 has an information acquisition unit 11, an object position and orientation estimation unit 12, a determination unit 13, and a travel mode switching unit 14. The information acquisition unit 11, the object position and orientation estimation unit 12, the determination unit 13, and the travel mode switching unit 14 may be software or modules that are executed by a processor executing a program stored in memory. Alternatively, the information acquisition unit 11, the object position and orientation estimation unit 12, the determination unit 13, and the travel mode switching unit 14 may be hardware such as a circuit or a chip.

[0021] The information acquisition unit 11 acquires environmental information from sensors mounted on the mobile object. The sensors include cameras (e.g., stereo cameras), LiDAR, and ultrasonic sensors. The mobile object may be an autonomously moving vehicle, a remotely controlled vehicle, or a vehicle that moves with computer-assisted driver operation. The mobile object may also be a vehicle controlled by a non-holonomic system. The mobile object may also be a transport vehicle that transports luggage, such as a forklift. The mobile object may also be a mobile object that moves autonomously, i.e., an autonomous mobile robot.

[0022] The object position and orientation estimation unit 12 estimates the position and orientation information of an object in response to acquired environmental information. The object is an object that a moving object is approaching. The object may be, for example, a pallet being transported by a forklift. The object position and orientation estimation unit 12 may periodically determine whether or not the object has been recognized, and may estimate the position and orientation information of the object when it is determined that the object has been recognized.

[0023] The determination unit 13 determines whether the estimation accuracy of the position and orientation information of the object is low. The determination unit 13 may determine whether the estimation accuracy is low based on, for example, the variance of the position and orientation information of the object estimated in a certain time interval. The determination unit 13 may determine that the estimation accuracy is low when the variance is large.

[0024] If the estimation accuracy is low, the movement mode switching unit 14 switches the movement mode of the moving object to an approach mode in which the moving object approaches the target object while the sensor keeps capturing the target object. Note that if the movement mode is not the approach mode, the moving object may move along the shortest route, etc.

[0025] Next, the flow of the mobile object control method according to the present disclosure will be described with reference to FIG. 3. First, the information acquisition unit 11 acquires environmental information from a sensor mounted on the mobile object (step S201). Next, the object position and orientation estimation unit 12 estimates the position and orientation information of the object based on the acquired environmental information (step S202). Next, the determination unit 13 determines whether the estimation accuracy of the object position and orientation information is low (step S203). If the estimation accuracy is low, the travel mode switching unit 14 switches the travel mode of the mobile object to an approach mode in which the mobile object approaches the object while the sensor keeps capturing the object (step S204). Steps S201 to S204 may be executed repeatedly.

[0026] As described above, when the estimation accuracy of the position and orientation information of the object is low, the mobile body control device 10 switches the movement mode of the mobile body to the approach mode. As a result, the mobile body control device 10 can improve the estimation accuracy of the position and orientation information of the object.

[0027] Embodiment 2 Embodiment 2 is a specific example of embodiment 1. FIG. 4 is a block diagram showing the configuration of a mobile body control device 20 according to the present disclosure. The mobile body control device 20 is a specific example of the mobile body control device 10. The mobile body control device 20 may be a computer device that operates when a processor executes a program stored in a memory. The mobile body control device 20 controls the operation of a forklift, which is a mobile body. In embodiment 2, an example will be described in which the mobile body control device 20 controls the operation of a forklift, but the control target of the mobile body control device 20 is not limited to a forklift.

[0028] For example, the mobile object control device 20 may be mounted on a forklift and control the drive unit 213 of the forklift. Alternatively, the mobile object control device 20 may communicate with a computer device that controls the drive unit 213 of the forklift via a network and remotely control the forklift. Controlling the drive unit 213 may, for example, involve controlling the steering operation of the forklift. The steering operation may involve determining the angle of the direction of movement of the forklift relative to the forward direction of the forklift. The steering operation is determined, for example, according to the turning radius of the forklift.

[0029] The mobile object control device 20 generates a travel path for a forklift performing loading and unloading work, and the forklift travels along the travel path generated by the mobile object control device 20. The following describes an example in which the mobile object control device 20 generates a travel path for moving the forklift to a position directly opposite a pallet placed at a destination position. The position directly opposite the pallet is a position where the forks of the forklift can be inserted into holes in the pallet by moving the forklift forward. In other words, the position directly opposite the pallet may be a position where a straight line extending in the extension direction of the holes in the pallet into which the forks are inserted coincides with a straight line extending in the extension direction of the forks of the forklift. Moving the forklift to a position directly opposite the pallet may also be referred to as an alignment process.

[0030] The mobile body control device 20 includes a sensor information acquisition unit 21, a robot position and attitude acquisition unit 22, an object position and attitude estimation unit 23, an object position and attitude storage unit 24, a goal position and attitude setting unit 25, a path acquisition unit 26, an object stability determination unit 27, a sensing range prediction unit 28, a travel mode switching unit 29, a travel mode storage unit 210, a control target calculation unit 211, a machine control unit 212, and a drive unit 213. Each of the components constituting the mobile body control device 20 may be software or a module whose processing is performed by a processor executing a program stored in memory. Alternatively, each of the components constituting the mobile body control device 20 may be hardware such as a circuit or a chip. The information acquisition unit 11 in the mobile body control device 10 corresponds to the sensor information acquisition unit 21 in the mobile body control device 20. The object position and attitude estimation unit 12 in the mobile body control device 10 corresponds to the object position and attitude estimation unit 23 in the mobile body control device 20. The determination unit 13 in the mobile body control device 10 corresponds to the object stability determination unit 27 in the mobile body control device 20. The travel mode switching unit 14 in the mobile body control device 10 corresponds to the travel mode switching unit 29 in the mobile body control device 20 .

[0031] The sensor information acquisition unit 21 acquires environmental information from sensors (e.g., LiDAR, ultrasonic camera, stereo camera) attached to the forklift. Specifically, the sensor information acquisition unit 21 acquires images captured using a camera attached to the forklift. The camera may be attached, for example, at a position where it can capture an image of the front of the forklift. The front of the forklift may be the direction of travel when the forklift is moving forward. Furthermore, the camera may also be attached at a position where it can capture an image of the direction of travel when the forklift is moving backward. The image captured using the camera may be, for example, an RGB (Red Green Blue) image.

[0032] Specifically, the robot position and orientation acquisition unit 22 acquires the position and orientation of the forklift by Visual SLAM (Simultaneous Localization and Mapping) using camera images. The robot position and orientation acquisition unit 22 may estimate the position and orientation of the forklift based on environmental information acquired from a sensor other than a camera (e.g., LiDAR). The robot position and orientation acquisition unit 22 may also estimate the position and orientation of the forklift using a Global Navigation Satellite System (GNSS), detection results from magnetic tape, or Ultra Wide Band (UWB).

[0033] The object position and orientation estimation unit 23 estimates the position and orientation information of the pallet in response to the acquired environmental information. Specifically, the object position and orientation estimation unit 23 estimates the position and orientation of the pallet based on the captured video. The object position and orientation estimation unit 23 stores the estimated position and orientation information in the object position and orientation storage unit 24.

[0034] The object position and orientation storage unit 24 is a storage device such as a hard disk, a flash memory, etc. The object position and orientation storage unit 24 stores the estimation results of the position and orientation information of the pallet.

[0035] The goal position and attitude setting unit 25 sets a goal position and attitude at which the forklift directly faces the pallet based on the position and attitude of the forklift and the position and attitude information of the pallet. The goal position and goal attitude are updated according to the estimation of the position and attitude information of the pallet.

[0036] The path acquisition unit 26 acquires a travel path for the forklift to reach the goal position and goal posture. Specifically, the path acquisition unit 26 acquires a travel path based on an algorithm called ReedsShepp. ReedsShepp calculates a path connecting the start position and the goal position using circles and straight lines. The start position corresponds to the current position and posture of the forklift. The path acquisition unit 26 can acquire a travel path that is short in distance and conforms to the motion model of the forklift.

[0037] The object stability determination unit 27 determines whether the estimation accuracy of the object's position and orientation estimation information is low based on the variance of the pallet's position and orientation estimation information estimated in a certain time interval. The object stability determination unit 27, for example, calculates the variance of the goal position and goal orientation set in a certain time interval. The variance of the goal position and goal orientation corresponds to the variance of the pallet's position and orientation information. If the variance is equal to or greater than a threshold, the object stability determination unit 27 determines that the estimation accuracy of the object's position and orientation information is low. For example, if the goal position and orientation is (g ｘ , g ｙ , g θ ), g in a certain time interval ｘ , g ｙ , g θ The variances of are σ ｇｘ , σ ｇｙ , σ ｇθ Here, an arbitrary threshold is expressed as T ｘ , T ｙ , T θ However, T ｘ >0, T ｙ >0, T θ In this case, the object stability determination unit 27 determines whether (σ ｇｘ <T ｘ ) and (σ ｇｙ <T ｙ ) and (σ ｇθ <T θ ) is satisfied, the estimation accuracy is determined to be high, and if not, the estimation accuracy is determined to be low. The object stability determination unit 27 may perform a similar process using the variance of the position and orientation information stored in the object position and orientation storage unit 24.

[0038] The sensing range prediction unit 28 simulates the movement of the sensing range (e.g., shooting range) of the sensor as the mobile object moves. The sensing range is determined according to the position and posture of the mobile object. Specifically, the sensing range prediction unit 28 simulates the movement of the sensing range while the mobile object travels for a predetermined time (e.g., T seconds).

[0039] If the object stability determination unit 27 determines that the estimation accuracy is high, the travel mode switching unit 29 sets the travel mode of the forklift to normal mode. If the object stability determination unit 27 determines that the estimation accuracy is low, the travel mode switching unit 29 sets the travel mode of the forklift to approach mode. The travel mode switching unit 29 stores information indicating the travel mode of the forklift in the travel mode storage unit 210. A forklift set in normal mode travels along a travel path (e.g., shortest path) acquired by the path acquisition unit 26. A forklift set in approach mode approaches the pallet while the sensor keeps capturing the pallet.

[0040] The travel mode storage unit 210 is a storage device such as a hard disk, a flash memory, etc. The travel mode storage unit 210 stores information indicating the travel mode set for the forklift.

[0041] The control target calculation unit 211 refers to the travel mode storage unit 210 and calculates a control target value (e.g., steering angle) according to the travel mode of the forklift. When the travel mode of the forklift is the normal mode, the control target calculation unit 211 calculates a control target value (referred to as a first control target value) according to the travel path acquired by the path acquisition unit 26. The control target calculation unit 211 calculates the control target value using an algorithm such as Pure-Pursuit so as to minimize deviation of the position of the forklift from the travel path. In Pure-Pursuit, a target point is set on the path, and the steering angle is controlled so that the forklift moves toward the target point.

[0042] The control target calculation unit 211 calculates a control target value so that the forklift approaches the pallet while keeping the sensor in touch with the pallet. The control target value is based on the simulation results by the sensing range prediction unit 28. Specifically, the control target calculation unit 211 uses the sensing range prediction unit 28 to search for a control target value that satisfies the condition that the pallet is included in the sensor's sensing range when the forklift travels for T seconds according to the control target value. The control target calculation unit 211 performs, for example, a first process to determine whether the sensor is capturing an object when the control target value is a second control target value different from the first control target value. Then, if it is determined that the sensor is not capturing an object, the control target calculation unit 211 performs a second process to increase or decrease the second control target value by a predetermined amount. By repeating the first process and the second process, the control target calculation unit 211 can search for an appropriate control target value.

[0043] Control of the movement of a forklift will now be described. Let ω be the rotation speed of the steering, L be the wheelbase length of the forklift, x and y be the position of the forklift, and θ be the attitude angle representing the attitude of the forklift. Then, let φ be the steering angle and v be the speed of the forklift, and equation (1) holds. The sign of dφ is determined by the magnitude relationship between the target steering angle and the current steering angle. Equation (1)

[0044] If the relative position of the camera with respect to the forklift, the camera's angle of view, and the camera's recognizable distance are measured in advance, the sensing range prediction unit 28 can simulate movement of the sensing range using the model expressed by equation (1). The sensing range prediction unit 28 can predict the sensing range when the forklift moves along its movement path up to T seconds in the future. Therefore, the control target calculation unit 211 can use the sensing range prediction unit 28 to determine whether a pallet is present within the camera's shooting range T seconds in the future.

[0045] A method for calculating the control target value in the approach mode will be described. The first control target value of the steering for moving along the movement path is calculated as φ ｐａｔｈThe step size of the control target value is s, and the maximum value of the control target value is φ ｍａｘ The minimum control target value is set as φ ｍｉｎ Then, the steering is controlled to a first control target value φ ｐａｔｈ When a pallet is present in the direction cut in the positive direction from φ, the process performed by the mobile object control device 20 is shown in FIG. ｔａｒｇｅｔ Then, when it is determined that the camera continues to capture the pallet, the control target calculation unit 211 calculates the steering control target value φ ｔａｒｇｅｔ On the other hand, if it is not determined that the camera is still capturing the pallet, then φ ｔａｒｇｅｔ In addition, when the pallet is located in the direction in which the steering wheel is turned in the negative direction, the first line of FIG. 5 becomes for(φ=φ ｐａｔｈ -s; φ≦φ ｍｉｎ ;φ ｔａｒｇｅｔ =φ ｔａｒｇｅｔ -s). Furthermore, the control target calculation unit 211 may limit the speed of the forklift by controlling not only the steering but also the throttle.

[0046] 4, the machine control unit 212 controls the drive unit 213 in accordance with the control target value calculated by the control target calculation unit 211. The drive unit 213 may be a steering mechanism provided in the forklift. The drive unit 213 may include a throttle that controls the speed of the forklift.

[0047] 6 is a flowchart showing a mobile object control method according to the present disclosure. First, the object stability determination unit 27 determines whether the estimation accuracy of the position and orientation information of the pallet is high (step S301). Note that the position and orientation information of the pallet may also be estimated in step S301. If the estimation accuracy is high, the travel mode switching unit 29 sets the travel mode of the forklift to normal mode (step S302). Then, the control target calculation unit 211 calculates a control target value according to the travel route set by the route acquisition unit 26. As a result, the forklift moves along the travel route (step S303).

[0048] If the estimation accuracy is low, the travel mode switching unit 29 determines whether the travel mode of the forklift is the normal mode (step S304). If the travel mode of the forklift is not the normal mode, that is, if the travel mode of the forklift is the approach mode, step S307 is executed. On the other hand, if the travel mode of the forklift is the normal mode, the control target calculation unit 211 determines whether the pallet is photographed by the camera when the forklift travels along the travel path (step S305). If the pallet is photographed, step S303 is executed.

[0049] If the pallet is not photographed, the movement mode switching unit 29 sets the movement mode of the forklift to the approach mode (step S306), and the control target calculation unit 211 calculates a control target value so that the forklift approaches the pallet while the camera keeps photographing the pallet (step S307).

[0050] The mobile object control device 20 then determines whether the forklift has reached the goal position and goal posture (step S308). If the forklift has not reached the goal position and goal posture, step S301 is executed. If the forklift has reached the goal position and goal posture, the process ends.

[0051] As described above, the control target calculation unit 211 moves the forklift so that it approaches the pallet while the sensor keeps capturing the pallet. As a result, the probability that the forklift will be able to face the pallet directly increases. In addition, because the forklift does not stop, a decrease in transport efficiency can be prevented.

[0052] FIG. 7 is a block diagram showing a configuration example of the mobile object control device 10 and the mobile object control device 20 (hereinafter referred to as the mobile object control device 10, etc.) described in the above-mentioned embodiment. Referring to FIG. 7, the mobile object control device 10, etc. includes a network interface 1001, a processor 1002, and a memory 1003. The network interface 1001 may be used to communicate with a network node. The network interface 1001 may include, for example, a network interface card (NIC) conforming to the IEEE 802.3 series. IEEE stands for Institute of Electrical and Electronics Engineers.

[0053] The processor 1002 reads and executes software (computer programs) from the memory 1003 to perform the processing of the mobile object control device 10 and the like described using flowcharts in the above-described embodiments. The processor 1002 may be, for example, a microprocessor, an MPU, or a CPU. The processor 1002 may include multiple processors.

[0054] The memory 1003 is configured by a combination of volatile memory and non-volatile memory. The memory 1003 may include storage located remotely from the processor 1002. In this case, the processor 1002 may access the memory 1003 via an I / O (Input / Output) interface (not shown).

[0055] 7, the memory 1003 is used to store software modules. The processor 1002 reads and executes these software modules from the memory 1003, thereby performing the processing of the mobile object control device 10 and the like described in the above embodiment.

[0056] As explained using Figure 7, each of the processors possessed by the mobile body control device 10, etc. in the above-mentioned embodiments executes one or more programs including a group of instructions for causing a computer to perform the algorithm explained using the drawings.

[0057] In the above examples, the program includes instructions (or software code) that, when loaded into a computer, cause the computer to perform one or more functions described in the embodiments. The program may be stored on a non-transitory computer-readable medium or a tangible storage medium. By way of example and not limitation, computer-readable medium or tangible storage medium includes random-access memory (RAM), read-only memory (ROM), flash memory, solid-state drive (SSD) or other memory technology, CD-ROM, digital versatile disc (DVD), Blu-ray disc or other optical disk storage, magnetic cassette, magnetic tape, magnetic disk storage or other magnetic storage device. The program may also be transmitted on a transitory computer-readable medium or communication medium. By way of example and not limitation, transitory computer-readable medium or communication medium includes electrical, optical, acoustic, or other forms of propagated signals.

[0058] The technical ideas of the present disclosure are not limited to the above-described embodiments, and can be modified as appropriate within the scope of the gist of the present disclosure.

[0059] Some or all of the above embodiments may be described as, but are not limited to, the following supplementary notes. (Supplementary Note 1) A mobile body control device comprising: information acquisition means for acquiring environmental information from a sensor mounted on a mobile body; object position and orientation estimation means for estimating position and orientation information of an object in accordance with the acquisition of the environmental information; determination means for determining whether the estimation accuracy of the position and orientation information of the object is low; and movement mode switching means for, if the estimation accuracy is low, switching the movement mode of the mobile body to an approach mode in which the mobile body approaches the object while keeping the sensor capturing the object. (Supplementary Note 2) The mobile body control device according to Supplementary Note 1, further comprising: sensing range prediction means for simulating movement of a sensing range of the sensor accompanying movement of the mobile body; and control target calculation means for calculating a control target value based on a result of the simulation of movement of the sensing range when the movement mode is the approach mode. (Supplementary Note 3) The mobile body control device according to Supplementary Note 2, wherein the control target value represents a steering angle. (Supplementary Note 4) The mobile body control device according to Supplementary Note 2 or 3, further comprising: a goal position and attitude setting means for setting a goal position and goal attitude at which the mobile body faces the object based on the position and attitude information of the object; and a path acquisition means for acquiring a movement path for the mobile body to reach the goal position and the goal attitude. (Supplementary Note 5) The mobile body control device according to Supplementary Note 4, wherein, when the movement mode is not the approach mode, the control target calculation means calculates a first control target value according to the movement path. (Supplementary Note 6) The mobile body control device according to Supplementary Note 5, wherein, when the movement mode is the approach mode, the control target calculation means executes a process of determining whether the sensor has captured the object on the assumption that the control target value is a second control target value different from the first control target value, and a process of increasing or decreasing the second control target value by a predetermined amount when it is determined that the sensor has not captured the object. (Supplementary Note 7) The mobile body control device according to any one of Supplementary Notes 1 to 6, wherein the determining means determines whether the estimation accuracy is low based on a variance of the position and orientation information of the object estimated in a certain time interval.(Supplementary Note 8) The mobile body control device according to any one of Supplementary Notes 1 to 6, wherein the mobile body is a forklift that transports the object, which is a pallet. (Supplementary Note 9) A mobile body control method comprising: acquiring environmental information from a sensor mounted on the mobile body; estimating position and orientation information of the object in accordance with the acquired environmental information; determining whether the estimation accuracy of the position and orientation information of the object is low; and, if the estimation accuracy is low, switching the movement mode of the mobile body to an approach mode in which the mobile body approaches the object while keeping the sensor capturing the object. (Supplementary Note 10) The mobile body control method according to Supplementary Note 9, further comprising: simulating movement of a sensing range of the sensor accompanying movement of the mobile body; and, if the movement mode is the approach mode, calculating a control target value based on a result of the simulation of the movement of the sensing range. (Supplementary Note 11) The mobile body control method according to Supplementary Note 10, wherein the control target value represents a steering angle. (Supplementary Note 12) The mobile body control method according to Supplementary Note 10 or 11, wherein a goal position and goal attitude of the mobile body facing the object are set based on the position and attitude information of the object, and a movement path for the mobile body to reach the goal position and goal attitude is obtained. (Supplementary Note 13) The mobile body control method according to Supplementary Note 12, wherein calculating the control target value calculates a first control target value according to the movement path when the movement mode is not the approach mode. (Supplementary Note 14) The mobile body control method according to Supplementary Note 13, wherein when the movement mode is the approach mode, calculating the control target value includes a process of determining whether the sensor is capturing the object on the assumption that the control target value is a second control target value different from the first control target value, and a process of increasing or decreasing the second control target value by a predetermined amount when it is determined that the sensor is not capturing the object. (Supplementary Note 15) The mobile object control method according to any one of Supplementary Notes 9 to 14, wherein the determining determines whether the estimation accuracy is low based on a variance of the position and orientation information of the object estimated in a certain time interval.(Supplementary Note 16) The mobile body control method according to any one of Supplementary Notes 9 to 15, wherein the mobile body is a forklift that transports the object, which is a pallet. (Supplementary Note 17) A non-transitory computer-readable medium storing a program that causes a computer to execute the following processes: acquiring environmental information from a sensor mounted on the mobile body; estimating position and orientation information of the object in accordance with the acquisition of the environmental information; determining whether the estimation accuracy of the position and orientation information of the object is low; and, if the estimation accuracy is low, switching the movement mode of the mobile body to an approach mode in which the mobile body approaches the object while keeping the sensor capturing the object. (Supplementary Note 18) The non-transitory computer-readable medium according to Supplementary Note 17, wherein the program causes the computer to execute the following processes: simulating movement of a sensing range of the sensor accompanying movement of the mobile body; and, if the movement mode is the approach mode, calculating a control target value based on a result of the simulation of the movement of the sensing range. (Supplementary Note 19) The non-transitory computer-readable medium according to Supplementary Note 18, wherein the control target value represents a steering angle. (Supplementary Note 20) The non-transitory computer-readable medium according to Supplementary Note 18 or 19, wherein the program causes the computer to execute a process of setting a goal position and a goal orientation of the moving body facing the object based on the position and orientation information of the object, and a process of acquiring a movement path for the moving body to reach the goal position and the goal orientation. (Supplementary Note 21) The non-transitory computer-readable medium according to Supplementary Note 20, wherein the process of calculating the control target value includes a process of calculating a first control target value according to the movement path when the movement mode is not the approach mode. (Supplementary Note 22) The non-transitory computer-readable medium according to Supplementary Note 21, wherein when the movement mode is the approach mode, the process of calculating the control target value includes a process of determining whether the sensor is capturing the object on the assumption that the control target value is a second control target value different from the first control target value, and a process of increasing or decreasing the second control target value by a predetermined amount when it is determined that the sensor is not capturing the object.(Supplementary Note 23) The non-transitory computer-readable medium according to any one of Supplements 17 to 22, wherein the determining process determines whether the estimation accuracy is low based on a variance of the position and orientation information of the object estimated in a certain time interval. (Supplementary Note 24) The non-transitory computer-readable medium according to any one of Supplements 17 to 23, wherein the moving body is a forklift that transports the object, which is a pallet.

[0060] 10, 20 Mobile object control device 11 Information acquisition unit 12, 23 Object position and attitude estimation unit 13 Determination unit 14, 29 Travel mode switching unit 21 Sensor information acquisition unit 22 Robot position and attitude acquisition unit 24 Object position and attitude storage unit 25 Goal position and attitude setting unit 26 Path acquisition unit 27 Object stability determination unit 28 Sensing range prediction unit 210 Travel mode storage unit 211 Control target calculation unit 212 Machine body control unit 213 Drive unit

Claims

1. A mobile body control device comprising: an information acquisition means for acquiring environmental information from a sensor mounted on a mobile body; an object position and orientation estimation means for estimating position and orientation information of an object in accordance with the acquisition of the environmental information; a determination means for determining whether the estimation accuracy of the position and orientation information of the object is low; and a movement mode switching means for switching the movement mode of the mobile body to an approach mode in which the mobile body approaches the object while the sensor is still capturing the object.

2. A mobile body control device as described in claim 1, further comprising: a sensing range prediction means for simulating the movement of the sensing range of the sensor accompanying the movement of the mobile body; and a control target calculation means for calculating a control target value based on the simulation results of the movement of the sensing range when the movement mode is the approach mode.

3. The mobile object control device according to claim 2, wherein the control target value represents a steering angle.

4. A mobile body control device as described in claim 2 or 3, further comprising: a goal position and attitude setting means for setting a goal position and goal attitude at which the mobile body faces the object based on the position and attitude information of the object; and a path acquisition means for acquiring a travel path for the mobile body to reach the goal position and goal attitude.

5. A mobile object control device according to claim 4, wherein when the movement mode is not the approach mode, the control target calculation means calculates a first control target value in accordance with the movement route.

6. A mobile body control device as described in claim 5, wherein, when the travel mode is the approach mode, the control target calculation means executes a process of determining whether or not the sensor is capturing the object when it is assumed that the control target value is a second control target value different from the first control target value, and a process of increasing or decreasing the second control target value by a predetermined amount when it is determined that the sensor is not capturing the object.

7. A mobile control device according to any one of claims 1 to 6, wherein the determination means determines whether the estimation accuracy is low or not based on the variance of the position and orientation information of the object estimated in a certain time interval.

8. The mobile object control device according to any one of claims 1 to 6, wherein the mobile object is a forklift that transports the object, which is a pallet.

9. A method for controlling a moving body, comprising: acquiring environmental information from a sensor mounted on the moving body; estimating position and orientation information of an object in accordance with the acquired environmental information; determining whether the estimation accuracy of the position and orientation information of the object is low; and, if the estimation accuracy is low, switching the movement mode of the moving body to an approach mode in which the moving body approaches the object while the sensor is still capturing the object.

10. A method for controlling a moving body as described in claim 9, further comprising simulating a movement of the sensing range of the sensor accompanying the movement of the moving body, and, when the movement mode is the approach mode, calculating a control target value based on a result of the simulation of the movement of the sensing range.

11. The method for controlling a moving body according to claim 10, wherein the control target value represents a steering angle.

12. A method for controlling a moving body as described in claim 10 or 11, further comprising the steps of: setting a goal position and goal orientation for the moving body to directly face the object based on the position and orientation information of the object; and obtaining a movement path for the moving body to reach the goal position and goal orientation.

13. The moving object control method according to claim 12, wherein calculating the control target value includes calculating a first control target value according to the moving path when the moving mode is not the approach mode.

14. A method for controlling a moving object as described in claim 13, wherein, when the movement mode is the approach mode, calculating the control target value includes a process of determining whether the sensor is capturing the object on the assumption that the control target value is a second control target value different from the first control target value, and a process of increasing or decreasing the second control target value by a predetermined amount when it is determined that the sensor is not capturing the object.

15. A mobile object control method according to any one of claims 9 to 14, wherein the determining step determines whether the estimation accuracy is low or not based on the variance of the position and orientation information of the object estimated in a certain time interval.

16. The mobile object control method according to any one of claims 9 to 15, wherein the mobile object is a forklift that transports the object, which is a pallet.

17. A non-transitory computer-readable medium storing a program that causes a computer to execute the following processes: acquiring environmental information from a sensor mounted on a moving body; estimating position and orientation information of an object based on the acquired environmental information; determining whether the estimation accuracy of the position and orientation information of the object is low; and, if the estimation accuracy is low, switching the movement mode of the moving body to an approach mode in which the moving body approaches the object while the sensor is still capturing the object.

18. The non-transitory computer-readable medium according to claim 17, wherein the program causes the computer to execute the following processes: a process of simulating movement of the sensing range of the sensor in association with movement of the moving object; and a process of calculating a control target value based on a result of the simulation of the movement of the sensing range when the movement mode is the approach mode.

19. The non-transitory computer-readable medium of claim 18, wherein the control target value represents a steering angle.

20. A non-transitory computer-readable medium as described in claim 18 or 19, wherein the program causes the computer to execute a process of setting a goal position and goal orientation at which the moving body faces the object based on the position and orientation information of the object, and a process of acquiring a movement path for the moving body to reach the goal position and goal orientation.

21. The non-transitory computer-readable medium according to claim 20, wherein the process of calculating the control target value calculates a first control target value according to the movement path when the movement mode is not the approach mode.

22. The non-transitory computer-readable medium of claim 21, wherein when the movement mode is the approach mode, the process of calculating the control target value includes a process of determining whether the sensor is capturing the object when the control target value is assumed to be a second control target value different from the first control target value, and a process of increasing or decreasing the second control target value by a predetermined amount when it is determined that the sensor is not capturing the object.

23. A non-transitory computer-readable medium according to any one of claims 17 to 22, wherein the determining process determines whether the estimation accuracy is low or not based on the variance of the position and orientation information of the object estimated in a certain time interval.

24. The non-transitory computer-readable medium of any one of claims 17 to 23, wherein the moving object is a forklift that transports the object, which is a pallet.

Images