Information processing device, mobile device, and computer program

Abstract

To automatically reset position measurement processing in the event that controlling a moving body through position measurement is suspended.SOLUTION: The position of a moving body is measured. Success / failure information concerning success or failure of position measurement is acquired. History information that is at least one of position history information concerning a history of positions of the moving body, driving history information concerning a driving history of the moving body, illuminance history information concerning a history of illuminance values of the surroundings of the moving body, and imaging direction history information concerning a history of directions of an imaging device mounted in the moving body is acquired. Whether driving control of the moving body is suspended is decided based on the success / failure information. When it is decided that the driving control of the moving body is suspended, a restart method for restarting the driving control of the moving body is determined based on at least one of the pieces of history information.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to an information processing device, a mobile device, a computer program, and the like that are capable of measuring the position of a mobile object. [Background technology]

[0002] The technology of autonomously moving robots and other mobile objects is becoming more widespread through position measurement processing using images taken by a camera. In particular, attention is being paid to the automation of logistics by installing this technology on AGVs (Automatic Guided Vehicles) that operate autonomously in factories and warehouses.

[0003] In the above-described device, if the position measurement process fails or the reliability of the position measurement process is low, autonomous travel becomes impossible and control of the moving body becomes unstable. In order to control the moving body again and make it travel independently, it is necessary to reset the position measurement process. On the other hand, Patent Document 1 discloses a technology in which, in an image processing device that estimates the position and orientation of an object based on an image of the object, if the estimation of the position and orientation of the object fails, a reference image is displayed and the position and orientation estimation process is performed. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Publication No. 2020-3870 Summary of the Invention [Problem to be solved by the invention]

[0005] However, the technology described in Patent Document 1 is a technology for estimating the position and orientation of an object, and even if this technology is applied to an autonomously traveling mobile body, it is not possible to automatically and appropriately resume the position measurement process. SUMMARY OF THE INVENTION It is therefore an object of the present invention to provide an information processing device that can determine an appropriate restart method when position measurement of a moving object fails. [Means for solving the problem]

[0006] An information processing device according to one aspect of the present invention includes: a position measurement means for measuring the position of the moving object using a captured image of the surroundings of the moving object; a success / failure information acquiring means for acquiring success / failure information regarding the success or failure of position measurement by the position measuring means; a history acquisition means for acquiring location history information relating to the location history of the moving object and information on the history of the captured images; a determination means for determining whether or not to suspend drive control of the moving body based on the success / failure information acquired by the success / failure information acquisition means; a determination means for determining, when the determination means determines that drive control of the moving body should be suspended, a restart method for restarting drive control of the moving body by moving the moving body until a similarity between an image in the history of captured images for which the success / failure information is successful and a currently captured image becomes equal to or greater than a threshold; The present invention is characterized by having the following. [Effects of the Invention]

[0007] The present invention can realize an information processing device that can determine an appropriate restart method when position measurement of a moving object fails. [Brief explanation of the drawings]

[0008] [Figure 1] 1 is a functional block diagram of a mobile object including an information processing apparatus according to a first embodiment. [Figure 2] 1 is a flowchart showing information processing according to the first embodiment. [Figure 3] FIG. 10 is a diagram showing an example of a restart method displayed on a return method presentation unit according to the first embodiment. [Figure 4] FIG. 10 is a functional block diagram of a mobile object including an information processing apparatus according to a second embodiment. [Figure 5] FIG. 10 is a diagram illustrating an example of a restart method presentation unit that displays a plurality of restart methods according to the second embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0009] Hereinafter, preferred embodiments of the present invention will be described by way of example with reference to the accompanying drawings. In each drawing, the same members or elements are designated by the same reference numerals, and duplicated descriptions will be omitted or simplified. In the following embodiments, an example of applying an information processing device to an autonomously moving body (mobile device) such as an AGV will be described, but the moving body (mobile device) is not limited to AGVs and can also be applied to drones, various robots, etc. [Example]

[0010] FIG. 1 is a functional block diagram of a mobile object including an information processing apparatus according to a first embodiment. 1, reference numeral 1000 denotes a moving body as a moving device, 100 denotes a sensor unit including an imaging element such as a CMOS sensor for acquiring images of the periphery of the moving body, 110 denotes a position measurement unit, 120 denotes a moving body control unit, 130 denotes an information processing device, and 180 denotes a restart method presentation unit. The imaging element of sensor unit 100 is installed at a predetermined position, for example, in front of the moving body, and is configured to acquire an image within a predetermined angle of view range as seen from the front of the moving body. The information processing device 130 includes a position measurement status acquisition unit 140, a first input unit 150, a determination unit 160, a restart method determination unit 170, etc.

[0011] The mobile object 1000 has a drive unit such as a motor or engine (not shown) that serves as a drive source for moving the mobile object, and the drive unit is controlled by a mobile object control unit 120 that serves as a mobile object control means. The restart method presentation unit 180 is, for example, a display of a user-operated terminal such as a smartphone or PC that is separate from the mobile object 1000 and connected via a wireless network. The restart method presentation unit 180 serving as a user operation terminal may be provided in the mobile object 1000 .

[0012] The mobile object 1000 has a built-in CPU as a computer, which functions as a control means for controlling the operation of each part of the entire device based on a computer program stored in memory. The functional blocks shown in Fig. 1 represent functions realized by the computer. The position measurement unit 110 (position measurement means) periodically measures the position of the moving body using images of the surroundings of the moving body captured by the sensor unit 100, and stores and outputs information such as the measured position of the moving body as history information.

[0013] That is, the position measurement unit 110 has an internal memory and stores and outputs various measurement results when periodically performing position measurement processing as history information. The history information includes a history of position information at each point when the position measurement processing was performed, i.e., position history information related to the history of the position of the mobile object, and history information of the date and time when the position measurement was performed at each point. It also includes history information of success / failure information related to the success / failure of the position measurement. Here, the success / failure information includes reliability information related to the reliability of the position measurement processing. The success / failure information is, for example, information on whether the reliability of the position measurement processing result is equal to or greater than a predetermined threshold.

[0014] The position measurement status acquisition unit 140 functions as a success / failure information acquisition means, and acquires at least success / failure information regarding the success / failure of position measurement (reliability information regarding the reliability of the position measurement process, etc.) from the position measurement unit 110. The success / failure information acquired here is input to the determination unit 160, which determines whether or not to interrupt drive control of the moving body (movement of the moving body, rotation control of the moving body's direction, etc.) based on the success / failure information. If the judgment unit 160 determines that the drive control of the moving body will not be interrupted, i.e., if the control of the moving body will be continued, the moving body control unit 120 controls the drive of the moving body, such as autonomous driving, using the position measurement results of the moving body obtained by the position measurement unit 110.

[0015] On the other hand, if the judgment unit 160 determines that the drive control of the moving body should be suspended, the moving body control unit 120 suspends the drive control of the moving body. The restart method determination unit 170 functions as a determination means that determines a restart method for the drive control of the moving body based on at least one of the various pieces of history information described above acquired by the first input unit 150 from the position measurement unit 110. Here, the restart method includes at least one of a return position, which is a point for resuming the drive control of the moving body, and a drive method (movement path, movement speed, rotation, etc.) for the moving body to return to the return position. Here, the first input unit 150 functions as a history acquisition unit that acquires the various types of history information described above from the position measurement unit 110.

[0016] When the return position, which is the restart method, and the method for driving the moving body to return to the return position are determined, the moving body control unit 120 restarts control of the moving body in accordance with the determined restart method. Furthermore, the restart method presentation unit 180, as a restart method presentation means, presents the restart method determined by the restart method determination unit 170 to the user who is using or operating the moving body. The restart method presentation unit 180 may also present the success or failure of the position measurement acquired by the position measurement status acquisition unit 140.

[0017] FIG. 2 is a flowchart showing information processing according to the first embodiment. The computer in the mobile unit 1000 executes a computer program stored in memory to perform the operations of the steps in the flowchart of FIG. In step S200, initial values ​​to be used for the return position and movement method, threshold values ​​required for determining whether or not control of the moving body needs to be interrupted, etc. are set.

[0018] In step S201 (position measurement step), a position measurement process is performed. In this embodiment, the image captured by the sensor unit 100 is subjected to SLAM (Simultaneous Localization and Mapping) technology to simultaneously measure the position of the moving body and create an environmental map showing the placement of indices for position measurement. In step S202, the moving body control unit 120 controls the moving body so that it moves independently from the position thus determined.

[0019] Next, in step S203 (success / failure information acquisition step), the position measurement status acquisition unit 140 acquires the position measurement status. The position measurement status acquired here is the success / failure (reliability information, etc.) of the position measurement process performed in step S201. Based on the position measurement state acquired in step S203, the determination unit 160 determines whether or not it is necessary to suspend control of the moving object in step S204 (determination step), and determines whether or not to suspend control in step S205. If the determination is No, control of the moving object can be continued, and the process returns to step S201 to repeat the control. If it is determined that control of the moving object should be suspended (if the determination is Yes), the process proceeds to step S206.

[0020] In step S206, drive control of the moving body is interrupted. Then, in step S207 (history acquisition step), the first input unit 150 acquires information related to position measurement and various history information related to control of the moving body. In this embodiment, this history information is acquired as information related to position measurement to be used to determine the return position. That is, for example, position information and time of each point where position measurement processing has been successful up to that point are acquired as control information for the moving body to be used to determine the movement method, and history of the moving speed and moving direction of the moving body at each point where the position measurement unit 110 has measured the position is acquired.

[0021] Next, in step S208 (decision step), a restart method is determined based on the historical information of the positions and times of each point where position measurement has been successful, acquired in step S207, and historical information such as the moving speed and moving direction of the moving object. The restart method includes a return position, which is the point where control of the moving body is restarted, and a drive method such as movement to the return position. In this embodiment, the return position is the point immediately before the point where control of the moving body was interrupted where it was determined that control of the moving body can be continued (position measurement successful). Note that the return position may be any point before the point where control was interrupted.

[0022] The driving method for moving the moving object to the return position, etc., is determined by calculating the amount of movement and the movement route. In this embodiment, the time-series information on the moving speed and moving direction of the moving object from the return position to the current position acquired by the first input unit 150 is rearranged in reverse chronological order, and the amount of movement and the movement route are determined to control the moving object based on the rearranged information.

[0023] In step S209, the restart method determination unit 170 presents the restart method determined in step S208 to the user via, for example, a wireless network. FIG. 3 is a diagram illustrating an example of a restart method displayed on the restart method presentation unit according to the first embodiment. As a presentation method, a method of displaying on the UI of the user operation terminal 300 as the restart method presentation unit 180 as shown in Fig. 3 will be described. The user operation terminal 300 has a restart method display unit 310, a character display unit 320, up / down / left / right buttons 330 used to move up / down / left / right and to select an option, and an OK button 340 for accepting the displayed content. A joystick, a trackball, or the like may be used instead of the up / down / left / right buttons 330.

[0024] Furthermore, the user operation terminal 300 has a STOP button 350 for instructing to suspend control of the moving object, and a RESTART button 360 for instructing to resume the position measurement process. The text display section 320 can display in text a message that the position measurement process has failed, options for moving to a return position, etc.

[0025] The restart method display unit 310 displays a message such as "Position measurement failed. Would you like to return using the restart method shown below? Yes No" on the text display unit 320 so that the return position, the movement method and route to the return position, etc. can be visually recognized. If the user wants to correct or adjust the restart method, etc. determined by the restart method determination unit 170 in step S208, the user uses the up / down / left / right buttons 330 to select No and then presses the OK button 340 to select a change of restart method. Thereafter, the user uses the up / down / left / right buttons to move the cursor within the screen to adjust the restart method, such as the return position, and presses the OK button 340. The restart method determination unit 170 stores the restart method changed by the user as the updated restart method.

[0026] In this embodiment, the restart method display unit 310 has been described as being included in a user terminal, such as a smartphone, that is separate from the mobile object. However, the restart method display unit 310 may be mounted on the mobile object, or as described above, may be the screen of a smartphone that is an external terminal separate from the mobile object, or a PC monitor connected to a PC, and is not limited to the configuration of this embodiment. Furthermore, the buttons and display units that are mounted on the unit are not limited to those shown in FIG. 3.

[0027] After the restart method determined in step S208 is presented to the user in step S209, if the user does not change the restart method, the mobile object control unit 120 resumes control of the mobile object in step S210 using the restart method determined in step S208. On the other hand, if the user changes the restart method, the restart method determination unit 170 resumes drive control of the mobile object in the mobile object control unit 120 in step S210 based on the changed restart method.

[0028] Here, the moving body control unit 120 functions as a control unit that controls the driving of the moving body, and the control unit executes the restart method determined by the restart method determination unit 170 in step S208 or the restart method specified by the user. The moving object control unit 120 drives the moving object toward the return position (including the rotation position) based on the return position and the drive method for return finally determined by the restart method determination unit 170, and the position measurement unit 110 resumes the position measurement process. At this time, the position measurement may be performed by referring to the environmental map obtained by SLAM.

[0029] The above is the processing flow according to this embodiment. Note that, although this embodiment has been described as suspending control of the moving object and determining a method of resuming control when position measurement fails, it is also possible to suspend control of the moving object and determine a method of resuming control when the reliability of the position measurement process falls below a certain reference threshold value even when position measurement does not fail.

[0030] The history information regarding the control of the moving object acquired in step S207 may be odometry information, such as the amount of tire rotation required to move the moving object, and the amount of movement may be calculated from the odometry information. Alternatively, an IMU (Inertial Measurement Unit) may be installed in the moving object, and the amount of movement and the movement route may be calculated from the IMU information and the movement time. Alternatively, a GPS may be installed in the moving object, and the amount of movement and the movement route may be calculated from the GPS movement history. Alternatively, the moving object may be photographed by a separate monitoring camera, and the amount of movement and the movement route may be calculated from the monitoring camera image.

[0031] Furthermore, the movement method may be determined by tracing back and executing commands in the reverse direction based on the control command information of the moving body from the return position to the point where it is determined that control of the moving body needs to be suspended. Furthermore, the method of presenting the resumption method in step S209 may be a method using a UI, or may be an audio guide that reads out the movement method. When displaying using a UI, the return position may be superimposed on the environmental map obtained by SLAM.

[0032] The presentation unit may also be provided with a decision input unit for the user to decide whether or not to actually execute the restart method, or the presented restart method may be automatically executed if there is no input from the user for a predetermined time or more after the presentation of the restart method in step S209. Alternatively, multiple restart methods may be presented, rather than just one, so that the user can specify which restart method to use.

[0033] Also, a return warning unit may be provided that warns that the robot is in the process of returning while moving to the return position. The return warning unit may use a variety of warning methods, such as displaying a message on the same UI as the restart method presentation unit 180, informing the robot that the robot is in the process of returning by voice, or playing a predetermined warning melody during the process of returning.

[0034] As described above, according to the information processing apparatus of this embodiment, even if control of the moving body is interrupted due to a position measurement failure or the like, it is possible to quickly restore position and orientation measurement and resume control of the moving body. Furthermore, the determined restart method can be displayed on a GUI (Graphical User Interface) and presented to the user, allowing the user to select the most suitable restart method at their discretion. [Example]

[0035] In the second embodiment, which is a modification of the first embodiment, a restart method is determined using an image captured for measuring the position of a moving object. The overall configuration diagram and processing flow are the same as those in the first embodiment, so the explanation will be omitted, and the differences from the first embodiment will be mainly briefly explained.

[0036] In this embodiment, the history information acquired from the position measurement unit 110 includes a history of images of the surroundings taken at each point when position measurement was performed. That is, the first input unit 150 acquires the history of captured images and position information at each point where the position measurement process was performed as information related to the position measurement. Then, the resumption method determination unit 170 determines a movement method to move to a point where an image similar to the captured image at the return position can be captured. As in the first embodiment, the return position is a point immediately before the point where control of the moving object was interrupted where it was determined that control of the moving object can be continued. Note that the return position may be any point before the point where control was interrupted.

[0037] The method for determining the movement method will be described below. The point where it is determined that control of the moving object needs to be suspended is set as the starting point, and the images captured by the first input unit 150 at each point from the starting point to the return position are set as a reference image group. The moving body rotates and moves back and forth and left and right little by little from the start point, and moves until a matching process is performed between the image in the reference image group immediately before the start point and the current captured image, and a match with a predetermined reliability or higher is achieved. In other words, the restart method determination unit determines the restart method for the moving body based on the matching result between the history of captured images and the current captured image.

[0038] Thereafter, based on the position information of the location where the reference image was captured, acquired by the first input unit 150, the moving body moves to the position where the next reference image was captured, and a matching process is performed between the reference image and the currently captured image. In this way, the moving body repeats the movement and matching process, moves to the return position, and when matching with the reference image at the return position is achieved, the movement is terminated. The matching criterion is that a matching is achieved if the similarity of the matching process result is a certain value or more, such as 90% or more.

[0039] In this embodiment, the matching process and movement were repeated using multiple reference images from the start point to the return position, but the matching process and movement may also be repeated using a single reference image at the return position. In this case, rotation and movement back and forth and left and right from the start point are repeated to perform the matching process between the captured image of the current point and the return position. When matching feature points are found, the moving object is moved in a direction that reduces the discrepancy between the features in the matching result, and the matching process is performed again. In this way, the moving object's movement and matching process are repeated, and when matching that meets or exceeds a predetermined standard is achieved, it is determined that the movement to the return position is complete.

[0040] In this embodiment, the matching process is performed while moving little by little from the start point, but the start point for starting the matching process can also be any point designated by the user. The method for calculating the similarity of the matching processing result is not limited, and may be SSD (Sum of Squared Difference), NCC (Normalized Cross Correlation), etc. Furthermore, the matching criterion may be determined by setting a different criterion value as the similarity for each feature point. [Example]

[0041] In the third embodiment, as a modification of the first embodiment, a method of determining a return position from a success rate (reliability) of position measurement calculated in advance will be described. The overall configuration diagram and processing flow are the same as those in the first embodiment, so a description thereof will be omitted, and only the determination of the return position, which is different from that in the first embodiment, will be briefly described. In this embodiment, the history information acquired from the position measurement unit 110 includes history such as success / failure information regarding the success / failure of position measurement at each point when position measurement was performed (reliability information regarding the reliability of the position measurement process) and the number of feature points.

[0042] In the position measurement process, the greater the number of features in the image registered in the environmental map, the higher the success rate (reliability). In this embodiment, the number of feature points in the image when creating the environmental map is used as the success rate (reliability) of the position measurement process, and is used by the restart method determination unit 170 when determining the return position. The information related to the position measurement acquired by the first input unit 150 is used as the success rate of the position measurement process, and the restart method determination unit 170 determines a point where the success rate is equal to or higher than a certain level as the return position. In addition, the method of moving to the return position is determined from the information related to the control of the moving object.

[0043] While this embodiment has been described using SLAM processing based on feature points in an image, this embodiment is not limited to SLAM based on feature points, nor is it limited to SLAM using captured images. Furthermore, while the number of feature points in an image is used as the degree of success, the relationship between the number of feature points in an image and the reliability of the position measurement processing may be calculated from past position measurement processing results, and this may also be used as the degree of success.

[0044] Furthermore, although the degree of success of the position measurement process is calculated from the number of feature points in the image, it may also be calculated by creating a database of success degrees from past position measurement results. Alternatively, the degree of success may be calculated by taking into consideration whether the spatial distribution of features in the image is uniform, or whether the distribution of features in the image is random rather than periodic. Furthermore, the degree of success may be calculated by combining conditions that affect the results of the position measurement process, such as environmental conditions in which a moving object moves, such as illuminance and unevenness of the ground. [Example]

[0045] In the fourth embodiment, a method for determining a restart method that avoids the cause of the failure of the position measurement process will be described. 4 is a functional block diagram of a mobile body including an information processing device according to the second embodiment. Explanation of the same parts as in the first embodiment will be omitted, and only a second input unit 450 and a restart method determination unit 470, which are different from those in the first embodiment, will be briefly explained. 2000 is a mobile body, and 131 is an information processing device. Furthermore, when the position measurement unit 110 fails to measure the position of a moving object, the position measurement unit 110 also outputs information that may be the cause of the failure. Furthermore, in this embodiment, the restart method determination unit 470 determines at least one of a return position and a movement method that avoids the cause of the failure.

[0046] That is, the second input unit 450 acquires history information that may be a cause of a position measurement process failure, in addition to information about the position measurement process performed by the position measurement unit 110 and information about the control of the mobile body. In this embodiment, the information about the cause of failure includes at least one of history information about the illuminance around the mobile body, history information about the orientation of the image sensor of the sensor unit 100 mounted on the mobile body, and history information about the number of feature points in the image. That is, the history information from the position measurement unit 110 includes history information about the position at each point where the position measurement process was performed, time information, history information about the success or failure of the position measurement process, and information about the cause of the position measurement failure.

[0047] As described above, in this embodiment, the history information acquired from the position measurement unit 110 includes a history of information that may be a cause of failure regarding the success or failure of position measurement at each point when position measurement is performed. Illuminance history information relating to the illuminance around the moving object, imaging direction history information relating to the history of the direction of the imaging element in the sensor unit 100, and drive history information relating to the history of driving the moving object (movement direction, amount of movement, movement speed, etc.) are obtained from the position measurement unit 110. Note that in this embodiment, no pan / tilt mechanism for changing the direction of the imaging element is provided, and the direction of the imaging element can be changed by changing the direction of the moving object, but a pan / tilt mechanism may be provided.

[0048] The restart method determined by the restart method determination unit 470 will be explained for each cause of the position measurement failure. For example, if it is determined that the cause of the position measurement failure is a sudden change in illuminance or abnormal illuminance, the restart method determination unit 470 determines a return position and movement method so that the illuminance around the moving object can be obtained stably and appropriately. That is, based on the illuminance information around the moving object acquired by the second input unit 450, it determines the direction of the image sensor of the sensor unit where the position measurement process is likely to be successful and the direction of the image sensor of the sensor unit where the position measurement process is likely to fail. The illuminance related to the success or failure of the position measurement process is calculated or measured in advance.

[0049] Next, the orientation of the moving body is compared with that of the imaging element of the sensor unit 100, and if the imaging element is facing a direction of low illuminance or high illuminance where imaging by the imaging element will fail, the orientation of the moving body itself is changed (rotated) so that the imaging element faces a direction of appropriate illuminance where position measurement processing will be successful. Therefore, the restart method determination unit 470 sets the return position as the current position and determines that the movement method is to rotate the moving body so that the imaging element faces a direction of appropriate illuminance where position measurement processing will be successful.

[0050] If the illuminance around the moving object is so low that the position measurement process fails, the return position is set to a point before that point where the illuminance is such that the position measurement process is successful. The movement method is the same as that described in the first embodiment. Furthermore, if it is determined that the cause of the position measurement failure is a lack of feature points in the image, the restart method determination unit 470 determines a movement method to change the direction to a direction in which feature points in the image can be sufficiently detected. The return position is set to the current location. Alternatively, as in the third embodiment, the degree of success may be calculated in advance from the number of feature points in the image, and a point with a certain level of success or higher may be set as the return position. The movement method is determined as described in the third embodiment.

[0051] The above determination of the return position and movement method is one example, and any return position and movement method may be used as long as they can avoid the content of the warning reason. Alternatively, the restart method may be determined from information related to the control of the moving object. For example, if the position measurement process fails after changing the traveling direction, the restart method determination unit 470 determines the restart method by setting the return position to "current position" and the movement method to "return to the previous traveling direction."

[0052] As described above, when multiple restart methods are possible, the restart method presenting unit 180 presents multiple restart method candidates as shown in FIG. Fig. 5 is a diagram showing an example of a restart method presentation unit that displays multiple restart methods in Example 2. As shown in Fig. 5, a message such as "Position measurement failed. Which restart method would you like?" is displayed, allowing the user to select from the candidates. In Fig. 5, two restart method candidates, Method 1 and Method 2, are displayed. The user operates the up, down, left, and right buttons 330 to select the restart method they want to implement from the candidates, and then operates the OK button 340 to confirm the selection.

[0053] As described above, the position measurement unit 110 in the embodiment periodically measures the position of a moving object using captured images of the surroundings of the moving object captured by the sensor unit 100, and stores information such as the measured position of the moving object as history information. The position measurement unit 110 also has an internal memory, and stores and outputs the measurement results obtained when periodically performing position measurement processing as history information. The history information includes, as in Example 1, a history of position information at each point, i.e., position history information relating to the history of the position of the moving body, a history of date and time information when the position was measured at each point, etc. Furthermore, as in Example 2, the history information may include a history of photographed images of the surroundings at each point.

[0054] The history information may also include a history of success / failure information regarding the success / failure of position measurement at each point (reliability information regarding the reliability of the position measurement process) as in Example 3. Furthermore, as in Example 4, the history information may also include illuminance history information regarding the history of illuminance around the moving body, which affects the success / failure of position measurement, and imaging direction history information regarding the history of the direction of the imaging element in the sensor unit 100 mounted on the moving body. Furthermore, the history information may also include drive history information regarding the history of driving of the moving body (movement direction, movement amount, movement speed, etc.), history information regarding the number of feature points, etc.

[0055] The present invention has been described in detail above based on its preferred embodiments, but the present invention is not limited to the above embodiments, and various modifications are possible based on the gist of the present invention, and these modifications are not excluded from the scope of the present invention. Note that a computer program that realizes part or all of the control in this embodiment and the functions of the above-described embodiment may be supplied to an information processing device or the like via a network or various storage media. Then, a computer (or a CPU, MPU, or the like) in the information processing device or the like may read and execute the program. In this case, the program and the storage medium storing the program constitute the present invention. [Explanation of symbols]

[0056] 1000 Mobile 100 Sensor unit 110 Position measurement unit 120 Mobile control unit 130 Information processing equipment 140 Position measurement status acquisition unit 150 first input section 160 Judgment Department 170 Resumption method determination section 180 Restart method presentation section

Claims

1. a position measurement means for measuring the position of the moving object using a captured image of the surroundings of the moving object; a success / failure information acquiring means for acquiring success / failure information regarding the success or failure of position measurement by the position measuring means; a history acquisition means for acquiring location history information relating to the location history of the moving object and information on the history of the captured images; a determination means for determining whether or not to suspend drive control of the moving body based on the success / failure information acquired by the success / failure information acquisition means; a determination means for determining, when the determination means determines that drive control of the moving body should be suspended, a restart method for restarting drive control of the moving body by moving the moving body until a similarity between an image in the history of captured images for which the success / failure information is successful and a currently captured image becomes equal to or greater than a threshold; An information processing device comprising:

2. The information processing device according to claim 1 ; A sensor for capturing surrounding images; and a moving body control means for controlling the movement of the moving body.

3. 10. A computer program for controlling the information processing device according to claim 1 or the means of the mobile device according to claim 2 by a computer.

4. a position measurement step of measuring the position of the moving object using a captured image of the periphery of the moving object; a success / failure information acquisition step of acquiring success / failure information regarding success or failure of the position measurement by the position measurement step; a history acquisition step of acquiring location history information relating to the location history of the moving object and information on the history of the captured images; a determination step of determining whether or not to suspend drive control of the moving body based on the success / failure information acquired in the success / failure information acquisition step; a determining step, when it is determined in the determining step that drive control of the moving body is to be interrupted, of determining a restart method for restarting drive control of the moving body such that the moving body is moved until a similarity between an image in the history of the captured images, the success / failure information of which is successful, and a currently captured image becomes equal to or greater than a threshold value; An information processing method comprising:

Images