Information processing device and information processing method

Abstract

To enable a user to confirm without performing automatic traveling, that there is a higher probability not to enable performing the automatic traveling in a place where teaching traveling has been performed.SOLUTION: An information processing device includes: an extraction section for receiving image information being the information obtained by photographing the periphery of a mobile body when teaching traveling is performed, so as to extract information to be used for estimating an own position obtained from the received image information; and an output control section for outputting to an output section, instruction information different from information in a case where the amount of the information to be used in own position estimation is the second amount greater than the first amount when the amount of the information to be used in the own position estimation extracted by the extraction section is the first amount. A start position of the teaching traveling is the start position of automatic traveling, and an end position of the teaching traveling is a parking target position. When the amount of the information to be used in the own position estimation is the second amount, the output control section notifies a user of the normal end of the teaching traveling, and the instruction information is output before performing the automatic traveling based on traveling route data.SELECTED DRAWING: Figure 2

Description

Technical Field

[0001] The present disclosure relates to an information processing apparatus and an information processing method.

Background Art

[0002] Conventionally, a technique for creating map data based on the characteristics of objects existing around a vehicle during travel has been disclosed. Then, by comparing the created map data with an image acquired by a photographing device mounted on the vehicle, self-position estimation, which is a process of estimating where the vehicle is located in the map data, is executed, and a technique for performing automatic driving along the traveled route has been disclosed.

Prior Art Document

Patent Document

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] By the way, generally, in such a technique, the user manually drives the vehicle (hereinafter referred to as teacher driving) in advance at the place where automatic driving is to be performed, and creates map data and driving route data indicating the driving route used for self-position estimation necessary for performing automatic driving at that place. After that, automatic driving is realized by using the created map data and driving route data.

[0005] During automatic driving, the map data used for self-position estimation is data that stores the three-dimensional positions of feature points of objects existing in the actual scene. During automatic driving, feature points are extracted from an image captured by the vehicle, and it is collated which of the feature points in the map data the extracted feature points are. At this time, if the number of feature points that match the map data is equal to or more than a predetermined number, it is determined that the self-position of the vehicle has been estimated.

[0006] However, when performing teaching driving to make the vehicle travel in order to create map data of the surroundings in a place where the characteristics of the environment around the vehicle are insufficient, such as a place surrounded by objects with no unevenness or color changes, the number of feature points included in the created map data is insufficient compared to a predetermined number required for self-position estimation. Therefore, there is a high possibility that self-position estimation cannot be performed. However, in the prior art, until actually performing autonomous driving, the user could not confirm that there was a high possibility that autonomous driving could not be performed at the place where teaching driving was performed.

[0007] The problem to be solved by the present disclosure is to provide an information processing apparatus and an information processing method that enable a user to confirm, without performing autonomous driving, that there is a high possibility that autonomous driving cannot be performed at the place where teaching driving was performed.

Means for Solving the Problem

[0008] The information processing apparatus of the present disclosure is an information processing apparatus that performs autonomous driving based on driving route data indicating a driving route when a user manually performs teaching driving of a vehicle. During the teaching driving, an extraction unit that receives image information, which is information obtained by photographing the periphery of the moving body, and extracts information used for self-position estimation obtained from the received image information, and an output control unit that outputs, to an output unit, instruction information different from the case where the amount of information used for self-position estimation extracted by the extraction unit is a first amount when the amount of information used for self-position estimation is a second amount greater than the first amount. The start position of the teaching driving is the start position of the autonomous driving, the end position of the teaching driving is a parking target position, and when the amount of information used for self-position estimation is the second amount, the output control unit notifies the user of the normal end of the teaching driving, and the instruction information is output before performing the autonomous driving based on the driving route data.

Effect of the Invention

[0009] According to the information processing apparatus and the information processing method according to the present disclosure, a user can confirm, without performing autonomous driving, that there is a high possibility that autonomous driving cannot be performed at the place where teaching driving was performed.

Brief Description of the Drawings

[0010]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Modes for Carrying Out the Invention

[0011] <Embodiment 1> With reference to the accompanying drawings, embodiments of an information processing apparatus, an information processing method, and a moving body according to the present disclosure will be described.

[0012] FIG. 1 is a diagram showing an example of a moving body 10 according to the present embodiment.

[0013] The moving body 10 includes an information processing apparatus 20, an output unit 10A, an input unit 10B, an internal sensor 10C, a photographing device 10D, a drive control unit 10F, and a drive unit 10G.

[0014] The information processing apparatus 20 is, for example, a dedicated or general-purpose computer. In the present embodiment, an example in which the information processing apparatus 20 is mounted on the moving body 10 will be described.

[0015] The moving body 10 is a movable object. In the present embodiment, the moving body 10 is an object that a user can ride on. The moving body 10 is, for example, a vehicle. The vehicle is a two-wheeled motor vehicle, a three-wheeled motor vehicle, or a four-wheeled motor vehicle, etc. Also, the moving body 10 is, for example, a moving body that advances through a driving operation by a person, or a moving body that can automatically advance (autonomous driving) without a driving operation by a person. In the present embodiment, an example in which the moving body 10 is a vehicle will be described.

[0016] The output unit 10A outputs information. In the present embodiment, the output unit 10A outputs information such as instruction information generated by the information processing apparatus 20. Details of the instruction information will be described later.

[0017] The output unit 10A has a display function for displaying information. Note that the output unit 10A may further have a communication function for transmitting information to an external device or the like, a sound output function for outputting sound, a function for lighting or flashing light, and the like. For example, the output unit 10A includes at least one of a display unit 10K, a communication unit 10H, a speaker 10I, and a lighting unit 10J. In the present embodiment, an example in which the output unit 10A includes the communication unit 10H, the speaker 10I, the lighting unit 10J, and the display unit 10K will be described.

[0018] The communication unit 10H transmits information to other devices. For example, the communication unit 10H transmits information to other devices via a known communication line. The speaker 10I outputs sound. The lighting unit 10J is a light that lights or blinks. The display unit 10K displays information. The display unit 10K is, for example, a known organic EL (Electro Luminescence) display, a liquid crystal display (Liquid Crystal Display), or a projection device.

[0019] The installation position of the output unit 10A may be any position where a user riding in the moving body 10 can confirm the information output from the output unit 10A.

[0020] For example, the moving body 10 includes a plurality of display units (hereinafter, collectively referred to as "display unit 10K"). These plurality of display units 10K are arranged at different positions in the passenger compartment of the moving body 10. Note that the number of display units 10K provided in the moving body 10 may be one or more, and is not limited to a plurality. In the present embodiment, the moving body 10 having one display unit 10K will be described as an example.

[0021] Specifically, the display surface of the display unit 10K is pre-adjusted so that the user U riding in the moving body 10 can view the display surface.

[0022] Returning to FIG. 1 and continuing the description. The input unit 10B receives an instruction or input of information from the user U. The input unit 10B is, for example, at least one of an instruction input device that receives input by the operation input of the user U and a microphone that receives input of sound. The instruction input device is, for example, a button, a pointing device such as a mouse or a trackball, or a keyboard. The instruction input device may be an input function in a touch panel provided integrally with the display unit 10K.

[0023] The internal sensor 10C is a sensor that observes information of the moving body 10 itself. The internal sensor 10C detects the position of the moving body 10, the speed of the moving body 10, or the acceleration of the moving body 10.

[0024] The internal sensor 10C is, for example, an inertial measurement unit (IMU), a speed sensor, or a GPS (Global Positioning System), etc.

[0025] The imaging device 10D is a sensor that observes the surroundings of the moving body 10. The imaging device 10D may be mounted on the moving body 10 or may be mounted outside the moving body 10. The outside of the moving body 10 indicates, for example, another moving body or an external device, etc.

[0026] The surroundings of the moving body 10 are an area within a predetermined range from the moving body 10. This range is the observable range of the imaging device 10D. This range may be set in advance.

[0027] The imaging device 10D observes the surroundings of the moving body 10 and acquires surrounding information. The surrounding information includes at least one of an image of the surroundings of the moving body 10 and information indicating the distance and direction between an object in the surroundings of the moving body 10 and the moving body 10.

[0028] The imaging device 10D obtains captured image data (hereinafter referred to as a captured image) by imaging. This imaging device is a digital camera, a stereo camera, etc. The captured image is digital image data that defines a pixel value for each pixel.

[0029] In the present embodiment, a case where the surrounding information acquired by the imaging device 10D is a captured image of the surroundings of the moving body 10 will be described as an example. Hereinafter, the captured image of the surroundings of the moving body 10 will be described as a surrounding image.

[0030] The installation position and the imaging angle of the imaging device 10D are adjusted in advance so that the surroundings of the moving body 10 can be imaged. In the present embodiment, the moving body 10 includes a plurality of imaging devices 10D with different imaging directions.

[0031] FIG. 2 is a schematic diagram showing an example of the installation position of the imaging device 10D. For example, the moving body 10 includes four imaging devices 10D. Note that the number of imaging devices 10D provided in the moving body 10 is not limited to four.

[0032] Note that the installation position and the number of the imaging devices 10D may be adjusted as long as the imaging devices 10D can acquire imaging images in the directions of substantially all regions (e.g., 360°) centered on the moving body 10 in the horizontal plane, and are not limited to the installation position and the number shown in FIG. 2.

[0033] Returning to FIG. 1, the description will be continued. The drive unit 10G is a drive device mounted on the moving body 10. The drive unit 10G is, for example, an engine, a motor, wheels, or the like.

[0034] The drive control unit 10F controls the drive unit 10G in order to automatically drive the moving body 10. The drive control unit 10F controls the drive unit 10G based on information obtained from the internal sensor 10C or the imaging device 10D, or information received from the information processing device 20. By the control of the drive control unit 10F, the accelerator amount, the brake amount, the steering angle, etc. of the moving body 10 are controlled. For example, the drive control unit 10F controls the moving body 10 so as to enter the moving body 10 into the path indicated by the information received from the information processing device 20 and stop or travel.

[0035] Next, the hardware configuration of the information processing device 20 will be described. FIG. 3 is an example of a hardware configuration diagram of the information processing device 20.

[0036] The information processing device 20 includes a CPU (Central Processing Unit) 11A, a ROM (Read Only Memory) 11B, a RAM (Random Access Memory) 11C, and an I / F 11D, etc., which are interconnected by a bus 11E, and has a hardware configuration using a normal computer.

[0037] The CPU 11A is an arithmetic unit that controls the information processing apparatus 20 of the present embodiment. The ROM 11B stores programs and the like for realizing the processing by the CPU 11A. The RAM 11C stores data necessary for the processing by the CPU 11A. The I / F 11D is an interface for transmitting and receiving data.

[0038] The program for executing the information processing executed by the information processing apparatus 20 of the present embodiment is provided by being pre - incorporated in the ROM 11B or the like. Note that the program executed by the information processing apparatus 20 of the present embodiment may be configured to be stored in a computer - readable storage medium (for example, a flash memory) in a file in a form installable or executable on the information processing apparatus 20 and provided.

[0039] Next, the functional configuration of the mobile body 10 will be described. FIG. 4 is a block diagram showing an example of the functional configuration of the mobile body 10.

[0040] The mobile body 10 includes an information processing apparatus 20, an output unit 10A, an input unit 10B, an internal sensor 10C, a photographing device 10D, a drive control unit 10F, and a drive unit 10G.

[0041] The information processing apparatus 20, the output unit 10A, the input unit 10B, the internal sensor 10C, the photographing device 10D, and the drive control unit 10F are connected via the bus 10L so as to be able to exchange data or signals. The drive control unit 10F is connected to the drive unit 10G so as to be able to exchange data or signals.

[0042] The information processing apparatus 20 has a storage unit 20B and a processing unit 20A. The processing unit 20A and the storage unit 20B are connected via the bus 10L so as to be able to exchange data or signals. Also, the output unit 10A, the input unit 10B, the internal sensor 10C, the photographing device 10D, and the drive control unit 10F and the processing unit 20A are connected via the bus 10L so as to be able to exchange data or signals.

[0043] Note that at least one of the storage unit 20B, the output unit 10A (communication unit 10H, speaker 10I, lighting unit 10J, display unit 10K), the input unit 10B, the internal sensor 10C, the imaging device 10D, and the drive control unit 10F may be connected to the processing unit 20A either wired or wirelessly. Further, at least one of the storage unit 20B, the output unit 10A (communication unit 10H, speaker 10I, lighting unit 10J, display unit 10K), the input unit 10B, the internal sensor 10C, the imaging device 10D, and the drive control unit 10F and the processing unit 20A may be connected via a network.

[0044] The storage unit 20B stores data. The storage unit 20B is, for example, a semiconductor memory element such as a RAM (Random Access Memory), a flash memory, a hard disk, an optical disk, or the like. Note that the storage unit 20B may be a storage device provided outside the information processing apparatus 20. Further, the storage unit 20B may store or temporarily store a program or information downloaded via a LAN (Local Area Network) or the Internet. Further, the storage unit 20B may be configured from a plurality of storage media.

[0045] The moving body travels in any one of a plurality of traveling modes according to a user's instruction. The traveling modes include a teaching traveling mode in which the user performs teaching travel to create map data around the moving body and an automatic traveling mode in which the moving body automatically travels at the location where the teaching travel was performed.

[0046] The processing unit 20A includes a teaching travel processing unit 20A1 that performs processing in the teaching travel mode and an automatic travel processing unit 20A2 that performs processing in the automatic travel mode. The teaching travel processing unit 20A1 is the processing unit 20A that executes the teaching travel mode. The automatic travel processing unit 20A2 is the processing unit 20A that executes the automatic travel mode.

[0047] The teaching travel processing unit 20A1 includes an acquisition unit 20C, an extraction unit 20D, a creation unit 20E, a determination unit 20F, a specification unit 20G, an output control unit 20H, and a reception unit 20I.

[0048] The acquisition unit 20C, the extraction unit 20D, the creation unit 20E, the determination unit 20F, the specification unit 20G, the output control unit 20H, and the reception unit 20I are realized by, for example, one or a plurality of processors. For example, each of the above units may be realized by causing a processor such as a CPU to execute a program, that is, by software. Each of the above units may be realized by a processor such as a dedicated IC (Integrated Circuit), that is, by hardware. Each of the above units may be realized by using a combination of software and hardware. When using a plurality of processors, each of the plurality of processors may realize one of the plurality of units, or may realize two or more of the plurality of units.

[0049] The processor realizes the plurality of units by reading and executing the program stored in the storage unit 20B. Instead of storing the program in the storage unit 20B, it may be configured to directly incorporate the program into the circuit of the processor. In this case, the processor realizes the plurality of units by reading and executing the program incorporated in the circuit.

[0050] The acquisition unit 20C acquires peripheral information from the imaging device 10D. As described above, in the present embodiment, the imaging device 10D obtains a peripheral image 40, which is a captured image around the moving body 10, as peripheral information. Therefore, the acquisition unit 20C acquires the peripheral image 40 of the moving body 10 from the imaging device 10D.

[0051] The imaging device 10D obtains the peripheral image 40 at predetermined timings along the time series. Then, each time the imaging device 10D acquires the peripheral image 40, the acquired peripheral image 40 is output to the teacher driving processing unit 20A1. Therefore, the acquisition unit 20C of the teacher driving processing unit 20A1 sequentially acquires the peripheral image 40 from the imaging device 10D.

[0052] The extraction unit 20D extracts feature points around the path traveled by the moving body 10 by analyzing the peripheral image 40 acquired by the acquisition unit 20C from the imaging device 10D.

[0053] For the method of extracting feature points by the extraction unit 20D, a known method may be used, and the extraction method is not limited.

[0054] The extraction unit 20D assigns identification information to the extracted feature points, and stores the identification information in association with the information indicating the position and range of the feature points in the real space in the storage unit 20B. The extraction unit 20D may extract the information indicating the position and range of the feature points in the real space by using the position information of the moving body 10 obtained from the inner sensor 10C.

[0055] The creation unit 20E creates map data having the position information of the feature points extracted by the extraction unit 20D.

[0056] The map data created by the creation unit 20E is information used for self-position estimation, and for each of a plurality of feature points in the real scene obtained in advance in a wide area (including the area around the moving body 10), it stores in association the three-dimensional position in the real space and the feature quantity which is data indicating the features of the feature points extracted from the image information taken at the time of creating the map data. The feature points stored as the map data are, for example, parts (such as corner parts) from which characteristic image patterns can be obtained from the image information of objects (such as buildings, signs, or billboards, etc.) that can serve as landmarks in the real scene. Also, as the position information in the real scene, pre-stored landmarks such as position markers may be used. Note that the plurality of feature points of the map data are stored so as to be individually identifiable by, for example, identification numbers.

[0057] The three-dimensional position in the real space of the feature points stored in the map data is represented in a three-dimensional orthogonal coordinate system (X, Y, Z) based on, for example, latitude, longitude, and height. Note that the three-dimensional position in the real space of the feature points is stored by, for example, measurement based on the principle of triangulation from camera images taken at a plurality of positions, or measurement using LIDAR (Light Detection and Ranging) or a stereo camera.

[0058] As the feature amount of the feature points stored in the map data, in addition to the luminance and density of the image information, SIFT (Scale Invariant Feature Transform) feature amount, or SURF (Speeded Up Robust Features) feature amount, etc. are used. Note that the feature amount data of the feature points stored in the map data may be stored separately for each shooting position and shooting direction of the shooting device when shooting the feature points, even if they are feature points at the same three-dimensional position. Also, the feature amount data of the feature points stored in the map data may be stored in association with the image information of the object having the feature points.

[0059] For the method of creating map data by the creation unit 20E, a known method may be used, and the method of creating map data is not limited. For example, the creation unit 20E creates map data having the position information of the feature points by creating position information indicating the positions of the feature points extracted by the extraction unit 20D.

[0060] FIG. 5 is a schematic diagram showing an example of an application scenario of the mobile body 10 of the present embodiment. For example, for the purpose of learning the route for automatically driving the mobile body 10 from point A to point B along the arrow, a scenario is assumed in which the user U is performing a teaching drive of the mobile body 10 from point A to point B. Point A is, for example, the position where the user U gets off the mobile body 10 and is also the start position for automatically driving the mobile body 10 unmanned. Point B is, for example, the parking target position of the mobile body 10. The driver U1 is an example of the user U who has boarded the mobile body 10.

[0061] In this case, the shooting device 10D provided in the mobile body 10 shoots the periphery of the mobile body 10, so that the acquisition unit 20C acquires the peripheral image 40 of the periphery. Therefore, the extraction unit 20D acquires the peripheral image 40 of the periphery of the mobile body 10 from the acquisition unit 20C. Then, the extraction unit 20D extracts the feature points included in the peripheral image 40 by analyzing the peripheral image 40. Further, the creation unit 20E creates map data having the position information of the feature points extracted by the extraction unit 20D. The creation unit 20E stores the created map data in the map data storage unit 20B3.

[0062] Returning to FIG. 4, the description will be continued. The determination unit 20F extracts a part of the route traveled by the moving body 10 according to a predetermined condition, and determines whether the number of the extracted feature points is insufficient with respect to a predetermined number required for self-position estimation in each of the extracted routes (hereinafter referred to as "partial routes"). If the number of the extracted feature points is larger than the predetermined number, it is considered to be a place where the possibility of successful self-position estimation is high. The predetermined condition is, for example, that the vehicle moves a predetermined distance (for example, 20 cm), or that a predetermined number or more of identical feature points are continuously extracted in the temporally continuous images acquired by the imaging device 10D, and the like.

[0063] The determination unit 20F repeats the determination for each partial route, and identifies a location where the number of feature points is insufficient with respect to the predetermined number.

[0064] The identification unit 20G executes self-position estimation in order to identify the position of the moving body 10 during automatic driving. The identification unit 20G compares the map data created by the creation unit 20E with the images acquired by the imaging device 10D mounted on the moving body 10. Thereby, the identification unit 20G can identify where in the map data created by the moving body 10 the moving body 10 is located.

[0065] Next, the output control unit 20H will be described. The output control unit 20H outputs information to the output unit 10A. In the present embodiment, the output control unit 20H outputs instruction information to the output unit 10A.

[0066] The indication information indicates information different from the case where the number of feature points is not insufficient with respect to a predetermined number when it is determined by the determination unit 20F that the number of feature points is insufficient with respect to the predetermined number. Note that the indication information may be information indicating that the number of feature points is insufficient with respect to the predetermined number when it is determined by the determination unit 20F that the number of feature points is insufficient with respect to the predetermined number. Further, the indication information may be information indicating a location where the number of feature points is insufficient with respect to the predetermined number when it is determined by the determination unit 20F that the number of feature points is insufficient with respect to the predetermined number. In addition, the indication information may be information for notifying a location where an object containing a large amount of feature amounts (hereinafter referred to as a feature object) or a position marker that is a landmark stored in the teacher driving processing unit 20A1 in advance is to be installed in order to add position information to the map data when it is determined by the determination unit 20F that the number of feature points is insufficient with respect to the predetermined number. The location where the feature object or the position marker is to be installed is a location where there is a high possibility that the shortage of the number of feature points will be resolved when the teacher driving is performed again with the feature object or the position marker installed. That is, the location is an effective location for resolving the shortage of the number of feature points.

[0067] In the present embodiment, as an example, a form in which the indication information notifies a location where a feature object or a position marker is to be installed in order to add position information to the map data when it is determined by the determination unit 20F that the number of feature points is insufficient with respect to the predetermined number will be described.

[0068] The indication information may be any of an indication image represented by an image, an indication voice represented by a voice, and an indication character represented by text. Further, a combination of these may also be used. Note that when the indication information is an image, the indication information will be described as an indication image.

[0069] FIG. 6 is a schematic diagram showing an example of the indication image 52. As shown in FIG. 6, the indication image 52 is an image for notifying a location where a feature object or a position marker is to be installed in the vicinity of the moving body 10 in order to add position information to the map data.

[0070] The output control unit 20H generates an instruction image 52 using the map data created by the creation unit 20E, the information determined by the determination unit 20F as to whether the number of feature points is insufficient with respect to a predetermined number for each partial path, and the position information of the moving body 10 estimated by the specifying unit 20G.

[0071] For example, when the determination unit 20F determines that the number of feature points is insufficient with respect to a predetermined number, the output control unit 20H generates an instruction image 52 indicating a location where the number of feature points is insufficient with respect to the predetermined number. Specifically, when the determination unit 20F determines that the number of feature points is insufficient with respect to a predetermined number, the output control unit 20H generates the instruction image 52 for a location where the number of feature points is insufficient with respect to the predetermined number, as a location where a feature object or a position marker should be installed.

[0072] Here, a partial path in which the number of feature points is insufficient with respect to a predetermined number required to estimate the self-position of the moving body 10 is hereinafter referred to as a non-automatic-drivable path RP. As shown in FIG. 6, it is preferable that the output control unit 20H generates an instruction image 52 in which the non-automatic-drivable path RP is emphasized. For example, the non-automatic-drivable path RP is a frame image that is an image indicating the non-automatic-drivable path RP, or a color image that is an image in which the color of the non-automatic-drivable path RP is shown in a color that attracts the attention of the user U. The color that attracts attention is, for example, yellow or red, but is not limited to these colors. By using an image as the instruction information to show the user, the user can know the specific position where the non-automatic-drivable path RP is located.

[0073] Then, the output control unit 20H outputs the generated instruction image 52 to the output unit 10A in response to an instruction to end the teaching run.

[0074] In the present embodiment, the output control unit 20H displays the instruction image 52 on one or more of the plurality of display units 10K provided on the moving body 10.

[0075] Incidentally, when using an instruction voice expressed by voice as instruction information, the output control unit 20H may output the instruction voice from the speaker 10I. For example, the output control unit 20H may output an instruction voice such as "There is a place where automatic driving seems difficult. The teaching run will end." from the speaker 10I. Additionally, the display of an instruction image may also be performed.

[0076] Returning to FIG. 4, the description will be continued. Next, the reception unit 20I will be described. When the determination unit 20F determines that the number of feature points is insufficient with respect to a predetermined number, the reception unit 20I receives an input of an instruction to retry the teaching run in the instruction image 52.

[0077] For example, it is assumed that the instruction image 52 shown in FIG. 6 is displayed on the display unit 10K. When the determination unit 20F determines that the number of feature points is insufficient with respect to a predetermined number in the instruction image 52, the user U installs a feature object or a position marker at the location where the number of feature points is insufficient, and then inputs an instruction to retry the teaching run.

[0078] The user U inputs an instruction to retry the teaching run in the instruction image 52 by operating the input unit 10B, for example. Specifically, when the input unit 10B is a touch panel, the user U touches the retry button displayed on this touch panel to input an instruction to retry the teaching run. The reception unit 20I receives the input of the instruction to retry the teaching run by receiving the information instructed by the user U from the input unit 10B.

[0079] Incidentally, the reception unit 20I may receive the input of the instruction to retry the teaching run by analyzing the gesture of the user U or the voice emitted from the user U.

[0080] Also, for example, the user U emits a voice indicating that the user desires a teaching run. In this case, the reception unit 20I may receive the input of the instruction to retry the teaching run by analyzing the voice data of the voice collected by the microphone as the input unit 10B using a known voice analysis method.

[0081] Upon receiving an instruction for a retrial of the instructor driving, the reception unit 20I outputs information indicating the instruction for the retrial of the instructor driving to the autonomous driving processing unit 20A2.

[0082] The autonomous driving processing unit 20A2 estimates its own position based on the map data stored in the storage unit 20B and performs autonomous driving.

[0083] The storage unit 20B includes an instructor driving program 20B1, an autonomous driving program 20B2, and a map data storage unit 20B3.

[0084] When the instructor driving program 20B1 is read by the instructor driving processing unit 20A1, processing in the instructor driving mode is executed based on the instructor driving program 20B1. When the autonomous driving program 20B2 is read by the autonomous driving processing unit 20A2, processing in the autonomous driving mode is executed based on the autonomous driving program 20B2. The map data storage unit 20B3 stores the map data created by the creation unit 20E.

[0085] Next, an example of the information processing procedure for instructor driving in Embodiment 1 executed by the moving body 10 will be described. FIG. 7 is a flowchart showing an example of the information processing procedure for instructor driving in Embodiment 1 executed by the moving body 10.

[0086] First, the acquisition unit 20C acquires the surrounding image 40 from the imaging device 10D (step S101). Next, the extraction unit 20D extracts feature points from the surrounding image 40 acquired in step S101 (step S103). That is, information used for self-position estimation is extracted from the surrounding image 40 acquired in step S101.

[0087] In step S105, the creation unit 20E adds map data having the position information of the feature points extracted in step S103 to the map data storage unit 20B3 (step S105).

[0088] In step S107, the determination unit 20F extracts a partial route from the created map data according to a predetermined condition (step S107). The predetermined condition is, for example, that a predetermined number or more of identical feature points are continuously extracted in images that are temporally continuous and acquired by the imaging device 10D, and so on.

[0089] Next, for each partial route, the determination unit 20F stores whether the number of feature points is insufficient with respect to a predetermined number, that is, whether it is less than the predetermined number (step S109).

[0090] In step S111, the input unit 10B determines whether or not the teaching run has ended by the user (step S111). If the teaching run has ended (step S111: Yes), the process proceeds to step S113. If the teaching run has not ended (step S111: No), the process returns to the above step S101. Note that a known method may be used for the method of recognizing that the end point of the teaching run has been reached, and the method is not limited. For example, by receiving an instruction to end the teaching run from the user via the input unit 10B, it is recognized that the end point of the teaching run has been reached. Alternatively, it may be recognized that the end point of the teaching run has been reached when the shift lever is placed in the parking range.

[0091] Next, the determination unit 20F determines, for each partial path, whether the number of feature points is insufficient with respect to a predetermined number required to estimate the self-position (step S113). That is, it is determined whether the amount of information used for self-position estimation is insufficient with respect to a predetermined value. If the number of feature points in any of the partial paths of the path traveled during teaching driving is not insufficient with respect to the predetermined number, that is, if it is more than the predetermined number (step S113: No), the process proceeds to step S115. In step S115, the user is notified of the normal end of the teaching driving, and this routine ends. When notifying the user of the end of the teaching driving, it may be notified that automatic driving in the automatic driving mode is possible and the end of the teaching driving. If any one or more of the extracted paths of the path traveled during teaching driving have a number of feature points insufficient with respect to the predetermined number, that is, if it is less than the predetermined number (step S113: Yes), the process proceeds to step S117.

[0092] If any one or more of the extracted paths have a number of feature points insufficient with respect to the predetermined number, that is, if it is less than the predetermined number (step S113: Yes), the output control unit 20H outputs the instruction information to the output unit 10A (step S117). The instruction information is, for example, information for the display unit 10K to notify that there is a high possibility that automatic driving is not possible in the instruction image 52. Thereby, the user can know that there is a high possibility that automatic driving is not possible after this teaching driving. That is, the user can know that there is a high possibility that automatic driving is not possible without attempting automatic driving. Note that this instruction information may be notified by voice. When the instruction information is notified by voice, the driver U1 can know that there is a high possibility that automatic driving is not possible without looking at the display unit 10K. Thereby, when the driver U1 is looking at a location other than the display unit 10K, it can be known that there is a high possibility that automatic driving is not possible.

[0093] Next, the input unit 10B determines whether an instruction to retry the teaching run has been input (step S119). For example, when the display unit 10K is a touch panel, the input unit 10B is a retry button displayed on this touch panel, and the user U inputs an instruction to retry the teaching run by touching the retry button on the display unit 10K. If an instruction to retry the teaching run has been input (step S119: Yes), the process returns to step S101 above. If an instruction to retry the teaching run has not been input (step S119: No), the process proceeds to step S121.

[0094] In step S121, after notifying the instruction information (step S117), it is determined whether a predetermined time has elapsed. The predetermined time is, for example, five minutes. If it is determined that the predetermined time has elapsed (step S121: Yes), the process proceeds to step S123. In step S123, an abnormal termination indicating that the teaching run has not ended normally is notified to the user, and this routine ends. Note that when notifying the user of the end of the teaching run, the notification method may be made different between step S115 and step S123. If it is determined that the predetermined time has not elapsed (step S121: No), the process returns to step S119 above.

[0095] Next, an example of the information processing procedure for automatic driving in Embodiment 1 executed by the mobile body 10 will be described. FIG. 8 is a flowchart showing an example of the information processing procedure for automatic driving in Embodiment 1 executed by the mobile body 10.

[0096] First, the automatic driving processing unit 20A2 estimates the initial position based on the map data stored in the map data storage unit 20B3 and the captured image (step S200). The estimated initial position is set as the self-position (step S201). Next, the automatic driving processing unit 20A2 performs automatic driving from the self-position estimated in step S201 (step S203). In step S205, the automatic driving processing unit 20A2 determines whether the moving body 10 has reached the destination (step S205). If it is determined that the moving body 10 has not reached the destination (step S205: No), the process returns to step S201 above. If it is determined that the moving body 10 has not reached the destination (step S205: No), the processes from step S201 to step 205 are repeatedly performed. In the repeatedly performed step S201, the self-position is estimated based on the map data stored in the map data storage unit 20B3 and the captured image. Also, in step S205, if it is determined that the moving body 10 has reached the destination (step S205: Yes), this routine is terminated.

[0097] In this way, even if the automatic driving is not actually performed, the user can know that there is a high possibility that the automatic driving cannot be performed, improving the convenience for the user.

[0098] <Modification Example 1> In the teaching driving in FIG. 7, when the number of feature points is insufficient with respect to the predetermined number, it is notified in step S117 that there is a place where there is a high possibility that automatic driving cannot be performed. However, since the user cannot confirm the place where the number of feature points is insufficient with respect to the predetermined number, there is a possibility that the number of feature points cannot be smoothly supplemented and the teaching driving cannot be terminated after making the automatic driving possible. Therefore, in step S117, an instruction image as shown in FIG. 6 may be used to notify the place where the number of feature points is insufficient with respect to the predetermined number as the place where the feature object should be installed. By doing so, the user can know the place where the feature object should be installed. If the user can know the place where the feature object should be installed, by installing the feature object at that place, the surrounding environment where automatic driving is possible can be prepared even in a place where automatic driving cannot be performed during teaching driving.

[0099] <Modification Example 2> In the teaching run of Modification Example 1, when the number of feature points is insufficient, the partial path where the number of feature points is insufficient with respect to the predetermined number is notified as the place where the feature object should be installed. However, it may be notified as the place where the position marker should be installed instead of the feature object. FIG. 9 is a flowchart showing an example of the information processing procedure of the teaching run in Modification Example 2 executed by the moving body 10. Hereinafter, the information processing procedure of the teaching run in Modification Example 2 shown in FIG. 9 will be described.

[0100] First, the determination unit 20F extracts a partial path according to a predetermined condition (step S301). The predetermined condition is, for example, that the same feature points are continuously extracted more than a predetermined number in the temporally consecutive images acquired by the imaging device 10D. Next, the acquisition unit 20C acquires the peripheral image 40 of the partial path extracted in step S301 (step S303).

[0101] In step S305, the extraction unit 20D determines whether a position marker has been extracted from the peripheral image 40 of the partial path acquired in step S303 (step S305). That is, it is determined whether information used for self-position estimation has been extracted from the peripheral image 40 of the partial path acquired in step S303. If no position marker is extracted from the peripheral image 40 of the partial path (step S305: No), the process proceeds to step S311. If a position marker is extracted from the peripheral image 40 of the partial path (step S305: Yes), the process proceeds to step S307.

[0102] In step S307, the creation unit 20E stores the map data having the position information of the position marker extracted in step S307 in the map data storage unit 20B3 (step S307). In the case of the second time and later, the position of the position marker extracted this time is added to the stored map data.

[0103] Next, the determination unit 20F determines whether the number of position markers on the partial path is insufficient with respect to a predetermined number required to estimate its own position (step S309). That is, it is determined whether the amount of information used for self-position estimation is insufficient with respect to a predetermined value. If the number of position markers on the partial path is not insufficient with respect to the predetermined number required to estimate its own position, that is, if it is more than the predetermined number (step S309: No), the process proceeds to step S317. If the number of position markers on the partial path is insufficient with respect to the predetermined number required to estimate its own position, that is, if it is less than the predetermined number (step S309: Yes), the process proceeds to step S311.

[0104] In step S311, the extraction unit 20D extracts feature points from the peripheral image 40 of the partial path acquired in step S303 (step S311). Next, the creation unit 20E adds map data having the position information of the feature points extracted in step S311 to the map data storage unit 20B3 (step S313). Next, in step S315, the determination unit 20F stores whether the number of feature points on the partial path is insufficient with respect to a predetermined number, that is, whether it is less than the predetermined number (step S315).

[0105] In step S317, it is determined whether the processes from step S301 to step S315 have been performed on the peripheral images 40 of all the partial paths up to the teacher driving end point (step S317). Note that a known method may be used to recognize that the teacher driving end point has been reached, and the method is not limited. For example, by receiving an instruction to end the teacher driving from the user via the input unit 10B, it is recognized that the teacher driving end point has been reached. Alternatively, it may be recognized that the teacher driving end point has been reached when the shift lever enters the parking range.

[0106] When it is determined that the peripheral image 40 has been processed for all partial routes up to the teacher driving end point (step S317: Yes), the process proceeds to step S319. When it is determined that the peripheral image 40 has not been processed for all partial routes up to the teacher driving end point (step S317: No), the process returns to step S301 above.

[0107] In step S319, it is determined whether there is a partial route in which the number of feature points is insufficient with respect to the predetermined number among all the partial routes in which the position markers are insufficient with respect to the predetermined number (step S319). When there is no partial route in which the number of feature points is insufficient with respect to the predetermined number among all the partial routes in which the position markers are insufficient with respect to the predetermined number (step S319: No), the process proceeds to step S321. In step S321, the user is notified of the normal end of the teacher driving, and this routine ends. When notifying the user of the end of the teacher driving, it may be notified that the automatic driving in the automatic driving mode is possible and the end of the teacher driving. When there is a partial route in which the number of feature points is insufficient with respect to the predetermined number among all the partial routes in which the position markers are insufficient with respect to the predetermined number, that is, when the route traveled in the teacher driving includes a partial route in which both the number of position markers and the number of feature points are insufficient with respect to the predetermined number (step S319: Yes), the process proceeds to step S323.

[0108] In step S323, the output control unit 20H outputs the instruction information to the output unit 10A (step S323). The instruction information is, for example, information for the display unit 10K to notify the installation location of the feature object or the position marker.

[0109] Next, the input unit 10B determines whether an instruction to retry the teaching run has been input (step S325). For example, when the display unit 10K is a touch panel, the input unit 10B is a retry button displayed on this touch panel, and the user U inputs an instruction to retry the teaching run by touching the retry button on the display unit 10K. If an instruction to retry the teaching run has been input (step S325: Yes), the process returns to step S301 above. If an instruction to retry the teaching run has not been input (step S325: No), the process proceeds to step S327.

[0110] In step S327, after notifying the instruction information (step S323), it is determined whether a predetermined time has elapsed. The predetermined time is, for example, five minutes. If it is determined that the predetermined time has elapsed (step S327: Yes), the process proceeds to step S329. In step S329, an abnormal end indicating that the teaching run has not ended normally is notified to the user, and this routine ends. When notifying the user of the end of the teaching run, the notification method may be made different between step S321 and step S329. If it is determined that the predetermined time has not elapsed (step S329: No), the process returns to step S325 above.

[0111] Next, an example of the information processing procedure for the automatic driving in the second modification executed by the mobile body 10 will be described. FIG. 10 is a flowchart showing an example of the information processing procedure for the automatic driving in the second modification executed by the mobile body 10.

[0112] First, the automatic driving processing unit 20A2 performs initial position estimation (step S401). Next, in step S403, the automatic driving processing unit 20A2 determines whether there is a position marker around the moving body 10 (step S403). If it is determined that there is a position marker around the moving body 10 (step S403: Yes), the process proceeds to step S405. In step S405, the automatic driving processing unit 20A2 specifies the position of the position marker from the map data stored in the map data storage unit 20B3 (step S405). Next, in step S407, the automatic driving processing unit 20A2 estimates its own position from the position of the position marker specified in step S405. (step S407).

[0113] If it is determined in step S403 that there is no position marker around the moving body 10 (step S403: No), the process proceeds to step S409. In step S409, the automatic driving processing unit 20A2 specifies the position of the feature point from the map data stored in the map data storage unit 20B3 (step S409). Next, in step S411, the automatic driving processing unit 20A2 estimates its own position from the position of the feature point specified in step S409. (step S411). Next, the automatic driving processing unit 20A2 performs automatic driving from the own position estimated in step S407 or step S411 (step S412).

[0114] In step S413, the automatic driving processing unit 20A2 determines whether it has arrived at the destination (step S413). If it is determined that it has not arrived at the destination (step S413: No), the process returns to step S401 above. If it is determined that it has arrived at the destination (step S413: Yes), this routine ends.

[0115] As described above, the information processing apparatus 20 of the present embodiment includes an extraction unit 20D, a determination unit 20F, and an output control unit 20H. The extraction unit 20D receives image information that is information obtained by photographing the periphery of the moving body, and extracts feature points of the received image information. The determination unit 20F determines whether the number of feature points extracted by the extraction unit 20D is insufficient with respect to a predetermined number. When the determination unit 20F determines that the number of feature points is insufficient with respect to the predetermined number, the output control unit 20H outputs instruction information (instruction image 52) different from the case where the number of feature points is not insufficient with respect to the predetermined number.

[0116] Here, in the prior art, when performing a teaching run in which a vehicle is driven to create map data of the surroundings in a location where the features of the environment around the vehicle are not sufficient, such as a location surrounded by an object having no unevenness or color change, the number of feature points included in the created map data is insufficient with respect to the predetermined number required to estimate the self-position. Therefore, there is a high possibility that the self-position cannot be estimated. However, in the prior art, until actually performing automatic driving, the user could not confirm that there is a high possibility that automatic driving cannot be performed at the location where the teaching run was performed.

[0117] On the other hand, the information processing apparatus 20 of the present embodiment outputs an instruction image 52 different from the case where the number of feature points around the moving body is not insufficient with respect to the predetermined number when the number of feature points around the moving body is insufficient with respect to the predetermined number by the teaching run. For this reason, when the determination unit 20F determines that the number of feature points is insufficient with respect to the predetermined number, the information processing apparatus 20 can output an instruction image 52 indicating that the number of feature points is insufficient with respect to the predetermined number. For example, without the user U actually performing automatic driving, information indicating a location where the number of feature points is insufficient with respect to the predetermined number can be output.

[0118] Therefore, in the information processing apparatus 20 of the present embodiment, the user can confirm, without performing automatic driving, that there is a high possibility that the vehicle cannot perform automatic driving at the location where the instructor has driven. In addition, since it may be difficult for the user to determine what kind of objects include a large amount of feature amounts, the user is provided with position markers prepared in advance for instructor driving, and by using the position markers, an environment where there is a high possibility that automatic driving cannot be performed can be changed into an environment where automatic driving can be performed in a simpler manner.

[0119] <Embodiment 2> In Embodiment 1, the amount of information used for self-position estimation uses the number of feature points. In Embodiment 2, as the information used for self-position estimation, information obtained by a tracking process, which is a process of tracking the same feature points extracted from continuous image information, is used, and the amount of information used for self-position estimation uses the number of feature points tracked as a result of this tracking process. Hereinafter, Embodiment 2 will be described in detail. For the same configurations as those in Embodiment 1, the same reference numerals will be given and the description thereof will be omitted.

[0120] FIG. 11 is a block diagram showing an example of the functional configuration of the moving body 10 in Embodiment 2. In Embodiment 2, the matching processing unit 20J, the tracking processing unit 20K, and the creation unit 20E are different from the configuration of FIG. 4 in Embodiment 1.

[0121] The matching processing unit 20J determines whether feature points extracted from continuous images are the same by using the similarity of feature amounts (hereinafter, this process is referred to as a matching process). That is, among the images used for the matching process, it is determined whether a feature point a1 included in a frame A, which is a past image, and a feature point b1 included in a frame B, which is a current image, are the same feature point. As a matching method, which is a method for determining whether feature points between frames are the same, a known method may be used, and the matching method is not limited.

[0122] The tracking processing unit 20K performs tracking processing on the feature points that are matched in the matching processing. The tracking processing unit 20K determines whether the distance between the position X estimated for the feature point a1 extracted from frame A in frame B and the position Y of the feature point b1 extracted from frame B is smaller than a predetermined value (hereinafter, this processing is referred to as tracking processing). If the distance between the estimated position X and the extracted position Y is smaller than the predetermined value, it is assumed that the feature point has been tracked. The predetermined value is, for example, 5 pixels. Note that a known method may be used for the tracking method, and the tracking method is not limited. For example, it is possible to estimate the position of the feature point in image B from the position of the feature point in image A by estimating the amount of movement of the imaging device from the speed of the moving body 10, the tire angle, etc. The position of the feature point is, for example, coordinates on the image. If the number of tracked feature points is more than a predetermined number, it is considered to be a place where the possibility of successful self-position estimation is high.

[0123] The creation unit 20E creates map data having the position information of the tracked feature points. Note that the creation unit 20E may not include the feature points that were not tracked as the positions of the feature points extracted by the extraction unit 20D.

[0124] FIG. 12 is a flowchart showing an example of the procedure of information processing for teaching driving in Embodiment 2 executed by the moving body 10.

[0125] First, the acquisition unit 20C acquires the peripheral image 40 from the imaging device 10D (step S501). Next, the extraction unit 20D extracts feature points from the peripheral image 40 acquired in step S501 (step S503). That is, information used for self-position estimation is extracted from the peripheral image 40 acquired in step S501.

[0126] In step S505, the matching processing unit 20J performs matching processing between the feature points extracted from the current image and the feature points extracted from the past images consecutive to the current image (step S505). Next, the tracking processing unit 20K performs tracking processing on the feature points matched in step S505 (step S507).

[0127] In step S509, the creation unit 20E creates map data having the position information of the feature points tracked in step S507 and stores it in the map data storage unit 20B3 (step S509). In the processing of step S509 after the second time, the map data created this time is added to the already stored map data.

[0128] In step S511, the determination unit 20F extracts a partial route from the map data according to a predetermined condition (step S511). The predetermined condition is, for example, that the same feature points are continuously extracted more than a predetermined number in the temporally consecutive images acquired by the imaging device 10D, etc.

[0129] Next, for each of the partial routes extracted by the determination unit 20F, it is stored whether the number of feature points is insufficient with respect to the predetermined number, that is, less than or more than the predetermined number (step S513).

[0130] In step S515, the input unit 10B determines whether an instruction to end the teaching run has been input from the user (step S515).

[0131] Next, the determination unit 20F determines, for each partial path, whether the number of tracked feature points is insufficient with respect to a predetermined number required to estimate the self-position (step S517). That is, it is determined whether the amount of information used for self-position estimation is insufficient with respect to a predetermined value. If the number of feature points in any of the partial paths is not insufficient with respect to the predetermined number, that is, if it is more than the predetermined number (step S517: No), the process proceeds to step S519. In step S519, the user is notified of the normal end of the teaching run, and this routine ends. When notifying the user of the end of the teaching run, it may be notified that automatic driving in the automatic driving mode is possible and the teaching run has ended. If one or more of the extracted paths have a number of tracked feature points that is insufficient with respect to the predetermined number, that is, if it is less than the predetermined number (step S517: Yes), the process proceeds to step S521.

[0132] If one or more of the extracted paths have a number of tracked feature points that is insufficient with respect to the predetermined number, that is, if it is less than the predetermined number (step S517: Yes), the output control unit 20H outputs instruction information indicating that there is a high possibility that automatic driving cannot be performed to the output unit 10A (step S521). The instruction information is, for example, information for the display unit 10K to notify that automatic driving is not possible in the instruction image 52. Thereby, the user can know that automatic driving is not possible after this teaching run. That is, the user can know that automatic driving is not possible without attempting automatic driving. Note that this instruction information may be notified by voice. When the instruction information is notified by voice, the driver U1 can know that automatic driving is not possible without looking at the display unit 10K. Thereby, when the driver U1 is looking at a location other than the display unit 10K, it can be known that automatic driving is not possible.

[0133] Next, the input unit 10B determines whether an instruction to retry the teaching run has been input (step S523). For example, when the display unit 10K is a touch panel, the input unit 10B is a retry button displayed on this touch panel, and the user U inputs an instruction to retry the teaching run by touching the retry button on the display unit 10K. If an instruction to retry the teaching run has been input (step S523: Yes), the process returns to step S501 above. If an instruction to retry the teaching run has not been input (step S523: No), the process proceeds to step S525.

[0134] In step S525, after notifying the instruction information (step S521), it is determined whether a predetermined time has elapsed. The predetermined time is, for example, 5 minutes. If it is determined that the predetermined time has elapsed (step S525: Yes), this routine ends. If it is determined that the predetermined time has not elapsed (step S525: No), the process returns to step S523 above.

[0135] Next, an example of the information processing procedure for automatic driving in Embodiment 2 executed by the moving body 10 will be described. FIG. 13 is a flowchart showing an example of the information processing procedure for automatic driving in Embodiment 2 executed by the moving body 10.

[0136] First, the automatic driving processing unit 20A2 estimates its own position based on the map data stored in the map data storage unit 20B3 and the captured image (step S601). Next, the automatic driving processing unit 20A2 performs automatic driving from the own position estimated in step S601 (step S603). In step S605, the automatic driving processing unit 20A2 determines whether the moving body 10 has reached the destination (step S605). If it is determined that the moving body 10 has not reached the destination (step S605: No), the process returns to step S601 above. If it is determined that the moving body 10 has reached the destination (step S605: Yes), this routine ends.

[0137] In this way, even if the automatic driving is not actually performed, the user can know that there is a high possibility that the automatic driving cannot be performed.

[0138] <Other Embodiments> In Modification 1, a partial path where the number of feature points is insufficient with respect to a predetermined number is notified as a location where a feature object should be installed. In Modification 2, a location where a position marker should be installed is notified instead of the feature object. However, the partial path where the number of feature points is insufficient with respect to the predetermined number may be notified as a location where a feature object or a position marker, or both a feature object and a position marker should be installed. Thereby, the degree of freedom of the object to be installed is increased, and the convenience for the user can be further improved.

[0139] In addition, in the present embodiment, the form in which the information processing apparatus 20 is mounted on the moving body 10 has been described as an example. However, the information processing apparatus 20 may be configured to be mounted outside the moving body 10. In this case, the information processing apparatus 20 may be configured to be communicable with each of the electronic devices such as the internal sensor 10C mounted on the moving body 10 via a network.

[0140] Note that the program for executing the above information processing in the above-described embodiment has a module configuration including each of the above plurality of functional units. As actual hardware, for example, a CPU (processor circuit) reads an information processing program from a ROM or an HDD and executes it, so that each of the above plurality of functional units is loaded onto a RAM (main memory), and each of the above plurality of functional units is generated on the RAM (main memory). Note that a part or all of each of the above plurality of functional units can also be realized using dedicated hardware such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field-Programmable Gate Array).

[0141] Incidentally, although the above describes embodiments, the above embodiments are presented as examples and are not intended to limit the scope of the present disclosure. The above novel embodiments can be implemented in various other forms, and various omissions, replacements, and changes can be made without departing from the gist of the invention. The above embodiments are included in the scope or gist of the present disclosure and are included in the invention described in the claims and the equivalent scope thereof.

Explanation of Signs

[0142] 10 Moving body 10A Output unit 10B Input unit 10C Inner world sensor 10D Imaging device 10F Drive control unit 10G Drive unit 10H Communication unit 10I Speaker 10J Lighting unit 10K Display unit 10L Bus 11A CPU 11B ROM 11C RAM 11D I / F 11E Bus 20 Information processing device 20A Processing unit 20A1 Teacher driving processing unit 20A2 Autonomous driving processing unit 20B Storage unit 20B1 Teacher driving program 20B2 Autonomous driving program 20B3 Map data storage unit 20C Acquisition unit 20D Extraction unit 20E Creation unit 20F Judgment unit 20G Identification unit 20H Output control unit 20I Reception unit 20J Matching processing unit 20K Tracking processing unit

Claims

1. In an information processing apparatus that performs automatic driving based on driving route data indicating a driving route when a user manually drives a vehicle, an extraction unit that receives image information, which is information obtained by photographing the periphery of a moving object during the teaching drive, and extracts information used for self-position estimation obtained from the received image information; an output control unit that outputs, to an output unit, instruction information different from a case where the amount of information used for the self-position estimation is a second amount that is greater than a first amount when the amount of information used for the self-position estimation extracted by the extraction unit is the first amount; comprising, the start position of the teaching drive is the start position of the automatic drive, the end position of the teaching drive is a parking target position, when the amount of information used for the self-position estimation is the second amount, the output control unit notifies the user of a normal end of the teaching drive, the instruction information is output before performing the automatic drive based on the driving route data, an information processing apparatus.

2. The first amount is an amount by which the amount of information used for the self-position estimation is insufficient with respect to a predetermined value, The information processing apparatus according to claim 1.

3. The second amount is an amount by which the amount of information used for the self-position estimation is sufficient with respect to a predetermined value, The information processing apparatus according to claim 1.

4. The instruction information is information indicating that the amount of information used for the self-position estimation is insufficient with respect to the predetermined value when the amount of information used for the self-position estimation is insufficient with respect to the predetermined value, The information processing apparatus according to claim 2.

5. The instruction information is information indicating that the amount of information used for the self-position estimation is sufficient with respect to the predetermined value when the amount of information used for the self-position estimation is sufficient with respect to the predetermined value, The information processing apparatus according to claim 3.

6. The output control unit, when the amount of information used for the self-position estimation is insufficient with respect to the predetermined value, outputs, as the instruction information, an instruction voice indicating information different from a case where the amount of information used for the self-position estimation is sufficient with respect to the predetermined value, The information processing apparatus according to claim 2.

7. The output control unit, when the amount of information used for the self-position estimation is sufficient with respect to the predetermined value, outputs, as the instruction information, an instruction voice indicating that the amount of information used for the self-position estimation is sufficient with respect to the predetermined value, The information processing apparatus according to claim 3.

8. The output control unit when the amount of information used for the self-position estimation is insufficient with respect to the predetermined value, outputs, as the instruction information, an instruction image indicating information different from the case where the amount of information used for the self-position estimation is sufficient with respect to the predetermined value The information processing apparatus according to claim 2.

9. The output control unit when the amount of information used for the self-position estimation is sufficient with respect to the predetermined value, outputs, as the instruction information, an instruction image indicating that the amount of information used for the self-position estimation is sufficient with respect to the predetermined value The information processing apparatus according to claim 3.

10. The amount of information used for the self-position estimation is represented by the number of feature points The information processing apparatus according to any one of claims 1 to 9.

11. In an information processing method for performing automatic driving based on travel route data indicating a travel route when a user manually performs a teaching run of a vehicle, information used for self-position estimation is extracted from image information, which is information obtained by photographing the periphery of the moving body during the teaching run, when the amount of information used for self-position estimation of the periphery of the moving body that is extracted is a first amount, different instruction information is output to an output unit from the case where the amount of information used for the self-position estimation is a second amount greater than the first amount including the start position of the teaching run is the start position of the automatic driving, the end position of the teaching run is a parking target position, when the amount of information used for the self-position estimation is the second amount, the user is notified of the normal end of the teaching run, the instruction information is output before performing the automatic driving based on the travel route data Information processing method

12. The first amount is an amount by which the amount of information used for the self-position estimation is insufficient with respect to the predetermined value The information processing method according to claim 11.

13. The second amount is an amount by which the amount of information used for the self-position estimation is sufficient with respect to the predetermined value The information processing method according to claim 11.

14. The instruction information is information indicating that the amount of information used for the self-position estimation is insufficient with respect to the predetermined value when the amount of information used for the self-position estimation is insufficient with respect to the predetermined value The information processing method according to claim 12.

15. The instruction information is information indicating that the amount of information used for the self-position estimation is sufficient with respect to the predetermined value when the amount of information used for the self-position estimation is sufficient with respect to the predetermined value The information processing method according to claim 13.

16. In the step of outputting the instruction information to the output unit, when the amount of information used for the self-position estimation is insufficient with respect to the predetermined value, an instruction voice indicating information different from the case where the amount of information used for the self-position estimation is sufficient with respect to the predetermined value is output as the instruction information. The information processing method according to claim 12.

17. In the step of outputting the instruction information to the output unit, when the amount of information used for the self-position estimation is sufficient with respect to the predetermined value, an instruction voice indicating that the amount of information used for the self-position estimation is sufficient with respect to the predetermined value is output as the instruction information. The information processing method according to claim 13.

18. In the step of outputting the instruction information to the output unit, when the amount of information used for the self-position estimation is insufficient with respect to the predetermined value, an instruction image indicating information different from the case where the amount of information used for the self-position estimation is sufficient with respect to the predetermined value is output as the instruction information. The information processing method according to claim 12.

19. In the step of outputting the instruction information to the output unit, when the amount of information used for the self-position estimation is sufficient with respect to the predetermined value, an instruction image indicating that the amount of information used for the self-position estimation is sufficient with respect to the predetermined value is output as the instruction information. The information processing method according to claim 13.

20. The amount of information used for the self-position estimation is represented by the number of feature points. The information processing method according to any one of claims 11 to 19.

Images