Information processing device and information processing method

Abstract

To enable a user to make sure that automatic driving can be impossible at a place where a teacher driving is completed, without performing the automatic driving.SOLUTION: An information processing device of the present disclosure comprises: an extraction unit for receiving image information, which is information on the surroundings of a moving body, and extracting from the received image information pieces of information used for self-position estimation; and an output control unit for, when the amount of information used for the self-position estimation extracted by the extraction unit is insufficient for a specified value in a running route, outputting to an output unit a piece of instruction information, which is different from one where the amount of information used for the self-position estimation is sufficient for the specified value. A start position of a teacher driving is a start position of an automatic driving. The start position of the teacher driving is a drop-off position of a user in the automatic driving. The output control unit notifies the user of the normal end of the teacher driving when the amount of information used for the self-position estimation is sufficient for the specified value, and the instruction information is outputted before the automatic driving based on running route data.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present disclosure relates to an information processing device and an information processing method. [Background technology]

[0002] A technology has been disclosed that creates map data based on the characteristics of objects around the vehicle, and then performs self-location estimation, which is a process of estimating where the vehicle is located in the map data, by comparing the created map data with an image captured by a photographing device mounted on the vehicle, and performs automatic driving along the route to be traveled. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2019-133318 A Summary of the Invention [Problem to be solved by the invention]

[0004] Generally, in such technology, the user manually drives the vehicle in advance in the location where automatic driving will be performed (hereinafter referred to as supervised driving), and creates map data used for self-position estimation and driving route data indicating the driving route required for automatic driving in that location, and thereafter, automatic driving is achieved by using the created map data and driving route data.

[0005] The map data used for self-location estimation during autonomous driving is data that stores the three-dimensional positions of feature points of objects present in the actual scene. During autonomous driving, feature points are extracted from an image captured by the vehicle, and the feature points are compared to which feature points included in the map data correspond. If the number of feature points that match the map data is equal to or greater than a predetermined number, it is determined that the vehicle's self-location has been estimated.

[0006] However, when performing supervised driving in which the vehicle is driven in order to create map data of a location where the environmental characteristics around the vehicle are insufficient, such as a location surrounded by objects with no unevenness or color changes, there is a high possibility that self-location estimation will not be possible because the number of feature points included in the created map data is insufficient for the predetermined number required for self-location estimation. However, with conventional technology, the user cannot confirm that there is a high possibility that autonomous driving will not be possible in the location where supervised driving has been performed until autonomous driving is actually performed.

[0007] The problem that the present disclosure aims to solve is to provide an information processing device and an information processing method that allow a user to confirm, without performing automatic driving, that there is a high possibility that automatic driving will not be possible in an area where teacher driving has been performed. [Means for solving the problem]

[0008] The information processing device disclosed herein is an information processing device that performs automatic driving based on driving route data indicating a driving route when a user manually drives a vehicle in a teacher driving mode, and includes an extraction unit that receives image information that is information obtained by photographing a periphery of a moving object during the teacher driving mode, and extracts information to be used for self-position estimation obtained from the received image information, and an output control unit that, when an amount of information to be used for self-position estimation extracted by the extraction unit on the driving route is insufficient for a predetermined value, outputs to an output unit instruction information that is different from an instruction information when the amount of information to be used for the self-position estimation is not insufficient for the predetermined value, and a start position of the teacher driving mode is a start position of the automatic driving mode. ,before When the amount of information used for the self-position estimation is not insufficient relative to the specified value, the output control unit notifies the user of the successful completion of the teacher driving, and the instruction information is output before the automatic driving based on the driving route data is performed. Effect of the Invention

[0009] According to the information processing device and information processing method disclosed herein, the user can confirm, without performing autonomous driving, that there is a high possibility that autonomous driving will not be possible in a location where teacher driving has been performed. [Brief description of the drawings]

[0010] [Figure 1] FIG. 1 is a diagram illustrating an example of a moving object according to the first embodiment. [Diagram 2] FIG. 2 is a schematic diagram showing an example of an installation position of an image capturing device according to the first embodiment. [Diagram 3] FIG. 3 is a diagram illustrating an example of a hardware configuration of the information processing device according to the first embodiment. [Figure 4] FIG. 4 is a block diagram illustrating an example of a functional configuration of a moving object according to the first embodiment. [Diagram 5] FIG. 5 is a schematic diagram showing an example of an application scene of a moving object according to the first embodiment. [Figure 6] FIG. 6 is a schematic diagram showing an example of an instruction image according to the first embodiment. [Figure 7] FIG. 7 is a flowchart illustrating an example of an information processing procedure for teacher driving according to the first embodiment. [Figure 8] FIG. 8 is a flowchart illustrating an example of an information processing procedure for automatic driving according to the first embodiment. [Figure 9] FIG. 9 is a flowchart illustrating an example of an information processing procedure for teacher driving according to the second modification. [Figure 10] FIG. 10 is a flowchart illustrating an example of an information processing procedure for automatic driving according to the second modification. [Figure 11] FIG. 11 is a block diagram illustrating an example of a functional configuration of a moving object according to the second embodiment. [Figure 12] FIG. 12 is a flowchart illustrating an example of an information processing procedure for teacher driving according to the second embodiment. [Figure 13] FIG. 13 is a flowchart illustrating an example of an information processing procedure for automatic driving according to the second embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] <Embodiment 1> Hereinafter, embodiments of an information processing device, an information processing method, and a mobile object according to the present disclosure will be described with reference to the accompanying drawings.

[0012] FIG. 1 is a diagram showing an example of a moving object 10 according to the present embodiment.

[0013] The moving object 10 includes an information processing device 20, an output unit 10A, an input unit 10B, an internal sensor 10C, an image capturing device 10D, a drive control unit 10F, and a drive unit 10G.

[0014] The information processing device 20 is, for example, a dedicated or general-purpose computer. In the present embodiment, a form in which the information processing device 20 is mounted on a moving object 10 will be described as an example.

[0015] The moving body 10 is an object that can move. In this embodiment, the moving body 10 is an object that a user can ride on. The moving body 10 is, for example, a vehicle. The vehicle is a two-wheeled vehicle, a three-wheeled vehicle, a four-wheeled vehicle, or the like. The moving body 10 is, for example, a moving body that moves through a driving operation by a person, or a moving body that can move automatically (autonomous movement) without a driving operation by a person. In this embodiment, a case where the moving body 10 is a vehicle will be described as an example.

[0016] The output unit 10A outputs information. In the present embodiment, the output unit 10A outputs information such as instruction information generated by the information processing device 20. The instruction information will be described in detail later.

[0017] The output unit 10A has a display function for displaying information. The output unit 10A may further have a communication function for transmitting information to an external device, a sound output function for outputting sound, a function for turning on or blinking light, and the like. For example, the output unit 10A includes a display unit 10K and at least one of a communication unit 10H, a speaker 10I, and an illumination unit 10J. In this embodiment, a case in which the output unit 10A includes the communication unit 10H, the speaker 10I, the illumination unit 10J, and the display unit 10K will be described as an example.

[0018] The communication unit 10H transmits information to another device. For example, the communication unit 10H transmits information to another device via a known communication line. The speaker 10I outputs sound. The illumination unit 10J is a light that turns on or blinks light. The display unit 10K displays information. The display unit 10K is, for example, a known organic EL (Electro Luminescence) display, a liquid crystal display, or a projection device.

[0019] The output unit 10A may be installed at any position where a user riding on the moving object 10 can confirm the information output from the output unit 10A.

[0020] For example, the moving body 10 is provided with a plurality of display units (hereinafter collectively referred to as "display unit 10K"). These plurality of display units 10K are arranged at different positions in the vehicle interior of the moving body 10. The number of display units 10K provided on the moving body 10 may be one, and is not limited to multiple. In this embodiment, a form in which the moving body 10 is provided with one display unit 10K will be described as an example.

[0021] In detail, the orientation of the display surface of the display unit 10K is adjusted in advance so that the user U riding on the moving object 10 can view the display surface.

[0022] Returning to FIG. 1, the explanation will be continued. The input unit 10B accepts input of instructions or information from the user U. The input unit 10B is, for example, at least one of an instruction input device that accepts input by operation input of the user U and a microphone that accepts voice input. The instruction input device is, for example, a button, a pointing device such as a mouse or a trackball, or a keyboard. The instruction input device may be an input function of a touch panel that is provided integrally with the display unit 10K.

[0023] The internal sensor 10C is a sensor that observes information about the moving body 10 itself. The internal sensor 10C detects the position of the moving body 10, the speed of the moving body 10, the acceleration of the moving body 10, or the like.

[0024] The internal sensor 10C is, for example, an inertial measurement unit (IMU), a speed sensor, or a global positioning system (GPS).

[0025] The image capturing device 10D is a sensor that observes the periphery of the moving body 10. The image capturing device 10D may be mounted on the moving body 10 or may be mounted outside the moving body 10. The outside of the moving body 10 refers to, for example, another moving body or an external device.

[0026] The surroundings of the moving object 10 is an area within a predetermined range from the moving object 10. This range is an observable range of the image capturing device 10D. This range may be set in advance.

[0027] The image capturing device 10D observes the surroundings of the moving body 10 and acquires surrounding information. The surrounding information includes at least one of an image of the surroundings of the moving body 10 and information indicating the distance and direction between the moving body 10 and objects in the surroundings of the moving body 10.

[0028] The photographing device 10D obtains photographed image data (hereinafter referred to as a photographed image) by photographing. This photographing device is a digital camera, a stereo camera, etc. The photographed image is digital image data in which a pixel value is defined for each pixel.

[0029] In the present embodiment, a case will be described as an example in which the surrounding information acquired by the image capturing device 10D is a captured image of the surroundings of the moving object 10. Hereinafter, the captured image of the surroundings of the moving object 10 will be described as a surrounding image.

[0030] The installation position and angle of view of the image capturing device 10D are adjusted in advance so as to be able to capture images of the surroundings of the moving object 10. In this embodiment, the moving object 10 is equipped with a plurality of image capturing devices 10D with different image capturing directions.

[0031] 2 is a schematic diagram showing an example of the installation positions of the image capturing devices 10D. For example, the moving object 10 includes four image capturing devices 10D. Note that the number of image capturing devices 10D provided on the moving object 10 is not limited to four.

[0032] In addition, the installation position and number of the photographing devices 10D need only be adjusted so that photographic images can be acquired in a horizontal plane in a direction substantially covering the entire area (e.g., 360°) centered on the moving body 10, and are not limited to the installation position and number shown in FIG. 2.

[0033] 1, the description will continue. The driving unit 10G is a driving device mounted on the moving body 10. The driving unit 10G is, for example, an engine, a motor, or wheels.

[0034] The drive control unit 10F controls the drive unit 10G to automatically drive the moving body 10. The drive control unit 10F controls the drive unit 10G based on information obtained from the internal sensor 10C or the image capture device 10D, or information received from the information processing device 20. The acceleration amount, braking amount, steering angle, and the like of the moving body 10 are controlled by the control of the drive control unit 10F. For example, the drive control unit 10F controls the moving body 10 to enter a route indicated by the information received from the information processing device 20 and to stop or run.

[0035] Next, a description will be given of a hardware configuration of the information processing device 20. FIG.

[0036] The information processing device 20 has a hardware configuration utilizing a conventional computer, in which a CPU (Central Processing Unit) 11A, a ROM (Read Only Memory) 11B, a RAM (Random Access Memory) 11C, an I / F 11D, etc. are interconnected via a bus 11E.

[0037] The CPU 11A is a calculation device that controls the information processing device 20 of this embodiment. The ROM 11B stores programs and the like that realize the processing by the CPU 11A. The RAM 11C stores data necessary for the processing by the CPU 11A. The I / F 11D is an interface for transmitting and receiving data.

[0038] The program for executing information processing executed by the information processing device 20 of this embodiment is provided by being pre-installed in the ROM 11B, etc. The program executed by the information processing device 20 of this embodiment may be provided by being stored in a computer-readable storage medium (e.g., a flash memory) in a format that can be installed in the information processing device 20 or in a format that can be executed.

[0039] Next, a description will be given of the functional configuration of the moving object 10. FIG 4 is a block diagram showing an example of the functional configuration of the moving object 10.

[0040] The moving object 10 includes an information processing device 20, an output unit 10A, an input unit 10B, an internal sensor 10C, an image capturing device 10D, a drive control unit 10F, and a drive unit 10G.

[0041] The information processing device 20, the output unit 10A, the input unit 10B, the internal sensor 10C, the image capturing device 10D, and the drive control unit 10F are connected to each other via a bus 10L so as to be able to exchange data or signals. The drive control unit 10F is connected to the drive unit 10G so as to be able to exchange data or signals.

[0042] The information processing device 20 has a storage unit 20B and a processing unit 20A. The processing unit 20A and the storage unit 20B are connected to each other via a bus 10L so as to be able to exchange data or signals. The output unit 10A, the input unit 10B, the internal sensor 10C, the image capture device 10D, and the drive control unit 10F are connected to the processing unit 20A so as to be able to exchange data or signals via the bus 10L.

[0043] At least one of the storage unit 20B, the output unit 10A (the communication unit 10H, the speaker 10I, the illumination unit 10J, the display unit 10K), the input unit 10B, the internal sensor 10C, the photographing device 10D, and the drive control unit 10F may be connected to the processing unit 20A by wire or wirelessly. At least one of the storage unit 20B, the output unit 10A (the communication unit 10H, the speaker 10I, the illumination unit 10J, the display unit 10K), the input unit 10B, the internal sensor 10C, the photographing device 10D, and the drive control unit 10F may be connected to the processing unit 20A via a network.

[0044] The storage unit 20B stores data. The storage unit 20B is, for example, a semiconductor memory element such as a random access memory (RAM), a flash memory, a hard disk, an optical disk, or the like. The storage unit 20B may be a storage device provided outside the information processing device 20. The storage unit 20B may also store or temporarily store programs or information downloaded via a local area network (LAN) or the Internet. The storage unit 20B may also be composed of multiple storage media.

[0045] The mobile object travels in one of a plurality of travel modes in response to a user's instruction. The travel modes include a teacher travel mode in which the user performs teacher travel to create map data of the surroundings of the mobile object, and an automatic travel mode in which the mobile object is automatically traveled in the place where the teacher travel was performed.

[0046] The processing unit 20A includes a teacher driving processing unit 20A1 that performs processing in the teacher driving mode, and an automatic driving processing unit 20A2 that performs processing in the automatic driving mode. The teacher driving processing unit 20A1 is the processing unit 20A that executes the teacher driving mode. The automatic driving processing unit 20A2 is the processing unit 20A that executes the automatic driving mode.

[0047] The teacher running processing unit 20A1 includes an acquisition unit 20C, an extraction unit 20D, a creation unit 20E, a determination unit 20F, a specification unit 20G, an output control unit 20H, and a reception unit 20I.

[0048] The acquisition unit 20C, the extraction unit 20D, the creation unit 20E, the determination unit 20F, the identification unit 20G, the output control unit 20H, and the reception unit 20I are realized, for example, by one or more processors. For example, each of the above units may be realized by having a processor such as a CPU execute a program, that is, by software. Each of the above units may be realized by a processor such as a dedicated IC (Integrated Circuit), that is, by hardware. Each of the above units may be realized by using both software and hardware. When multiple processors are used, each of the multiple processors may realize one of the multiple units, or may realize two or more of the multiple units.

[0049] The processor realizes each of the above-mentioned multiple units by reading and executing the program stored in the storage unit 20B. Note that instead of storing the program in the storage unit 20B, the program may be directly built into the circuit of the processor. In this case, the processor realizes each of the above-mentioned multiple units by reading and executing the program built into the circuit.

[0050] The acquisition unit 20C acquires surrounding information from the photographing device 10D. As described above, in the present embodiment, the photographing device 10D acquires, as surrounding information, a surrounding image 40 that is a photographed image of the surroundings of the moving object 10. For this reason, the acquisition unit 20C acquires the surrounding image 40 of the moving object 10 from the photographing device 10D.

[0051] The photographing device 10D obtains the surrounding image 40 at each predetermined timing in a time series. Then, the photographing device 10D outputs the obtained surrounding image 40 to the teacher running processing unit 20A1 every time the photographing device 10D obtains the surrounding image 40. Therefore, the obtaining unit 20C of the teacher running processing unit 20A1 obtains the surrounding images 40 sequentially from the photographing device 10D.

[0052] The extraction unit 20D extracts feature points around the route traveled by the moving object 10 by analyzing the surrounding image 40 acquired by the acquisition unit 20C from the imaging device 10D.

[0053] The extraction unit 20D may extract feature points using any known method, and the extraction method is not limited.

[0054] The extraction unit 20D assigns identification information to the extracted feature points, and stores the identification information in association with information indicating the position and range of the feature points in real space in the storage unit 20B. The extraction unit 20D may extract information indicating the position and range of the feature points in real space using position information of the moving body 10 acquired from the internal sensor 10C, etc.

[0055] The creation unit 20E creates map data having the position information of the feature points extracted by the extraction unit 20D.

[0056] The map data created by the creation unit 20E is information used for self-location estimation, and stores, for each of a plurality of feature points in a real scene previously obtained over a wide area (including the area around the mobile body 10), a three-dimensional position in real space and a feature amount, which is data indicating the features of the feature point extracted from image information captured when creating the map data, in association with each other. The feature points stored as the map data are, for example, parts (e.g., corners) from which a characteristic image pattern is obtained from image information of objects (e.g., buildings, signs, or billboards) that can serve as landmarks in the real scene. Furthermore, pre-stored landmarks such as position markers may be used as position information in the real scene. Note that the plurality of feature points in the map data are stored so as to be identifiable, for example, by identification numbers.

[0057] The three-dimensional positions of feature points in real space stored in the map data are expressed in a three-dimensional Cartesian coordinate system (X, Y, Z) based on latitude, longitude, and height, for example. The three-dimensional positions of feature points in real space are stored by, for example, measurements based on the principle of triangulation from camera images taken at multiple positions, or measurements using LIDAR (Light Detection and Ranging) or a stereo camera.

[0058] The feature amounts of the feature points stored in the map data may be SIFT (Scale Invariant Feature Transform) features or SURF (Speed ​​Up Robust Features) features, in addition to the brightness and density of the image information. The feature amount data of the feature points stored in the map data may be of feature points at the same three-dimensional position, or may be stored separately for each shooting position or shooting direction of the shooting device when the feature points were shot. The feature amount data of the feature points stored in the map data may be stored in association with image information of an object having the feature points.

[0059] The map data creation method by the creation unit 20E may be a known method, and the map data creation method is not limited. For example, the creation unit 20E creates position information indicating the positions of the feature points extracted by the extraction unit 20D, thereby creating map data having the position information of the feature points.

[0060] FIG. 5 is a schematic diagram showing an example of an application scene of the moving body 10 of this embodiment. For example, assume a scene in which a user U drives the moving body 10 from point A to point B as a teacher, in order to learn a route for automatically driving the moving body 10 from point A to point B along an arrow. Point A is, for example, a position where the user U gets off the moving body 10, and is also a starting position for automatically driving the moving body 10 without a driver. Point B is, for example, a parking target position of the moving body 10. A driver U1 is an example of a user U who gets on the moving body 10.

[0061] In this case, the imaging device 10D provided on the moving body 10 captures an image of the periphery of the moving body 10, and the acquisition unit 20C acquires a peripheral image 40 of the periphery. For this purpose, the extraction unit 20D acquires the peripheral image 40 of the periphery of the moving body 10 from the acquisition unit 20C. Then, the extraction unit 20D analyzes the peripheral image 40 to extract feature points included in the peripheral image 40. Furthermore, the creation unit 20E creates map data having position information of the feature points extracted by the extraction unit 20D. The creation unit 20E stores the created map data in the map data storage unit 20B3.

[0062] Returning to FIG. 4, the explanation will be continued. The determination unit 20F extracts a part of the route traveled by the moving body 10 according to a predetermined condition, and determines whether or not the number of extracted feature points in each of the extracted routes (hereinafter referred to as a "partial route") is insufficient for a predetermined number required for self-location estimation. If the number of extracted feature points is greater than the predetermined number, it is considered that the location is likely to be one where self-location estimation is successful. The predetermined condition is, for example, that the vehicle moves a predetermined distance (e.g., 20 cm), or that a predetermined number or more of the same feature points are continuously extracted in temporally consecutive images acquired by the image capture device 10D, etc.

[0063] The determination unit 20F repeats the determination for each partial path, and identifies a location where the number of feature points is insufficient with respect to a predetermined number.

[0064] The identification unit 20G performs self-location estimation to identify the location of the moving body 10 when performing automatic driving. The identification unit 20G compares the map data created by the creation unit 20E with the image captured by the imaging device 10D mounted on the moving body 10. This allows the identification unit 20G to identify where the moving body 10 is located in the created map data.

[0065] Next, the output control unit 20H will be described. The output control unit 20H outputs information to the output unit 10A. In the present embodiment, the output control unit 20H outputs instruction information to the output unit 10A.

[0066] The instruction information indicates information different from information when the number of feature points is not insufficient relative to the predetermined number when the determination unit 20F determines that the number of feature points is insufficient relative to the predetermined number. The instruction information may be information indicating that the number of feature points is insufficient relative to the predetermined number when the determination unit 20F determines that the number of feature points is insufficient relative to the predetermined number. The instruction information may also be information indicating a location where the number of feature points is insufficient relative to the predetermined number when the determination unit 20F determines that the number of feature points is insufficient relative to the predetermined number. In addition, the instruction information may be information notifying a location where an object containing a large amount of features (hereinafter referred to as a feature object) or a position marker that is a landmark stored in the teacher running processing unit 20A1 in advance is to be installed in order to add position information to the map data when the determination unit 20F determines that the number of feature points is insufficient relative to the predetermined number. The location where the feature object or the position marker is installed is a location where the shortage of the number of feature points is likely to be resolved when teacher running is performed again with the feature object or the position marker installed. In other words, the location is an effective location for resolving the shortage of the number of feature points.

[0067] In this embodiment, the instruction information is described as an example of a form in which, when the judgment unit 20F judges that the number of feature points is insufficient compared to a predetermined number, a location where a feature or a position marker is to be placed in order to add position information to the map data.

[0068] The instruction information may be any of an instruction image represented by an image, an instruction voice represented by a voice, and an instruction character represented by a text, or a combination of these. Note that when the instruction information is an image, the instruction information will be referred to as an instruction image in the following description.

[0069] Fig. 6 is a schematic diagram showing an example of the instruction image 52. As shown in Fig. 6, the instruction image 52 is an image that notifies a location where a feature or a position marker should be placed in order to add position information to map data in the vicinity of the moving object 10.

[0070] The output control unit 20H generates an instruction image 52 using the map data created by the creation unit 20E, information obtained by the determination unit 20F as to whether the number of feature points for each partial route is insufficient compared to a specified number, and the position information of the moving body 10 estimated by the identification unit 20G.

[0071] For example, when the determination unit 20F determines that the number of feature points is insufficient relative to a predetermined number, the output control unit 20H generates an instruction image 52 indicating the location where the number of feature points is insufficient relative to the predetermined number. In detail, when the determination unit 20F determines that the number of feature points is insufficient relative to the predetermined number, the output control unit 20H generates an instruction image 52 indicating the location where the number of feature points is insufficient relative to the predetermined number as a location where a feature object or a position marker should be placed.

[0072] Hereinafter, a partial route in which the number of feature points is insufficient for the predetermined number required to estimate the self-position of the moving body 10 is referred to as an automatic driving unsuitable route RP. As shown in FIG. 6, it is preferable that the output control unit 20H generates an instruction image 52 that highlights the automatic driving unsuitable route RP. For example, the automatic driving unsuitable route RP is a frame image that is an image showing the automatic driving unsuitable route RP, or a color image that is an image showing the color of the automatic driving unsuitable route RP in a color that attracts the attention of the user U. The color that attracts attention is, for example, yellow or red, but is not limited to these colors. By showing the user using an image as instruction information, the user can know the specific location of the automatic driving unsuitable route RP.

[0073] Then, in response to an instruction to end the teacher running, the output control unit 20H outputs the generated instruction image 52 to the output unit 10A.

[0074] In this embodiment, the output control unit 20H displays an instruction image 52 on one or more of the multiple display units 10K provided on the moving object 10.

[0075] When using an instruction voice represented by a voice as the instruction information, the output control unit 20H may output the instruction voice from the speaker 10I. For example, the output control unit 20H may output an instruction voice such as "There is a place where automatic driving is considered difficult. The teacher driving will end" from the speaker 10I. In addition, an instruction image may also be displayed.

[0076] Returning to Fig. 4, the description will be continued. Next, the reception unit 20I will be described. The reception unit 20I receives an input of an instruction to retry the teacher run in the instruction image 52 when the determination unit 20F determines that the number of feature points is insufficient for the predetermined number.

[0077] For example, assume that the instruction image 52 shown in Fig. 6 is displayed on the display unit 10K. When the determination unit 20F determines that the number of feature points in the instruction image 52 is insufficient for the predetermined number, the user U places a feature object or a position marker in the area where the number of feature points is insufficient for the predetermined number, and then inputs an instruction to retry the teacher run.

[0078] The user U, for example, operates the input unit 10B to input an instruction to retry the teacher run in the instruction image 52. In particular, if the input unit 10B is a touch panel, the user U inputs an instruction to retry the teacher run by touching a retry button displayed on the touch panel. The reception unit 20I receives the information instructed by the user U from the input unit 10B, thereby receiving the input of the instruction to retry the teacher run.

[0079] The reception unit 20I may receive an input of an instruction to retry the teacher run by analyzing a gesture of the user U or a voice uttered by the user U.

[0080] Also, for example, the user U emits a voice indicating that he / she desires to perform teacher running. In this case, the reception unit 20I may receive an input of an instruction to retry teacher running by analyzing the voice data of the voice collected by the microphone serving as the input unit 10B using a known voice analysis method.

[0081] In response to the instruction to retry the teacher-driven driving, the reception unit 20I outputs information indicating an instruction to retry the teacher-driven driving to the automatic driving processing unit 20A2.

[0082] The automatic driving processing unit 20A2 estimates the vehicle's own position based on the map data stored in the storage unit 20B, and performs automatic driving.

[0083] The storage unit 20B includes a teacher driving program 20B1, an automatic driving program 20B2, and a map data storage unit 20B3.

[0084] When the teacher driving processing unit 20A1 reads out the teacher driving program 20B1, the teacher driving mode is processed based on the teacher driving program 20B1. When the automatic driving processing unit 20A2 reads out the automatic driving program 20B2, the automatic driving mode is processed based on the automatic driving program 20B2. The map data storage unit 20B3 stores the map data created by the creation unit 20E.

[0085] Next, a description will be given of an example of a procedure for information processing of teacher traveling in the first embodiment executed by the moving body 10. Fig. 7 is a flowchart showing an example of information processing procedure of teacher traveling in the first embodiment executed by the moving body 10.

[0086] First, the acquisition unit 20C acquires the surrounding image 40 from the image capturing device 10D (step S101). Next, the extraction unit 20D extracts feature points from the surrounding image 40 acquired in step S101 (step S103). That is, information used for self-position estimation is extracted from the surrounding image 40 acquired in step S101.

[0087] In step S105, the creation unit 20E adds the map data having the position information of the characteristic points extracted in step S103 to the map data storage unit 20B3 (step S105).

[0088] In step S107, the determination unit 20F extracts partial routes from the created map data according to a predetermined condition (step S107). Note that the predetermined condition is, for example, that a predetermined number or more of identical feature points are continuously extracted in temporally consecutive images captured by the image capture device 10D.

[0089] Next, the determination unit 20F stores, for each partial path, whether or not the number of feature points is insufficient with respect to a predetermined number, that is, whether or not the number is less than the predetermined number (step S109).

[0090] In step S111, the input unit 10B judges whether or not the teacher driving has been ended by the user (step S111). If the teacher driving has been ended (step S111: Yes), the process proceeds to step S113. If the teacher driving has not been ended (step S111: No), the process returns to step S101. Note that the method of recognizing that the teacher driving end point has been reached may be a known method, and the method is not limited thereto. For example, the arrival of the teacher driving end point is recognized by receiving an instruction to end the teacher driving from the user via the input unit 10B. Alternatively, the arrival of the teacher driving end point may be recognized by the shift lever being in the parking range.

[0091] Next, the determination unit 20F determines whether the number of feature points is insufficient for each partial route relative to a predetermined number required to estimate the self-position (step S113). That is, it determines whether the amount of information used for self-position estimation is insufficient for a predetermined value. If the number of feature points is not insufficient for the predetermined number for any of the partial routes among the routes traveled by teacher, that is, if it is greater than the predetermined number (step S113: No), proceed to step S115. In step S115, the user is notified of the normal end of teacher travel, and this routine is terminated. When notifying the user of the end of teacher travel, it may be notified of the end of teacher travel together with the fact that automatic travel in automatic travel mode is possible. If the number of feature points is insufficient for the predetermined number for any one or more of the routes extracted among the routes traveled by teacher, that is, if it is less than the predetermined number (step S113: Yes), proceed to step S117.

[0092] When the number of feature points of one or more of the extracted routes is insufficient for the predetermined number, that is, when the number is less than the predetermined number (step S113: Yes), the output control unit 20H outputs instruction information to the output unit 10A (step S117). The instruction information is, for example, information for the display unit 10K to notify that there is a place in the instruction image 52 where automatic driving is highly likely to be impossible. This allows the user to know that there is a place where automatic driving is highly likely to be impossible after this teacher driving. In other words, the user can know that automatic driving is highly likely to be impossible even without attempting automatic driving. Note that this instruction information may be notified by voice. When the instruction information is notified by voice, the driver U1 can know that automatic driving is highly likely to be impossible without looking at the display unit 10K. This allows the driver U1 to know that automatic driving is highly likely to be impossible when looking at a place other than the display unit 10K.

[0093] Next, the input unit 10B judges whether an instruction to retry the teacher run has been input (step S119). For example, if the display unit 10K is a touch panel, the input unit 10B is a retry button displayed on the touch panel, and the user U inputs an instruction to retry the teacher run by touching the retry button on the display unit 10K. If an instruction to retry the teacher run has been input (step S119: Yes), the process returns to step S101. If an instruction to retry the teacher run has not been input (step S119: No), the process proceeds to step S121.

[0094] In step S121, it is determined whether a predetermined time has elapsed since the instruction information was notified (step S117). The predetermined time is, for example, five minutes. If it is determined that the predetermined time has elapsed (step S121: Yes), the process proceeds to step S123. In step S123, the user is notified of an abnormal end, indicating that the teacher run did not end normally, and this routine ends. When notifying the user of the end of the teacher run, the method of notification may be different between step S115 and step S123. If it is determined that the predetermined time has not elapsed (step S121: No), the process returns to step S119.

[0095] Next, a description will be given of an example of an information processing procedure for autonomous driving in the first embodiment executed by the moving body 10. Fig. 8 is a flowchart showing an example of an information processing procedure for autonomous driving in the first embodiment executed by the moving body 10.

[0096] First, the automatic driving processing unit 20A2 performs an initial position estimation based on the map data stored in the map data storage unit 20B3 and the captured image (step S200). The estimated initial position is set as the self-position (step S201). Next, the automatic driving processing unit 20A2 performs automatic driving from the self-position estimated in step S201 (step S203). In step S205, the automatic driving processing unit 20A2 judges whether the moving body 10 has arrived at the destination (step S205). If it is judged that the moving body 10 has not arrived at the destination (step S205: No), the process returns to the above step S201. If it is judged that the moving body 10 has not arrived at the destination (step S205: No), the process from step S201 to step S205 is repeated. In the repeated step S201, the self-position estimation is performed based on the map data stored in the map data storage unit 20B3 and the captured image. Moreover, if it is determined in step S205 that the moving object 10 has arrived at the destination (step S205: Yes), this routine ends.

[0097] In this way, the user can know that there is a high possibility that the vehicle will not be able to travel autonomously, even if the vehicle does not actually travel autonomously, thereby improving user convenience.

[0098] <Variation 1> In the supervised driving in FIG. 7, if the number of feature points is insufficient for the predetermined number, a notification is given in step S117 that there is a location where automatic driving is highly likely not possible. However, since the user cannot confirm the location where the number of feature points is insufficient for the predetermined number, there is a possibility that the user cannot smoothly supplement the number of feature points and end the supervised driving after making the vehicle in an automatic driving state. Therefore, in step S117, an instruction image as shown in FIG. 6 may be used to notify the location where the number of feature points is insufficient for the predetermined number as a location where a feature should be installed. In this way, the user can know the location where the feature should be installed. If the user can know the location where the feature should be installed, the user can install the feature in that location to prepare a surrounding environment where automatic driving is possible even in a location where automatic driving is not possible during supervised driving.

[0099] <Variation 2> In the supervised driving in the modified example 1, if the number of feature points is insufficient, a partial route where the number of feature points is less than a predetermined number is notified as a place where a feature should be placed, but a position marker may be notified as a place where a feature should be placed instead of a feature. Fig. 9 is a flowchart showing an example of the procedure of information processing of the supervised driving in the modified example 2 executed by the moving body 10. The procedure of information processing of the supervised driving in the modified example 2 shown in Fig. 9 will be described below.

[0100] First, the determination unit 20F extracts a partial path according to a predetermined condition (step S301). The predetermined condition is, for example, that a predetermined number or more of feature points that are considered to be the same are continuously extracted in temporally consecutive images acquired by the image capture device 10D. Next, the acquisition unit 20C acquires a peripheral image 40 of the partial path extracted in step S301 (step S303).

[0101] In step S305, the extraction unit 20D determines whether or not a position marker has been extracted from the peripheral image 40 of the partial route acquired in step S303 (step S305). That is, it determines whether or not information used for self-position estimation has been extracted from the peripheral image 40 of the partial route acquired in step S303. If a position marker has not been extracted from the peripheral image 40 of the partial route (step S305: No), the process proceeds to step S311. If a position marker has been extracted from the peripheral image 40 of the partial route (step S305: Yes), the process proceeds to step S307.

[0102] In step S307, the creation unit 20E stores the map data having the position information of the position marker extracted in step S307 in the map data storage unit 20B3 (step S307). If it is the second or subsequent time, the position of the currently extracted position marker is added to the stored map data.

[0103] Next, the determination unit 20F determines whether the number of position markers on the partial route is insufficient for the predetermined number required to estimate the self-location (step S309). That is, it determines whether the amount of information used for self-location estimation is insufficient for a predetermined value. If the number of position markers on the partial route is not insufficient for the predetermined number required to estimate the self-location, that is, if it is greater than the predetermined number (step S309: No), proceed to step S317. If the number of position markers on the partial route is insufficient for the predetermined number required to estimate the self-location, that is, if it is less than the predetermined number (step S309: Yes), proceed to step S311.

[0104] In step S311, the extraction unit 20D extracts feature points from the peripheral image 40 of the partial route acquired in step S303 (step S311). Next, the creation unit 20E adds map data having position information of the feature points extracted in step S311 to the map data storage unit 20B3 (step S313). Next, in step S315, the determination unit 20F stores whether or not the number of feature points of the partial route is insufficient for a predetermined number, that is, whether or not the number is less than the predetermined number (step S315).

[0105] In step S317, it is determined whether the processes from step S301 to step S315 have been performed on the peripheral images 40 of all partial routes up to the teacher travel end point (step S317). The method for recognizing that the teacher travel end point has been reached may be a known method, and the method is not limited. For example, the arrival of the teacher travel end point is recognized by receiving an instruction to end the teacher travel from the user via the input unit 10B. Alternatively, the arrival of the teacher travel end point may be recognized by the shift lever being in the parking range.

[0106] If it is determined that the peripheral images 40 have been processed for all partial routes up to the teacher travel end point (step S317: Yes), proceed to step S319. If it is determined that the peripheral images 40 have not been processed for all partial routes up to the teacher travel end point (step S317: No), return to step S301.

[0107] In step S319, it is determined whether or not there is a partial route in which the number of feature points is less than the predetermined number among all partial routes in which the number of position markers is less than the predetermined number (step S319). If there is no partial route in which the number of feature points is less than the predetermined number among all partial routes in which the number of position markers is less than the predetermined number (step S319: No), proceed to step S321. In step S321, the user is notified of the normal end of the teacher driving, and this routine is terminated. When notifying the user of the end of the teacher driving, the end of the teacher driving may be notified together with the fact that automatic driving in the automatic driving mode is possible. If there is a partial route in which the number of feature points is less than the predetermined number among all partial routes in which the number of position markers is less than the predetermined number, that is, if the route traveled in the teacher driving includes a partial route in which both the number of position markers and the number of feature points are less than the predetermined number (step S319: Yes), proceed to step S323.

[0108] In step S323, the output control unit 20H outputs instruction information to the output unit 10A (step S323). The instruction information is, for example, information for the display unit 10K to notify the installation location of a feature object or a position marker.

[0109] Next, the input unit 10B judges whether an instruction to retry the teacher run has been input (step S325). For example, if the display unit 10K is a touch panel, the input unit 10B is a retry button displayed on the touch panel, and the user U inputs an instruction to retry the teacher run by touching the retry button on the display unit 10K. If an instruction to retry the teacher run has been input (step S325: Yes), the process returns to step S301. If an instruction to retry the teacher run has not been input (step S325: No), the process proceeds to step S327.

[0110] In step S327, it is determined whether a predetermined time has elapsed since the instruction information was notified (step S323). The predetermined time is, for example, five minutes. If it is determined that the predetermined time has elapsed (step S327: Yes), the process proceeds to step S329. In step S329, the user is notified of an abnormal end, indicating that the teacher run did not end normally, and this routine ends. When notifying the user of the end of the teacher run, the method of notification may be different between step S321 and step S329. If it is determined that the predetermined time has not elapsed (step S329: No), the process returns to step S325.

[0111] Next, a description will be given of an example of an information processing procedure for the autonomous driving in the modified example 2 executed by the moving body 10. Fig. 10 is a flowchart showing an example of an information processing procedure for the autonomous driving in the modified example 2 executed by the moving body 10.

[0112] First, the automatic driving processing unit 20A2 performs an initial position estimation (step S401). Next, in step S403, the automatic driving processing unit 20A2 judges whether or not there is a position marker around the moving body 10 (step S403). If it is judged that there is a position marker around the moving body 10 (step S403: Yes), the process proceeds to step S405. In step S405, the automatic driving processing unit 20A2 identifies the position of the position marker from the map data stored in the map data storage unit 20B3 (step S405). Next, in step S407, the automatic driving processing unit 20A2 estimates its own position from the position of the position marker identified in step S405. (step S407).

[0113] In step S403, if it is determined that there is no position marker around the moving body 10 (step S403: No), the process proceeds to step S409. In step S409, the automatic driving processing unit 20A2 identifies the positions of the characteristic points from the map data stored in the map data storage unit 20B3 (step S409). Next, in step S411, the automatic driving processing unit 20A2 estimates its own position from the positions of the characteristic points identified in step S409 (step S411). Next, the automatic driving processing unit 20A2 performs automatic driving from the own position estimated in step S407 or step S411 (step S412).

[0114] In step S413, the automatic driving processing unit 20A2 judges whether or not the destination has been reached (step S413). If it is judged that the destination has not been reached (step S413: No), the process returns to step S401. If it is judged that the destination has been reached (step S413: Yes), the process ends this routine.

[0115] As described above, the information processing device 20 of this embodiment includes an extraction unit 20D, a determination unit 20F, and an output control unit 20H. The extraction unit 20D receives image information that is information obtained by photographing the periphery of a moving object, and extracts feature points from the received image information. The determination unit 20F determines whether or not the number of feature points extracted by the extraction unit 20D is insufficient for a predetermined number. When the determination unit 20F determines that the number of feature points is insufficient for the predetermined number, the output control unit 20H outputs instruction information (instruction image 52) that is different from when the number of feature points is not insufficient for the predetermined number.

[0116] Here, in the conventional technology, when a vehicle is driven in a supervised driving mode to create map data of a location where the environmental characteristics around the vehicle are insufficient, such as a location surrounded by objects with no unevenness or color changes, there is a high possibility that self-location estimation will not be possible because the number of feature points included in the created map data is insufficient for the predetermined number required to estimate the vehicle's self-location. However, in the conventional technology, the user cannot confirm that there is a high possibility that autonomous driving will not be possible in the location where the supervised driving mode has been performed until autonomous driving is actually performed.

[0117] On the other hand, when the number of feature points around the moving object is insufficient for the predetermined number due to supervised driving, the information processing device 20 of this embodiment outputs an instruction image 52 that is different from the case where the number of feature points is not insufficient for the predetermined number. Therefore, when the determination unit 20F determines that the number of feature points is insufficient for the predetermined number, the information processing device 20 can output an instruction image 52 indicating that the number of feature points is insufficient for the predetermined number. For example, it is possible to output information indicating the location where the number of feature points is insufficient for the predetermined number without the user U actually performing automatic driving.

[0118] Therefore, in the information processing device 20 of this embodiment, the user can confirm that there is a high possibility that automatic driving is not possible in the place where the supervised driving has been performed, without performing automatic driving. Also, since it may be difficult for the user to determine what an object containing many features is, a position marker prepared in advance for supervised driving is provided to the user, and the user can use the position marker to change an environment where automatic driving is highly unlikely to be possible into an environment where automatic driving is possible in a simpler manner.

[0119] <Embodiment 2> In the first embodiment, the amount of information used for self-location estimation is the number of feature points, but in the second embodiment, the information used for self-location estimation is information obtained by a tracking process, which is a process of tracking the same feature points extracted from consecutive image information, and the amount of information used for self-location estimation is the number of feature points tracked as a result of this tracking process. Hereinafter, the second embodiment will be described in detail, but the same components as those in the first embodiment will be denoted by the same reference numerals and description thereof will be omitted.

[0120] Fig. 11 is a block diagram showing an example of the functional configuration of the moving object 10 in the embodiment 2. In the embodiment 2, a matching processing unit 20J, a tracking processing unit 20K, and a creation unit 20E are different from the configurations in Fig. 4 of the embodiment 1.

[0121] The matching processing unit 20J uses the similarity of the feature amounts to determine whether feature points extracted from consecutive images are the same or not (hereinafter, this process is referred to as matching processing). That is, among the images used in the matching processing, it determines whether a feature point a1 included in frame A, which is a past image, and a feature point b1 included in frame B, which is a current image, are the same feature point or not. As a matching method for determining whether feature points between frames are the same or not, any known method may be used, and the matching method is not limited.

[0122] The tracking processing unit 20K performs tracking processing on the feature points matched in the matching processing. The tracking processing unit 20K judges whether or not the distance between the position X of the feature point a1 extracted from frame A estimated in frame B and the position Y of the feature point b1 extracted from frame B is smaller than a predetermined value (hereinafter, this processing is referred to as tracking processing). If the distance between the estimated position X and the extracted position Y is smaller than the predetermined value, the feature point is considered to have been tracked. The predetermined value is, for example, 5 pixels. Note that the tracking method may be any known method, and is not limited to this tracking method. For example, by estimating the amount of movement of the imaging device from the speed and tire angle of the moving body 10, it is possible to estimate the position of the feature point in image B from the position of the feature point in image A. The position of the feature point is, for example, a coordinate on the image. If the number of tracked feature points is greater than a predetermined number, it is considered to be a place where the self-location estimation is likely to be successful.

[0123] The creation unit 20E creates map data having position information of the tracked feature points. Note that the creation unit 20E may not include feature points that were not tracked as the positions of the feature points extracted by the extraction unit 20D.

[0124] FIG. 12 is a flowchart showing an example of a procedure for information processing of teacher driving in the second embodiment executed by the moving object 10.

[0125] First, the acquisition unit 20C acquires the surrounding image 40 from the image capturing device 10D (step S501). Next, the extraction unit 20D extracts feature points from the surrounding image 40 acquired in step S501 (step S503). That is, information used for self-position estimation is extracted from the surrounding image 40 acquired in step S501.

[0126] In step S505, the matching processing unit 20J performs a matching process between the feature points extracted from the current image and the feature points extracted from the past image subsequent to the current image (step S505). Next, the tracking processing unit 20K performs a tracking process on the feature points matched in step S505 (step S507).

[0127] In step S509, the creation unit 20E creates map data having the position information of the feature points tracked in step S507, and stores the map data in the map data storage unit 20B3 (step S509). In the process of step S509 from the second time onwards, the created map data is added to the already stored map data.

[0128] In step S511, the determination unit 20F extracts partial routes from the map data based on a predetermined condition (step S511). The predetermined condition is, for example, that a predetermined number or more of identical feature points are continuously extracted in temporally consecutive images captured by the image capture device 10D.

[0129] Next, for each of the extracted partial paths, the determination unit 20F stores whether or not the number of feature points is insufficient with respect to a predetermined number, that is, whether the number is less than or more than the predetermined number (step S513).

[0130] In step S515, the input unit 10B determines whether or not an instruction to end the teacher running has been input by the user (step S515).

[0131] Next, the determination unit 20F determines whether the number of tracked feature points is insufficient for each partial route relative to a predetermined number required to estimate the self-position (step S517). That is, it determines whether the amount of information used for self-position estimation is insufficient for a predetermined value. If the number of feature points is not insufficient for the predetermined number for any of the partial routes, that is, if it is greater than the predetermined number (step S517: No), proceed to step S519. In step S519, the user is notified of the normal end of the teacher driving, and this routine is terminated. When notifying the user of the end of the teacher driving, it may be notified of the end of the teacher driving together with the fact that automatic driving in the automatic driving mode is possible. If the number of tracked feature points is insufficient for the predetermined number for any one or more of the extracted routes, that is, if it is less than the predetermined number (step S517: Yes), proceed to step S521.

[0132] When the number of tracked feature points of any one or more of the extracted routes is insufficient for a predetermined number, that is, when the number is less than the predetermined number (step S517: Yes), the output control unit 20H outputs instruction information indicating that there is a place where automatic driving is highly likely not possible to the output unit 10A (step S521). The instruction information is, for example, information for the display unit 10K to notify that automatic driving is not possible in the instruction image 52. This allows the user to know that automatic driving is not possible after this teacher driving. In other words, the user can know that automatic driving is not possible without attempting automatic driving. Note that this instruction information may be notified by voice. When the instruction information is notified by voice, the driver U1 can know that automatic driving is not possible without looking at the display unit 10K. This allows the driver U1 to know that automatic driving is not possible when looking at a place other than the display unit 10K.

[0133] Next, the input unit 10B judges whether an instruction to retry the teacher run has been input (step S523). For example, if the display unit 10K is a touch panel, the input unit 10B is a retry button displayed on the touch panel, and the user U inputs an instruction to retry the teacher run by touching the retry button on the display unit 10K. If an instruction to retry the teacher run has been input (step S523: Yes), the process returns to step S501. If an instruction to retry the teacher run has not been input (step S523: No), the process proceeds to step S525.

[0134] In step S525, it is determined whether a predetermined time has elapsed since the instruction information was notified (step S521). The predetermined time is, for example, 5 minutes. If it is determined that the predetermined time has elapsed (step S525: Yes), this routine is terminated. If it is determined that the predetermined time has not elapsed (step S525: No), the process returns to step S523.

[0135] Next, a description will be given of an example of an information processing procedure for autonomous driving in the second embodiment executed by the moving body 10. Fig. 13 is a flowchart showing an example of an information processing procedure for autonomous driving in the second embodiment executed by the moving body 10.

[0136] First, the automatic driving processing unit 20A2 estimates its own position based on the map data stored in the map data storage unit 20B3 and the captured image (step S601). Next, the automatic driving processing unit 20A2 performs automatic driving from the self-position estimated in step S601 (step S603). In step S605, the automatic driving processing unit 20A2 determines whether the moving body 10 has arrived at the destination (step S605). If it is determined that the moving body 10 has not arrived at the destination (step S605: No), the process returns to step S601. If it is determined that the moving body 10 has arrived at the destination (step S605: Yes), the process ends this routine.

[0137] In this way, the user can know that there is a high possibility that the vehicle will not be able to travel autonomously, even if the vehicle does not actually travel autonomously.

[0138] <Other embodiments> In the first modification, a partial route where the number of feature points is less than a predetermined number is notified as a place where a feature should be placed, and in the second modification, a partial route where the number of feature points is less than a predetermined number is notified as a place where a feature should be placed, or a position marker should be placed instead of a feature, but a partial route where the number of feature points is less than a predetermined number may be notified as a place where a feature or a position marker, or both a feature and a position marker should be placed. This increases the degree of freedom in the objects to be placed, and further improves user convenience.

[0139] In the present embodiment, the information processing device 20 is mounted on the moving body 10 as an example. However, the information processing device 20 may be mounted on the outside of the moving body 10. In this case, the information processing device 20 may be configured to be able to communicate with each of the electronic devices such as the internal sensor 10C mounted on the moving body 10 via a network.

[0140] In the above-mentioned embodiment, the program for executing the information processing has a modular configuration including each of the above-mentioned multiple functional units, and as actual hardware, for example, a CPU (processor circuit) reads out and executes the information processing program from a ROM or HDD, whereby each of the above-mentioned multiple functional units is loaded onto a RAM (main memory), and each of the above-mentioned multiple functional units is generated on the RAM (main memory). It is also possible to realize a part or all of each of the above-mentioned multiple functional units using dedicated hardware such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field-Programmable Gate Array).

[0141] Although the embodiment has been described above, the embodiment is presented as an example and is not intended to limit the scope of the present disclosure. The above novel embodiment can be implemented in various other forms, and various omissions, substitutions, and modifications can be made without departing from the gist of the invention. The above embodiment is included in the scope or gist of the present disclosure, and is included in the scope of the invention and its equivalents described in the claims. [Explanation of symbols]

[0142] 10 Mobile 10A output section 10B Input section 10C Internal Sensor 10D Imaging Device 10F Drive control unit 10G Drive Unit 10H Communications Department 10I Speaker 10J lighting section 10K display 10L Bus 11A CPU 11B ROM 11C RAM 11D Interface 11E Bus 20 Information processing device 20A Processing section 20A1 Teacher driving processing unit 20A2 Automatic driving processing unit 20B Storage section 20B1 Teacher Driving Program 20B2 Autonomous Driving Program 20B3 Map data storage unit 20C Acquisition Department 20D extraction section 20E Creation Department 20F Judgment section 20G specific part 20H Output control section 20I Reception 20J Matching processing section 20K tracking processing unit

Claims

1. An information processing device that performs automatic driving based on driving route data indicating a driving route when a user manually drives a vehicle, an extraction unit that receives image information, which is information obtained by photographing the periphery of the moving object during the teacher traveling, and extracts information to be used for self-position estimation obtained from the received image information; an output control unit that outputs, when an amount of information used for the self-position estimation extracted by the extraction unit on the travel route is insufficient with respect to a predetermined value, instruction information to an output unit that is different from instruction information output when an amount of information used for the self-position estimation is not insufficient with respect to the predetermined value; Equipped with the start position of the teacher driving is the start position of the automatic driving, the output control unit notifies the user of a normal end of teacher traveling when an amount of information used for the self-position estimation is not insufficient for the predetermined value; the instruction information is output before the automatic traveling based on the traveling route data is performed. Information processing device.

2. The instruction information is information indicating, when an amount of information used for the self-location estimation is insufficient with respect to the predetermined value, that an amount of information used for the self-location estimation is insufficient with respect to the predetermined value. The information processing device according to claim 1 .

3. The instruction information is information indicating that an amount of information used for the self-location estimation is not insufficient for the predetermined value when the amount of information used for the self-location estimation is not insufficient for the predetermined value. The information processing device according to claim 1 .

4. A creating unit that creates map data from the extracted information used for self-location estimation, The instruction information is information indicating, when an amount of information used for the self-location estimation is insufficient with respect to the predetermined value, a portion of the map data where an amount of information used for the self-location estimation is insufficient with respect to the predetermined value.

3. The information processing device according to claim 1 or 2.

5. The instruction information is information for informing a location where an object including a large amount of information to be used for the self-location estimation is to be installed in correspondence with a location where an amount of information to be used for the self-location estimation is insufficient with respect to the predetermined value, so that the information to be used for the self-location estimation is added to the map data. The information processing device according to claim 4.

6. The instruction information is information for informing a location where a pre-stored landmark such as a position marker should be installed in response to a portion where an amount of information used for the self-location estimation is insufficient with respect to the predetermined value, so that the information used for the self-location estimation is added to the map data.

6. The information processing device according to claim 4 or 5.

7. The output control unit is When the amount of information used for the self-location estimation is insufficient with respect to the predetermined value, an instruction voice indicating information different from that when the amount of information used for the self-location estimation is not insufficient with respect to the predetermined value is output as the instruction information.

3. The information processing device according to claim 1 or 2.

8. The output control unit is When the amount of information used for the self-location estimation is not insufficient with respect to the predetermined value, an instruction voice indicating that the amount of information used for the self-location estimation is not insufficient with respect to the predetermined value is output as the instruction information. The information processing device according to claim 3 .

9. The output control unit is When an amount of information used for the self-location estimation is insufficient with respect to the predetermined value, an indication image showing information different from that when the amount of information used for the self-location estimation is not insufficient with respect to the predetermined value is output as the indication information.

8. An information processing apparatus according to claim 1, 2, 4, 5, 6, or 7.

10. The output control unit is When the amount of information used for the self-location estimation is not insufficient with respect to the predetermined value, an indication image indicating that the amount of information used for the self-location estimation is not insufficient with respect to the predetermined value is output as the indication information.

9. The information processing device according to claim 3 or 8.

11. The amount of information used for the self-location estimation is represented by the number of feature points. The information processing device according to any one of claims 1 to 10.

12. the information used for the self-location estimation is information obtained by a tracking process that is a process of tracking the same feature point extracted from the successive pieces of image information; The amount of information used for the self-location estimation is represented by the number of feature points tracked as a result of the tracking process. The information processing device according to any one of claims 1 to 11.

13. An information processing method for performing automatic driving based on driving route data indicating a driving route when a user manually drives a vehicle, comprising: receiving image information obtained by photographing the periphery of the moving object during the teacher traveling, and extracting information to be used for self-location estimation from the received image information; a step of outputting, to an output unit, instruction information different from that outputted when an amount of information to be used for self-location estimation around the moving object, which is extracted on the travel route, is insufficient for a predetermined value; and Including, the start position of the teacher driving is the start position of the automatic driving, the output control unit notifies the user of a normal end of teacher traveling when an amount of information used for the self-position estimation is not insufficient for the predetermined value; the instruction information is output before the automatic traveling based on the traveling route data is performed. Information processing methods.

14. The instruction information is information indicating, when an amount of information used for the self-location estimation is insufficient with respect to the predetermined value, that an amount of information used for the self-location estimation is insufficient with respect to the predetermined value. The information processing method according to claim 13.

15. The instruction information is information indicating that an amount of information used for the self-location estimation is not insufficient for the predetermined value when the amount of information used for the self-location estimation is not insufficient for the predetermined value. The information processing method according to claim 13.

16. The method further includes the step of generating map data from the extracted information used for self-location estimation, The instruction information is information indicating, when an amount of information used for the self-location estimation is insufficient with respect to the predetermined value, a portion of the map data where an amount of information used for the self-location estimation is insufficient with respect to the predetermined value.

15. The information processing method according to claim 13 or 14.

17. The instruction information is information for informing a location where an object including a large amount of information to be used for the self-location estimation is to be installed in correspondence with a location where an amount of information to be used for the self-location estimation is insufficient with respect to the predetermined value, so that the information to be used for the self-location estimation is added to the map data. The information processing method according to claim 16.

18. The instruction information is information for informing a location where a pre-stored landmark such as a position marker should be installed in response to a portion where an amount of information used for the self-location estimation is insufficient with respect to the predetermined value, so that the information used for the self-location estimation is added to the map data.

18. The information processing method according to claim 16 or 17.

19. In the step of the output control unit outputting the instruction information to the output unit, When the amount of information used for the self-location estimation is insufficient with respect to the predetermined value, an instruction voice indicating information different from that when the amount of information used for the self-location estimation is not insufficient with respect to the predetermined value is output as the instruction information.

15. The information processing method according to claim 13 or 14.

20. In the step of the output control unit outputting the instruction information to the output unit, When the amount of information used for the self-location estimation is not insufficient with respect to the predetermined value, an instruction voice indicating that the amount of information used for the self-location estimation is not insufficient with respect to the predetermined value is output as the instruction information. The information processing method according to claim 15.

21. In the step of the output control unit outputting the instruction information to the output unit, When an amount of information used for the self-location estimation is insufficient with respect to the predetermined value, an indication image showing information different from that when the amount of information used for the self-location estimation is not insufficient with respect to the predetermined value is output as the indication information.

20. The information processing method according to claim 13, 14, 16, 17, 18 or 19.

22. In the step of the output control unit outputting the instruction information to the output unit, When the amount of information used for the self-location estimation is not insufficient with respect to the predetermined value, an indication image indicating that the amount of information used for the self-location estimation is not insufficient with respect to the predetermined value is output as the indication information.

21. The information processing method according to claim 15 or 20.

23. The amount of information used for the self-location estimation is represented by the number of feature points.

23. The information processing method according to any one of claims 13 to 22.

24. the information used for the self-location estimation is information obtained by a tracking process that is a process of tracking the same feature point extracted from the successive pieces of image information; The amount of information used for the self-location estimation is represented by the number of feature points tracked as a result of the tracking process.

24. The information processing method according to any one of claims 13 to 23.

Images