Information processing method and information processing device

Abstract

This information processing method for executing control to cause a vehicle to travel automatically along a travel route to a set destination includes: acceptance processing for accepting a movement instruction from a user located outside the vehicle; detection processing for detecting a specific object among at least a portion of the user and the surroundings of the user; and setting processing for setting the position of the user as the destination and setting a stopping position at the destination on the basis of the detection result of the specific object.

Description

Information processing method and information processing device

[0001] The present invention relates to an information processing method and an information processing device that executes control to automatically drive a vehicle along a route to a destination.

[0002] Conventionally, technologies for automatically driving a vehicle along a route to a destination have been proposed. For example, JP2018-045397A proposes a technology for recognizing movement instruction information from a guide instructing the vehicle to move from an image captured by an onboard camera, creating a driving plan based on the movement instruction information, and automatically driving the vehicle according to the driving plan. Summary of the Invention

[0003] In the above-described conventional technology, it is possible to create a driving plan in response to instructions given by gestures from a guide and automatically drive a vehicle according to the driving plan. Here, it is conceivable that when a user of a vehicle calls for the vehicle, the vehicle may automatically drive to the user's location. In this case, depending on the user's situation, it is conceivable that the location where the vehicle stops may be inconvenient for the user. For example, when a vehicle is automatically driven to the user's location in response to instructions from a user carrying a large amount of luggage, it is conceivable that the door of the vehicle loading the luggage may not be located at the user's location. In this case, the user needs to walk to the door where the luggage is loaded and open it, which may result in a cumbersome movement.

[0004] The present invention aims to stop a vehicle at an appropriate location depending on the circumstances of the user summoning the vehicle or the surrounding environment.

[0005] One aspect of the present invention is an information processing method for controlling a vehicle to automatically travel along a route to a set destination. The information processing method includes a reception process for receiving a movement instruction from a user located outside the vehicle, a detection process for detecting a specific object within the user and at least a portion of the user's surroundings, and a setting process for setting the user's location as a destination and setting a stopping position at the destination based on the detection result of the specific object.

[0006] FIG. 1 is a simplified diagram showing the positional relationship between a vehicle and a user. FIG. 2 is a simplified diagram showing the positional relationship between a vehicle and a user. FIG. 3 is a simplified diagram showing the user himself or her surroundings. FIG. 4 is a simplified diagram showing an example of a display screen. FIG. 5 is a block diagram showing an example of the system configuration of an information processing system. FIG. 6 is a diagram showing an example of the configuration of a stop position setting information DB. FIG. 7 is a diagram showing an example of the configuration of a determination result DB. FIG. 8 is a diagram showing the relationship between a stop position of a vehicle and a user's position. FIG. 9 is a diagram showing the relationship between a stop position of a vehicle and a user's position. FIG. 10 is a diagram showing the relationship between a stop position of a vehicle and a user's position. FIG. 11 is a diagram showing the relationship between a stop position of a vehicle and a user's position. FIG. 12 is a flowchart showing an example of vehicle movement processing.

[0007] Hereinafter, an embodiment of the present invention will be described with reference to the accompanying drawings.

[0008] 1 and 2 are simplified diagrams showing the positional relationship between a vehicle C1 and a user U1 who owns the vehicle C1. In FIGS. 1 and 2, an example is shown in which the user U1 issues a travel instruction to automatically drive the vehicle C1 to the user U1's location. That is, the example shows a case in which the user U1 calls the vehicle C1 to his or her own location.

[0009] FIG. 1 illustrates an example in which a user U1 issues a movement instruction to a vehicle C1 parked in a parking space PS1 in a parking lot P1. In FIG. 1, each parking space in the parking lot P1 (including parking space PS1) is simplified and indicated by a thick line. Also, in FIG. 1, a temporary parking space for loading luggage is indicated as a luggage loading area LL1. The parking lot P1 may be, for example, a commercial facility such as a shopping center, or various public facilities. Also, FIG. 1 illustrates an example of an environment in which there is an obstacle, such as another vehicle, between the vehicle C1 and the user U1, the user U1 is located in a position that is not visible from the vehicle C1, and the user U1's voice cannot reach the vehicle C1. In this case, for example, a movement instruction to the vehicle C1 can be issued using an electronic device MC1.

[0010] 2 shows an example of a case where a user U1 issues a movement instruction to a vehicle C1 parked in a wide, open space M1 with good visibility. FIG. 2 shows an example of an environment in which there are no obstacles between the vehicle C1 and the user U1, the user U1 is located in a position visible from the vehicle C1, and the user U1's voice can be heard by the vehicle C1. The wide, open space M1 may be, for example, a large home garden or a vacant lot. In this case, it is possible to issue a movement instruction to the vehicle C1 by, for example, a specific movement by the user U1 or a voice S1 emitted by the user U1.

[0011] Here, assume that user U1 issues a movement instruction to vehicle C1 in order to use vehicle C1. In this case, for example, user U1 may issue a movement instruction to vehicle C1, causing vehicle C1 to automatically travel to user U1's location. For example, in the example shown in FIG. 1, user U1 cannot call vehicle C1 by gesture or voice, so it is conceivable that user U1 issues a movement instruction using electronic device MC1. Furthermore, in the example shown in FIG. 2, user U1 may issue a movement instruction to vehicle C1 by gesture or voice S1 in addition to issuing a movement instruction using electronic device MC1. In these cases, even if vehicle C1 is parked in front of user U1, it is conceivable that the parked location will be inconvenient for user U1.

[0012] For example, assume that user U1 is carrying a large amount of luggage BG1. In such a case, it is assumed that user U1 loads the large amount of luggage BG1 into the trunk (rear luggage compartment) of vehicle C1 and then gets into the driver's seat. Here, in response to a movement instruction from user U1, it is conceivable that vehicle C1 is stopped so that the front seat (driver's seat or passenger seat) of vehicle C1 is positioned at user U1's position. However, if user U1 intends to load the large amount of luggage BG1 into the trunk of vehicle C1, the trunk of vehicle C1 carrying the large amount of luggage BG1 will be misaligned with the position of user U1. In this case, user U1 will need to walk to the trunk where the large amount of luggage BG1 is loaded and open the trunk door (back door), which may result in cumbersome movements and the process of opening the back door.

[0013] Therefore, in this embodiment, the location where the vehicle C1 is stopped is set to be a location that is inconvenient for the user U1 depending on the situation of the user U1. That is, the vehicle C1 is set to stop at an appropriate location depending on the situation of the user U1 who issued the movement instruction to the vehicle C1 or the surrounding situation.

[0014] [Example of a user calling a vehicle or surrounding circumstances] FIG. 3 is a simplified diagram showing a typical example of a user U1 calling a vehicle C1 or surrounding circumstances.

[0015] 3A and 3D show an example in which user U1 is carrying large luggage BG2 and BG3. FIG. 3B shows an example in which other users U2 and U3 are riding in vehicle C1 with user U1. FIG. 3C shows an example in which user U1 is holding baby U4. FIG. 3E shows an example in which child users U5 and U6 are riding in vehicle C1 with user U1. FIG. 3F shows an example in which user U1 is not carrying any luggage or the like, and no other users are present.

[0016] [Example of Instructions When a User Causes a Vehicle to Automatically Drive to the User's Location] Here, a movement instruction method for instructing a vehicle C1 to automatically drive to a location where a user U1 is present will be described. For example, the user U1 can issue a movement instruction using an electronic device MC1. This movement instruction method will be described in detail with reference to FIGS. 4 and 5. The electronic device MC1 is an electronic device such as a smartphone, a tablet terminal, or a key for the vehicle C1. For example, a function for calling the vehicle C1 can be realized by downloading a specific application to the smartphone. An example configuration of the electronic device MC1 will be described in detail with reference to FIG. 5.

[0017] Furthermore, for example, user U1 can instruct the vehicle C1 to move by making a predetermined gesture. For example, if user U1 is present in a position where the user U1 is included in the imaging range of the vehicle exterior image acquisition unit (sensors 210 (see FIG. 5 )) of the vehicle C1, the movement of user U1 can be detected based on the captured image generated by the vehicle exterior image acquisition unit. Therefore, it is possible to determine whether user U1 has made a predetermined gesture based on whether the movement of user U1 detected based on the captured image is a predetermined movement. If it is determined that user U1 has made a specific gesture, it is determined that user U1 has instructed the vehicle C1 to move. Note that a specific gesture can be defined as an action of repeatedly moving a hand from front to back with either the left or right arm raised, as shown in FIGS. 3A to 3F . Also, a known image recognition technology can be used to determine the specific gesture. Note that if user U1 himself or herself can be identified, personal identification of the user who performed the specific gesture may be performed, and the movement instruction from user U1 may be determined on the condition that the user is recognized as user U1. It should be noted that for personal identification of the user U1, known personal identification technology, face identification technology, etc. can be used.

[0018] Furthermore, for example, the user U1 can instruct the vehicle C1 to move by uttering a predetermined voice. For example, when the user U1 is present in a position where the user U1's voice can be detected by the sound acquisition unit (sensors 210 (see FIG. 5 )) of the vehicle C1, the voice uttered by the user U1 can be detected based on the voice information acquired by the sound acquisition unit. Therefore, it is possible to determine whether the voice uttered by the user U1 detected based on the voice information is a predetermined voice (e.g., "Come pick me up here"), and based on this, it is possible to determine whether the user U1 has uttered a predetermined instruction message. If it is determined that a specific instruction message has been uttered, it is determined that the user U1 has instructed the vehicle C1 to move. The specific instruction message can be determined using a known voice recognition technology. Furthermore, if the user U1's voice can be identified, the user U1's voice can be identified, and the movement instruction issued by the user U1 can be determined based on the conditions that the user U1 is recognized as the user U1 and has uttered the specific instruction message. For personal identification of the user U1's voice, a known voice recognition technology or the like can be used.

[0019] Note that these movement instruction methods are merely examples, and other movement instruction methods may be used. For example, a specific gesture may be combined with a predetermined instruction message, or a specific gesture or a predetermined instruction message may be issued after a movement instruction operation is performed using the electronic device MC1.

[0020] 4 is a simplified diagram showing an example of a display screen 410 displayed on the display unit 407 of the electronic device MC1 when issuing a movement instruction for the vehicle C1 using the electronic device MC1. The movement instruction operation using the display screen 410 will be described in detail with reference to FIG.

[0021] [Configuration Example of Information Processing System] FIG. 5 is a block diagram showing an example of the system configuration of the information processing system 1 installed in the vehicle C1.

[0022] The information processing system 1 includes an information processing device 100, sensors 210, and an actuator 220. The information processing device 100 is an example of a device capable of executing control to automatically drive a vehicle C1 along a travel route to a set destination.

[0023] Furthermore, the information processing device 100 and the electronic device MC1 are connected by a communication method using wireless communication. For example, the information processing device 100 and the electronic device MC1 are connected to a network 20 by a communication method using wireless communication, and data is exchanged between the information processing device 100 and the electronic device MC1. The network 20 is a network such as a public line network or the Internet. Note that the information processing device 100 and the electronic device MC1 may be directly connected by using wireless communication (e.g., Wi-Fi (Wireless Fidelity) or Bluetooth (registered trademark)) without going through the network 20, and data may be exchanged between the information processing device 100 and the electronic device MC1.

[0024] The sensors 210 are various sensors installed in the vehicle C1 and output detection information acquired by each sensor to the information processing device 100. The sensors acquire information, for example, from a vehicle operation unit (e.g., a steering wheel, an accelerator pedal, and a brake pedal) and a vehicle signal acquisition unit (e.g., an acquisition unit that acquires each sensor value). These sensors include, for example, a light detection and ranging (LiDAR), a radio detection and ranging (RADAR), a sonar, a sound acquisition unit, an interior image acquisition unit, an exterior image acquisition unit, a position information acquisition unit, a vehicle speed sensor, an acceleration sensor, a yaw rate sensor, a steering sensor, an accelerator position sensor, a brake pedal, a seating sensor, and a seat belt sensor. Known sensors can be used as each of these sensors. The LiDAR, RADAR, and sonar are examples of sensors that detect the surrounding conditions of the vehicle C1. The vehicle speed sensor, acceleration sensor, yaw rate sensor, steering sensor, accelerator position sensor, and brake pedal are examples of sensors that detect information corresponding to the driving state of the vehicle C1. These are merely examples, and other sensors may be used, or only some of these sensors may be used.

[0025] The location information acquisition unit acquires location information regarding the location of the vehicle C1. For example, it can be realized by a GNSS (Global Navigation Satellite System) receiver that acquires location information using the GNSS. The location information includes data regarding the location, such as latitude, longitude, and altitude, at the time of receiving the GNSS signal. Alternatively, the location information may be acquired by other methods. For example, the location information may be derived using information from nearby access points or base stations. Alternatively, the location information may be acquired using a beacon.

[0026] The sound acquisition unit is provided inside and outside the vehicle C1, acquires sounds inside and outside the vehicle C1, and outputs sound information related to the acquired sounds to the information processing device 100. As the sound acquisition unit, for example, one or more microphones or sound acquisition sensors can be used.

[0027] The interior image acquisition unit captures an image of a subject inside the vehicle C1 to generate an image (image data), and outputs image information related to the generated image (interior image) to the information processing device 100. The interior image acquisition unit is provided at least inside the vehicle C1 (e.g., the ceiling) and captures images of each part inside the vehicle C1 (including each seat, occupants, and luggage) to generate an interior image. The interior image acquisition unit is configured, for example, with one or more camera devices or image sensors capable of capturing images of the interior of the vehicle C1. For example, one interior image acquisition unit may be provided in the front of the interior of the vehicle C1 (e.g., the ceiling) and capture images of the interior of the vehicle C1 from the front to generate an interior image. Furthermore, for example, another interior image acquisition unit may be provided in the rear of the interior of the vehicle C1 and capture images of the interior of the vehicle C1 from the rear to generate an interior image.

[0028] The outside-vehicle image acquisition unit captures an image of a subject outside the vehicle C1 to generate an image (image data), and outputs image information related to the generated image (outside-vehicle image) to the information processing device 100. Note that two or more outside-vehicle image acquisition units may be provided, and all or some of the images from these outside-vehicle image acquisition units may be used. For example, one outside-vehicle image acquisition unit may be provided in front of the vehicle C1 to capture an image of a subject from in front of the vehicle C1 to generate an outside-vehicle image, and another outside-vehicle image acquisition unit 104 may be provided behind the vehicle C1 to capture an image of a subject behind the vehicle C1 to generate an outside-vehicle image.

[0029] The interior image acquisition unit and the exterior image acquisition unit may each include, for example, an image sensor that receives light from a subject collected by a lens and an image processing unit that performs predetermined image processing on image data generated by the image sensor. The image sensor may be, for example, a charge-coupled device (CCD) or complementary metal oxide semiconductor (CMOS) image sensor. Alternatively, the interior image acquisition unit and the exterior image acquisition unit may each be one or more devices, such as a 360-degree camera, that can capture images of subjects in all directions around the vehicle C1 and subjects inside the vehicle C1.

[0030] The information processing device 100 includes a signal input unit 110, a control unit 120, an output unit 130, a storage unit 140, and a communication unit 150. The communication unit 150 exchanges various types of information with other devices using wired or wireless communication under the control of the control unit 120. For example, when the communication unit 150 receives movement instruction information instructing the vehicle C1 to travel automatically from the electronic device MC1, the communication unit 150 outputs the movement instruction information to the control unit 120.

[0031] The signal input unit 110 acquires each piece of information (output value of each sensor) output from each of the sensors 210 , and outputs the information to the control unit 120 .

[0032] The control unit 120 is a controller that controls each unit based on various programs stored in the storage unit 140. The control unit 120 is realized by a processing device such as a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit). The vehicle ECU (Electronic Control Unit) of the vehicle C1 may also be used as the control unit 120, or a processing device different from the vehicle ECU may be provided as the control unit 120.

[0033] The control unit 120 executes various controls based on information output from the sensors 210, the communication unit 150, etc., and information acquired from the storage unit 140. For example, the control unit 120 sets a destination for the vehicle C1 in response to a travel instruction from the user U1, and executes a control process to automatically drive the vehicle C1 along a driving route to the destination. Specifically, the control unit 120 includes a receiving unit 121, a detecting unit 122, a setting unit 123, a driving route generating unit 124, an operation control unit 125, and a determining unit 126.

[0034] The reception unit 121 receives information output from the sensors 210, the storage unit 140, the communication unit 150, etc., and supplies the received information to each unit. For example, the signal input unit 110 functions as a reception unit that receives input (e.g., an instruction to move the vehicle C1) from a user U1 located outside the vehicle C1.

[0035] The detection unit 122 detects objects inside and outside the vehicle C1, road surface conditions around the vehicle C1, and the like based on the information output from the reception unit 121, and outputs the detection results to the setting unit 123, the driving path generation unit 124, the operation control unit 125, and the determination unit 126. For example, the detection unit 122 executes a detection process to detect a specific object in at least a part of the user U1 who instructed the vehicle C1 to move and the user U1's surroundings based on the outside-of-vehicle image acquired by the outside-of-vehicle image acquisition unit. This detection process can use, for example, a known image recognition technique such as a pattern matching process that compares a preset pattern image with the outside-of-vehicle image.

[0036] Furthermore, for example, the detection unit 122 executes a detection process to detect the user U1 who instructed the vehicle C1 to move and a specific object in at least a part of the surroundings of the user U1, based on the movement instruction information transmitted from the electronic device MC1 acquired by the communication unit 150. This detection method will be described later.

[0037] Here, the specific object refers to a predetermined specific object among objects (e.g., a person or an object) that exist in the user U1 and at least a portion of the area around the user U1. Examples of the specific object include luggage that the user U1 is carrying, a baby that the user U1 is holding, and another user (e.g., a passenger) that exists in the area around the user U1. For example, the detection unit 122 detects an object that exists in the user U1 and at least a portion of the area around the user U1, and if the object is luggage that is larger than a predetermined size, a person with a predetermined attribute (e.g., a passenger, a baby, a child), or the like, detects the object as a specific object.

[0038] Furthermore, for example, the detection unit 122 can detect other objects (e.g., obstacles) present around the vehicle C1 and road surface conditions around the vehicle C1 (e.g., the position, size, shape, etc. of vacant parking spaces in the parking lot P1) based on the position information acquired by the position information acquisition unit and the map information DB in the storage unit 140. Furthermore, for example, the detection unit 122 can detect other objects (e.g., other vehicles, humans, animals such as dogs) present around the vehicle C1 based on sounds outside the vehicle acquired by the sound acquisition unit.

[0039] The setting unit 123 sets the stop position of the vehicle C1 instructed to move and the orientation of the vehicle C1 at the stop position based on the specific object detected by the detection unit 122 and the stop position setting information DB 160 of the storage unit 140. For example, if the detection unit 122 does not detect a specific object, the setting unit 123 sets the position where the front door is in front of the user U1 as the stop position. On the other hand, for example, if the detection unit 122 detects a specific object, the setting unit 123 sets the stop position and the door to be automatically opened based on the contents of the target position 162 and the target door 163 (see FIG. 6 ) corresponding to the detected specific object. Note that if information about the specific object detected by the detection unit 122 is stored in the determination result DB 170, the setting unit 123 may set the stop position of the vehicle C1 instructed to move and the orientation of the vehicle C1 at the stop position based on the specific object detected by the detection unit 122 and the determination result DB 170 of the storage unit 140. This example will be described in detail with reference to FIG. 12 and other figures.

[0040] The orientation of the vehicle C1 at the stopping position is set appropriately depending on the location of the user U1. For example, as shown in FIG. 1, if the location where the vehicle C1 will stop (luggage loading location LL1) is determined, the orientation of the vehicle C1 is set depending on that location. In the example shown in FIG. 1, the orientation of the vehicle C1 is set so that the passenger seat side faces the user U1. Also, for example, as shown in FIG. 2, if the location where the vehicle C1 will stop has not been determined, the orientation of the vehicle C1 is set based on a specific object detected by the detection unit 122. In the example shown in FIG. 2, the orientation of the vehicle C1 is set so that the back door side faces the user U1. Note that these setting methods are merely examples, and other settings may be used.

[0041] The driving route generation unit 124 creates a driving plan for the vehicle C1 to the destination based on the information output from the reception unit 121, the detection results by the detection unit 122, and the map information DB. For example, the driving route generation unit 124 calculates a driving route from the current location of the vehicle C1 to the destination based on the current location of the vehicle C1 acquired by the position information acquisition unit, the destination set in response to a travel instruction from the user U1, and the map information DB. Furthermore, for example, the driving route generation unit 124 can appropriately modify the driving route from the current location of the vehicle C1 to the destination based on the stopping position of the vehicle C1 set by the setting unit 123, the orientation of the vehicle C1 at the stopping position, and the map information DB while the vehicle C1 is traveling. Furthermore, the driving route generation unit 124 generates a driving route including a driving route driving speed, a target driving acceleration, and the like, as necessary.

[0042] For example, drivable routes within a predetermined area such as parking lot P1 (see FIG. 1 ) are stored in a map information DB. For example, various information such as the entrance and exit of parking lot P1, the luggage loading area LL1, and each parking space is stored in the map information DB. Therefore, the driving route generation unit 124 can generate a driving route from the current location of vehicle C1 in parking lot P1 to a destination (e.g., luggage loading area LL1) using the various information stored in the map information DB. In this case, if the detection unit 122 detects white lines or lanes indicating parking spaces within parking lot P1, or arrows indicating the vehicle's traveling direction within parking lot P1, the driving route generation unit 124 can also use the detection results to modify the driving route. Note that various information regarding parking lot P1 may be acquired from the parking lot itself via the communication unit 150.

[0043] The operation control unit 125 outputs a control signal for driving the vehicle C1 to the actuator 220. For example, when the vehicle C1 is to be driven automatically, the operation control unit 125 outputs a control signal for driving the vehicle C1 automatically to the actuator 220 based on the driving route generated by the driving route generation unit 124. Furthermore, for example, when the user U1 is to drive the vehicle C1 manually, the operation control unit 125 outputs a control signal for driving the vehicle C1 manually to the actuator 220 based on information detected by the sensors 210 (e.g., accelerator pedal sensor, brake pedal sensor, etc.).

[0044] When the vehicle C1 is traveling autonomously, the detection unit 122 sequentially detects obstacles, signs, etc. present around the vehicle C1 based on the situation around the traveling vehicle C1. When the detection unit 122 detects an obstacle, etc. in the traveling direction of the vehicle C1, the operation control unit 125 controls the actuator 220 to avoid contact with the obstacle, etc. For example, it is possible to output a drive stop signal to the actuator 220 to forcibly stop the vehicle C1.

[0045] Furthermore, when the vehicle C1 is autonomously traveling and moves close to the stopping position set by the setting unit 123, the operation control unit 125 controls the actuator 220 to stop the vehicle C1 at the stopping position. Furthermore, when the orientation of the vehicle C1 at the stopping position is set by the setting unit 123, the operation control unit 125 controls the actuator 220 to stop the vehicle C1 in that orientation.

[0046] The determination unit 126 determines whether the vehicle C1 is parked at a position that is consistent with the user U1's intention, based on the relationship between the vehicle C1's parked position set by the setting unit 123, the door set to open automatically, and the door into which the specific object detected by the detection unit 122 has been placed inside the vehicle C1. The determination unit 126 then stores the determination result in the determination result DB 170 of the storage unit 140. The determination process by the determination unit 126 will be described in detail with reference to FIGS. 7, 12, etc.

[0047] The actuator 220 realizes driving operations of the vehicle C1, including basic operations such as moving, turning, and stopping, based on control signals from the control unit 120. For example, the actuator 220 includes a drive actuator that generates a driving force for driving, a braking actuator that generates a braking force, and a steering actuator that generates a steering force. Here, if the vehicle C1 is a vehicle driven by an engine, the drive actuator is configured, for example, by a throttle actuator that controls the throttle opening. Furthermore, if the vehicle C1 is an electric vehicle or a hybrid vehicle, the drive actuator is configured, for example, by a driving motor. Furthermore, the braking actuator is configured, for example, by a hydraulic device that supplies hydraulic pressure to a brake disc. Furthermore, the steering actuator is configured, for example, by a steering motor that controls the steering torque of an electric power steering. In this way, the actuator 220 functions as an automatic driving unit for realizing automatic driving of the vehicle C1.

[0048] Furthermore, the actuator 220 can realize the operation (e.g., opening and closing operation) of a component (e.g., a door) of the vehicle C1 based on a control signal from the control unit 120. That is, the actuator 220 functions as a vehicle component moving unit for controlling the operation of the component of the vehicle C1. Note that known control techniques can be used to control the operation of the component (e.g., a door) of the vehicle C1.

[0049] The storage unit 140 is a storage medium that stores various types of information. For example, the storage unit 140 stores various types of information (e.g., a control program, a stop position setting information DB 160, a determination result DB 170, and a map information DB) that are necessary for the control unit 120 to perform various processes. The storage unit 140 also stores various types of information acquired via the communication unit 150. The storage unit 140 may be, for example, a read-only memory (ROM), a random access memory (RAM), a static random access memory (SRAM), a hard disk drive (HDD), a solid state drive (SSD), or a combination thereof.

[0050] The stop position setting information DB 160 stores various information related to the specific object detected by the detection unit 122, the stop position of the vehicle C1, the door that opens automatically after the vehicle C1 stops, and priority. The stop position setting information DB 160 will be described in detail with reference to FIG. 6.

[0051] The determination result DB 170 stores various information (e.g., determination results by the determination unit 126) when the vehicle C1 is moved by automatic driving based on a movement instruction from the user U1. The determination result DB 170 will be described in detail with reference to FIG. 7 .

[0052] The map information DB stores map information such as road information related to roads required for the automated driving of the vehicle C1. The map information includes road information such as the location and shape of the road (e.g., road gradient, road intersections, the number of lanes on the road, road width information, road undulation information, rough road conditions, speed limits, one-way streets, crosswalks, dividing lines, and road structures (e.g., traffic lights and telephone poles)). The map information also includes information such as the location of various facilities and the shape of drivable areas within the facilities (e.g., the location and shape of parking lot P1 (see FIG. 1 )). Instead of storing the map information DB in the memory unit 140, necessary information from the map information DB may be obtained from outside the vehicle C1 via the communication unit 150 and used.

[0053] [Configuration Example of Electronic Device] The electronic device MC1 includes a communication unit 401, a control unit 402, a storage unit 403, an operation unit 404, a sound acquisition unit 405, a sound output unit 406, a display unit 407, and a location information acquisition unit 408. The electronic device MC1 may be an information processing device such as a smartphone or a tablet terminal, or may be a device related to the vehicle C1, such as a key for the vehicle C1 (capable of communicating with the vehicle C1 using a vehicle signal).

[0054] The communication unit 401 exchanges various types of information with other devices using wired or wireless communication under the control of the control unit 402 .

[0055] The control unit 402 controls each unit based on various programs stored in the storage unit 403. The control unit 402 is realized by a processing device such as a CPU, for example.

[0056] The storage unit 403 is a storage medium that stores various types of information. For example, the storage unit 403 stores various types of information (e.g., control programs, movement instruction applications) required for the control unit 402 to perform various processes. The storage unit 403 also stores various types of information acquired via the communication unit 401. The storage unit 403 can be, for example, a ROM, a RAM, an SRAM, an HDD, an SSD, or a combination thereof.

[0057] The operation unit 404 accepts various operations from the user U1, and outputs operation information relating to the accepted operation content to the control unit 402.

[0058] The sound acquisition unit 405 acquires sounds around the electronic device MC1 and outputs sound information related to the acquired sounds to the control unit 402. As the sound acquisition unit, for example, one or more microphones or sound acquisition sensors can be used.

[0059] The sound output unit 406 outputs various types of audio information based on the control of the control unit 402. Note that the sound output unit 406 may be, for example, one or more speakers.

[0060] The display unit 407 displays various images based on the control of the control unit 402. The display unit 407 may be, for example, a display panel such as an organic EL (Electro Luminescence) panel or an LCD (Liquid Crystal Display) panel. The operation unit 404 and the display unit 407 may be configured as a touch panel that allows a user to input operations by touching or approaching the display surface with their finger, or may be configured as a separate user interface. The separate user interface (operation unit 404) is, for example, a physical operation member (for example, an operation button or an operation lever).

[0061] The location information acquisition unit 408 acquires location information relating to the location where the electronic device MC1 is located, and outputs the acquired location information to the control unit 402. For example, similar to the location information acquisition unit of the sensors 210, the location information acquisition unit 408 can be realized by a GNSS receiver that acquires location information using GNSS.

[0062] [Example of Operation for Instructing Movement Using Electronic Device] Next, an example in which the user U1 instructs the vehicle C1 to move using the electronic device MC1 will be described. Here, an example in which the user U1 instructs the vehicle C1 to move using the display screen 410 shown in FIG.

[0063] For example, user U1 can display a display screen 410 on the display unit 407 and issue a movement instruction to vehicle C1 by selecting a vehicle call button 411 on the display screen 410. In this case, user U1 can select the luggage selection area 412, the baby selection area 413, or the multiple passengers selection area 414 depending on the situation in which user U1 is waiting for vehicle C1, and then perform the selection operation of selecting the vehicle call button 411. For example, as shown in FIGS. 3A and 3D, when user U1 is carrying luggage BG2 and BG3, the luggage selection area 412 is selected. Furthermore, as shown in FIGS. 3B and 3E, when user U1 is waiting with other users U2, U3, U5, and U6, the multiple passengers selection area 414 is selected. Furthermore, as shown in FIG. 3C, when user U1 is waiting with baby U4, the baby selection area 413 is selected. Also, for example, as shown in FIG. 3(F), when user U1 is waiting alone without any luggage, the user performs a selection operation to select the vehicle call button 411 without selecting any of the selection areas.

[0064] When a selection operation is performed to select the vehicle call button 411 (see FIG. 4 ), the control unit 402 transmits to the vehicle C1 the location information of the electronic device MC1 acquired by the location information acquisition unit 408 and movement instruction information for automatically driving the vehicle C1 to a location corresponding to the location information. Note that when at least one of the luggage presence selection area 412, the baby presence selection area 413, and the multiple passengers presence selection area 414 is in a selected state, the control unit 402 includes the user situation information corresponding to the selected area in the movement instruction information and transmits it to the vehicle C1.

[0065] When movement instruction information is received from the electronic device MC1, the travel route generation unit 124 of the information processing device 100 detects the position of the user U1 based on the position information of the electronic device MC1 included in the movement instruction information. Then, the travel route generation unit 124 generates a travel route from the position of the vehicle C1 to the position of the user U1. A known route generation method can be used to generate this travel route.

[0066] In this way, when calling the vehicle C1 to the user U1's location, the user U1 only needs to inform the vehicle C1 of his / her situation, and does not need to inform the vehicle C1 of the location where the vehicle C1 will be parked. Depending on where the user U1 is waiting for the vehicle C1, it may be possible for the vehicle C1 to park with either the left or right side facing the user U1. Therefore, the user U1 may be able to select on the display screen 410 of the electronic device MC1 whether the vehicle C1 will park with either the left or right side facing the user U1. For example, the user U1 may be able to select whether the vehicle C1 will park on the driver's seat side or the passenger seat side facing the user U1.

[0067] Although the above example illustrates an example in which a movement instruction for the vehicle C1 is given by operating the display screen 410, the user U1 may also give a movement instruction for the vehicle C1 via voice to the electronic device MC1. For example, the user U1 can give a movement instruction for the electronic device MC1 by uttering a specific voice. In this case, the control unit 402 acquires the specific voice uttered by the user U1 from the sound acquisition unit 405 and can detect that the user U1 has given a movement instruction based on the specific voice. Furthermore, when the control unit 402 detects a movement instruction from the user U1 based on the specific voice, it transmits to the vehicle C1 the location information of the electronic device MC1 acquired by the location information acquisition unit 408 and movement instruction information for automatically driving the vehicle C1 to a location corresponding to the location information. Note that the user U1 can also emit other specific voices (e.g., "There's luggage," "There's a baby," "There are multiple passengers") to include user situation information in the movement instruction information and transmit it to the vehicle C1.

[0068] 1 , it is assumed that an obstacle such as another vehicle exists between the vehicle C1 and the user U1. In this case, the travel route generation unit 124 can generate a travel route that avoids the obstacle existing between the vehicle C1 and the user U1 using a known route generation method. In addition, the travel route generation unit 124 can appropriately modify the travel route while the vehicle C1 is traveling on the initially generated travel route based on an outside-vehicle image (an image of the surroundings of the vehicle C1) acquired by the outside-vehicle image acquisition unit (sensors 210) of the vehicle C1.

[0069] [Configuration Example of Stop Position Setting Information DB 160] FIG. 6 is a diagram showing a simplified configuration example of the stop position setting information DB 160 stored in the storage unit 140. As shown in FIG.

[0070] The stopping position setting information DB160 is a database that stores the relationship between the specific object detected by the detection unit 122, the stopping position of the vehicle C1, and the door that opens automatically after the vehicle C1 stops. Specifically, the specific object 161, the target position 162, the target door 163, and the priority 164 are stored in association with each other in the stopping position setting information DB160.

[0071] The specific object 161 is information indicating a specific object to be detected by the detection unit 122 when the user U1 issues an instruction to move the vehicle C1. For example, a baby held by the user U1, one or more users (passengers) waiting with the user U1, and various types of luggage (for example, small luggage, medium-sized luggage, and large luggage) are detected as specific objects.

[0072] The target position 162 is information indicating a stopping position at which the vehicle C1 is to be stopped when the detection unit 122 detects the specific object stored in the specific object 161. For example, when the detection unit 122 detects a baby, the stopping position of the vehicle C1 is set so that the seat where the baby seat is installed (e.g., the rear seat) is located directly in front of the user U1. The seat where the baby seat is installed can be determined based on an interior image acquired by the interior image acquisition unit (sensors 210 (see FIG. 5 )). Similarly, the status of each seat and trunk of the vehicle C1 (e.g., whether or not there is luggage) can be determined based on an interior image acquired by the interior image acquisition unit. For example, if luggage or the like is placed in any of the seats or trunk of the vehicle C1, the seats or trunk other than the position where the luggage or the like is placed can be set as the target position.

[0073] The target door 163 is information indicating a door that will be automatically opened after the vehicle C1 stops when the specific object stored in the specific object 161 is detected by the detection unit 122. For example, when a baby is detected by the detection unit 122, a door near the seat where a baby seat is installed (for example, a rear seat) is set as the door that will be automatically opened.

[0074] The priority 164 is information indicating a priority used to set a specific object to be prioritized when multiple specific objects are detected by the detection unit 122. For example, when a baby and a large piece of luggage are detected by the detection unit 122, the target position 162 and the target door 163 are set based on the position of the baby, which has a higher priority.

[0075] The priority 164 may be set manually by the user U1, or the preset content may be appropriately modified based on the behavior history of the user U1.

[0076] [Configuration Example of Determination Result DB 170] FIG. 7 is a diagram showing a simplified configuration example of the determination result DB 170 stored in the storage unit 140. As shown in FIG.

[0077] The determination result DB 170 is a database that stores various information when the vehicle C1 is moved by automatic driving based on a movement instruction from the user U1. Specifically, a date and time 171, position information 172, specific object information 173, specific object identification information 174, boarding information 175, and user intention agreement information 176 are stored in association with each other in the determination result DB 170.

[0078] The date and time 171 is information indicating the date and time when the vehicle C1 was moved by automatic driving based on a movement instruction from the user U1. For example, the date and time 171 stores information about the time when the vehicle C1 stopped after being moved by automatic driving based on a movement instruction from the user U1.

[0079] The location information 172 is information about a location to which the vehicle C1 has been moved by automatic driving based on a movement instruction from the user U1. For example, the location information 172 stores location information about facilities, places, etc. identified based on the location information of the vehicle C1 acquired by a location information acquisition unit (sensors 210 (see FIG. 5 )) and the map information DB stored in the storage unit 140.

[0080] The specific object information 173 is information indicating a specific object detected by the detection unit 122 when a movement instruction is given by the user U1. For example, specific information for identifying a specific object detected by the detection unit 122 (see FIG. 5 ) is stored in the specific object information 173. Note that, for ease of explanation, FIG. 7 shows an example in which the name of the specific object is stored in the specific object information 173.

[0081] The specific object identification information 174 is identification information for identifying a specific object whose specific object information is stored in the specific object information 173. For example, when the detection unit 122 detects a specific object by image recognition processing, image determination information (e.g., a pattern image, a trained model) used in the image recognition processing is stored in the specific object identification information 174. Note that, in FIG. 7 , for ease of explanation, a simplified shape of the specific object is schematically shown in the specific object identification information 174.

[0082] The entry information 175 is information indicating the door through which the specific object detected when a movement instruction is given from the user U1 is put into the vehicle C1 after the vehicle C1 moves and stops by automatic driving based on the movement instruction. For example, when a movement instruction is given for the vehicle C1 at "ABC Shopping Center," if a "bag" is detected by the detection unit 122 and the bag is put into the passenger door, "the front door on the passenger side" is stored in the entry information 175.

[0083] The user intention agreement information 176 is estimated information indicating whether the stopping position of the vehicle C1 after the vehicle C1 automatically travels and the orientation of the vehicle C1 at the stopping position match the intention of the user U1 when a movement instruction is given by the user U1. For example, the determination result by the determination unit 126 is stored in the user intention agreement information 176. The determination process by the determination unit 126 will be described in detail with reference to FIG. 12 etc.

[0084] [Example of Relationship Between User and Vehicle Stop Position] FIGS. 8 to 11 are diagrams showing, in a simplified form, the relationship between the stop position of the vehicle C1 after automatic traveling in response to a movement instruction and the position of the user U1.

[0085] 8 shows an example in which the vehicle C1 is stopped at a position where the user U1 can load a large piece of luggage BG3 into the trunk of the vehicle C1, and the back door BD1 is automatically opened. That is, in FIG. 8, the detection unit 122 detects the large piece of luggage BG3 as a specific object, the setting unit 123 sets the stopping position so that the back door BD1 of the vehicle C1 is positioned directly in front of the user U1, and the back door BD1 is set to automatically open.

[0086] In this way, the vehicle C1 stops so that the back door BD1 of the vehicle C1 is positioned in front of the user U1, and the back door BD1 opens automatically, so that the user U1 can immediately load the large luggage BG3 that he or she is carrying into the trunk without moving.

[0087] 9 shows an example in which the vehicle C1 is stopped at a position where users U2 and U3 can sit in the rear seats of the vehicle C1, and the rear door RD1 is automatically opened. That is, in FIG. 9, the detection unit 122 detects the users U2 and U3 as specific objects, the setting unit 123 sets the stopping position so that the rear door RD1 of the vehicle C1 is positioned in front of the users U2 and U3, and the rear door RD1 is set to open automatically.

[0088] In this way, the vehicle C1 stops so that the rear door RD1 of the vehicle C1 is positioned in front of the users U2 and U3, and the rear door RD1 opens automatically, so that the users U2 and U3 can immediately get into the back seat without moving.

[0089] FIG. 10 illustrates an example in which the vehicle C1 is parked in a position where the baby U4 can sit in the rear seat of the vehicle C1, and the rear door RD1 is automatically opened. That is, FIG. 10 illustrates an example in which the detection unit 122 detects the baby U4 as a specific object, the setting unit 123 sets the parking position so that the rear door RD1 of the vehicle C1 is positioned directly in front of the baby U4, and the rear door RD1 is automatically opened. Here, the detection unit 122 can detect the seating status of each seat and the installation status of a child seat in each seat based on an interior image acquired by the interior image acquisition unit (sensors 210 (see FIG. 5 )). Therefore, when the detection unit 122 detects the baby U4 as a specific object, the setting unit 123 sets the position corresponding to the seat where the child seat is detected by the detection unit 122 as the parking position and sets the door corresponding to that position as the door to be opened.

[0090] In this way, the vehicle C1 stops so that the rear door RD1 of the vehicle C1 is positioned in front of the user U1, and the rear door RD1 opens automatically, so that the user U1 can immediately place the baby U4 in the child seat in the back seat without moving.

[0091] 11 shows an example in which the vehicle C1 is stopped at a position where the user U1 can sit in the driver's seat of the vehicle C1 and the front door FD1 is automatically opened. That is, in FIG. 11 , the detection unit 122 does not detect a specific object, so the setting unit 123 sets the stopping position so that the front door FD1 of the vehicle C1 is positioned directly in front of the user U1, and the front door FD1 is set to automatically open.

[0092] In this way, the vehicle C1 stops so that the front door FD1 of the vehicle C1 is positioned in front of the user U1, and the front door FD1 opens automatically, allowing the user U1 to immediately get into the driver's seat without moving.

[0093] [Example of operation of information processing system] Figure 12 is a flowchart showing an example of vehicle movement processing in the information processing device 100. This vehicle movement processing is executed by the control unit 120 (see Figure 5) based on a program stored in the storage unit 140 (see Figure 5). This vehicle movement processing is executed for each control cycle. This vehicle movement processing will be explained with appropriate reference to Figures 1 to 11.

[0094] In step S501, the driving route generation unit 124 determines whether or not a movement instruction for the vehicle C1 has been received. For example, if the detection unit 122 detects a specific gesture by the user U1, if the detection unit 122 detects a specific calling voice by the user U1, or if movement instruction information is received from the electronic device MC1, it is determined that a movement instruction for the vehicle C1 has been received. If a movement instruction for the vehicle C1 has been received, the process proceeds to step S502. On the other hand, if a movement instruction for the vehicle C1 has not been received, monitoring continues.

[0095] In step S502, the driving route generation unit 124 acquires the position of the user U1 who issued the movement instruction for the vehicle C1 received in step S501. For example, when the detection unit 122 detects a specific gesture by the user U1, the position of the user U1 can be acquired based on an exterior image of the vehicle C1 surrounding the vehicle C1 that includes the user U1, among the exterior images of the vehicle C1 acquired by the exterior image acquisition unit (sensors 210 (see FIG. 5)). In this case, for example, the direction of the user U1 relative to the vehicle C1 can be acquired based on the exterior image that includes the user U1. Furthermore, for example, the distance from the vehicle C1 to the user U1 can be acquired based on the size of the user U1 in the exterior image. As a result, the position of the user U1 can be acquired.

[0096] Furthermore, for example, when the detection unit 122 detects a specific calling voice of the user U1, it is possible to acquire the position of the user U1 based on the calling voice emitted by the user U1 among the sounds around the vehicle C1 acquired by the sound acquisition unit (sensors 210 (see FIG. 5)). In this case, for example, it is possible to acquire the direction of the user U1 relative to the vehicle C1 based on the calling voice emitted by the user U1. Also, for example, it is possible to acquire the distance from the vehicle C1 to the user U1 based on the volume of the calling voice emitted by the user U1. In this way, it is possible to acquire the position of the user U1. Note that the position of the user U1 acquired based on the specific gesture or calling voice can be appropriately corrected based on an outside image, etc. acquired while the vehicle C1 is automatically traveling along the traveling route generated by the traveling route generation unit 124.

[0097] Furthermore, for example, when movement instruction information is received from the electronic device MC1, the location of the user U1 can be obtained based on the location information of the electronic device MC1 included in the movement instruction information.

[0098] In step S503, the driving route generation unit 124 generates a driving route to the position of the user U1 acquired in step S502. As described above, the driving route generation unit 124 can generate a driving route from the current location of the vehicle C1 to the position of the user U1 based on the current location of the vehicle C1 acquired by the position information acquisition unit, the position of the user U1, and the map information DB. Furthermore, for example, if a road surface on which the vehicle C1 can travel exists and no obstacles exist on a linear route connecting the vehicle C1 and the user U1, the driving route generation unit 124 can generate the linear route as a driving route. On the other hand, if a road surface on which the vehicle C1 can travel does not exist on the linear route connecting the vehicle C1 and the user U1, or if an obstacle exists on the route, the driving route generation unit 124 can generate a driving route by selecting a route on which the vehicle C1 can automatically travel from one or more routes to the position of the user U1 that can be extracted based on the map information DB. It should be noted that known driving route generation techniques can be used to generate these driving routes.

[0099] In step S504, the operation control unit 125 confirms that the driving route generated in step S503 is suitable for automatic driving, and then starts control for causing the vehicle C1 to automatically drive. For example, the driving route generation unit 124 checks objects present around the vehicle C1 detected by the detection unit 122 to determine whether or not an obstacle exists on the driving route of the vehicle C1. For example, as shown in FIG. 1 , when the vehicle C1 automatically drives in a parking lot P1, another vehicle may be detected as an obstacle. In this case, the driving route generation unit 124 generates a new driving route that bypasses the detected obstacle. Furthermore, even after the vehicle C1 starts automatic driving, the driving route generation unit 124 continues to sequentially check whether or not an obstacle exists on the driving route of the vehicle C1, and if an obstacle is detected, sequentially generates a new driving route that bypasses the detected obstacle.

[0100] In step S505, the detection unit 122 executes a detection process to detect the user U1, specific objects, etc. based on the vehicle exterior image acquired by the vehicle exterior image acquisition unit (sensors 210). This detection process detects the number of users, user attributes (e.g., gender, age, generation, appearance), and specific objects associated with the users (e.g., luggage). For example, when there are multiple users, the detection unit 122 can detect which of them is the driver using personal identification technology. For example, when the user U1 is the driver, the detection unit 122 can identify the user U1 from among the multiple users and identify the user U1 as the driver of the vehicle C1 based on the identification result.

[0101] 12 shows an example in which the detection process for detecting the user U1, a specific object, etc. is executed after the vehicle C1 starts autonomous driving, but is not limited to this. For example, if it is possible to execute the detection process for detecting the user U1, a specific object, etc. immediately after receiving a movement instruction from the user U1 in step S501, the detection process may be executed at that timing.

[0102] In step S506, the setting unit 123 determines whether or not a specific object was detected in step S505. If a specific object was detected, the process proceeds to step S507. On the other hand, if a specific object was not detected, the process proceeds to step S508.

[0103] In step S507, the setting unit 123 sets the stopping position of the vehicle C1, the orientation of the vehicle C1 at the stopping position, and the door to be automatically opened, based on the specific object detected in step S505. Specifically, the setting unit 123 uses the stopping position setting information DB 160 (see FIG. 6 ) to extract information on the target position 162 and the target door 163 corresponding to the specific object 161 detected in step S505. The setting unit 123 then sets the stopping position of the vehicle C1 based on the position of the extracted target position 162, and sets the door of the extracted target door 163 as the door to be automatically opened. Regarding the orientation of the vehicle C1, if the stopping position of the vehicle C1 has been determined, as shown in FIG. 1 , the orientation of the vehicle C1 is set according to the stopping position. On the other hand, as shown in FIG. 2 , if the stopping position of the vehicle C1 has not been determined, the orientation of the vehicle C1 is appropriately set according to the travel route from the departure point of the vehicle C1 to the stopping position. For example, it is preferable to set a position and orientation that make it easy for the vehicle C1 to depart after stopping. For example, it is possible to set the orientation so that the forward direction of the vehicle C1 is the expected direction of travel of the user U1 in the vehicle C1. Furthermore, for example, the orientation of the vehicle C1 may be set based on the user U1's past history information. For example, if the user U1 often loads luggage or the like from the driver's seat side, the stopping position of the vehicle C1 and the orientation of the vehicle C1 at the stopping position can be set so that the driver's seat door is located directly in front of the user U1. Note that these are merely examples, and other setting methods may also be adopted. Furthermore, for example, the relationship between the stopping position of the vehicle C1, the orientation of the vehicle C1 at the stopping position, and the specific object can be appropriately set based on experiments, simulations, or the like.

[0104] Furthermore, if multiple specific objects are detected in step S505, the priority 164 (see FIG. 6 ) set for each specific object can be used to set the stopping position of the vehicle C1, the orientation of the vehicle C1 at that stopping position, and the door that will open automatically. For example, the setting unit 123 can set the target position 162 corresponding to the specific object with the highest priority 164 as the stopping position of the vehicle C1, and set the target door 163 corresponding to the specific object with the highest priority 164 as the door that will open automatically. In this case, the position and orientation of the specific object with the highest priority 164 that make it easy for a child to enter the automatically opening door can be set as the stopping position of the vehicle C1 and the orientation of the vehicle C1 at that stopping position. For example, if the specific object with the highest priority is a baby, the door near a seat where a baby seat is installed can be set as the door that will open automatically. In this case, the stopping position of the vehicle C1 and the orientation of the vehicle C1 at that stopping position can be set to a position and orientation that makes it easy for a person holding a baby to place the baby from a door near the seat where the baby seat is installed. Note that the orientation of the vehicle C1 at the stopping position can be set appropriately based on the stopping location of the vehicle C1, etc., as in the case where one specific object is detected. Furthermore, the relationship between the stopping position of the vehicle C1, the orientation of the vehicle C1 at that stopping position, and the specific object with the highest priority can be set appropriately based on experiments, simulations, etc.

[0105] In step S508, the setting unit 123 sets the stopping position of the vehicle C1 so that the position of the driver's seat or passenger seat of the vehicle C1 is the user's position. In other words, since there are no specific objects around the driver, user U1, it is assumed that only user U1 will be riding in the vehicle C1. Therefore, the stopping position of the vehicle C1 is set so that the position of the driver's seat or passenger seat of the vehicle C1 is the user's position, so that user U1 can easily get into the driver's seat of the vehicle C1. In addition, the door of any seat (driver's seat or passenger seat of the vehicle C1) that is the user's position at the set stopping position of the vehicle C1 is set as a door that can be opened automatically. For example, as shown in FIG. 1, if the stopping location of the vehicle C1 is determined, the stopping position of the vehicle C1 is set so that the position of the driver's seat or passenger seat of the vehicle C1 is the user's position, with the vehicle C1 facing in a direction corresponding to the stopping location. On the other hand, as shown in FIG. 2, if the stopping location of the vehicle C1 has not been determined, the stopping location of the vehicle C1 is set so that the position of the driver's seat of the vehicle C1 is the position of the user.

[0106] In step S509, the operation control unit 125 determines whether the vehicle C1, whose autonomous traveling has been started in step S504, has arrived at the stop position set in step S507 or S508. Specifically, the operation control unit 125 determines whether the vehicle C1 has arrived at the stop position based on the position information of the vehicle C1 acquired by the position information acquisition unit and the position information of the stop position set by the setting unit 123. For example, if the position information of the vehicle C1 acquired by the position information acquisition unit matches the position information of the stop position set by the setting unit 123, the operation control unit 125 can determine that the vehicle C1 has arrived at the stop position.

[0107] In step S510, the operation control unit 125 stops the vehicle C1 that has arrived at the stopping position set in step S507 or S508, and after the vehicle has stopped, executes control to open the door set in step S507 or S508. For example, as shown in Fig. 10, if a user U1 is holding a baby, the vehicle C1 stops so that the rear door RD1 of the rear seat where the baby seat is installed is positioned at the position of the user U1, and after the vehicle C1 has stopped, the rear door RD1 is opened.

[0108] In step S511, the determination unit 126 determines whether the intention of the user U1 who issued the instruction to move the vehicle C1 received in step S501 was correctly recognized. For example, the determination unit 126 can determine whether the intention of the user U1 was correctly recognized based on the relationship between the door automatically opened in step S510 and the door through which the specific object detected in step S505 entered the vehicle C1. Specifically, the detection unit 122 can detect the movement of the user U1 and the specific object after the vehicle C1 has stopped based on the vehicle exterior image and the vehicle interior image acquired by the vehicle exterior image acquisition unit and the vehicle interior image acquisition unit (sensors 210 (see FIG. 5)), respectively. This movement of the user U1 and the specific object can be detected using known moving object detection technology. Furthermore, the determination unit 126 determines whether the specific object detected in step S505 entered through the door automatically opened in step S510 based on the movement of the user U1 and the specific object after the vehicle C1 has stopped, detected by the detection unit 122.

[0109] For example, as shown in FIG. 8 , when the back door BD1 is automatically opened and the user U1 places the luggage BG3 in the trunk, it can be determined that a specific object was placed through the automatically opened door. In this case, the determination unit 126 can determine that the intention of the user U1 who issued the instruction to move the vehicle C1 was correctly recognized. On the other hand, as shown in FIG. 8 , when the back door BD1 is automatically opened and the user U1 opens the rear door RD2 and places the luggage BG3 in the rear seat, it can be determined that a specific object was not placed through the automatically opened door. In this case, the determination unit 126 can determine that the intention of the user U1 who issued the instruction to move the vehicle C1 was not correctly recognized. If the intention of the user U1 was correctly recognized, the process proceeds to step S512. On the other hand, if the intention of the user U1 was not correctly recognized, the process proceeds to step S513.

[0110] In addition, other determination methods may be used in addition to the determination method for determining whether the user U1's intention was correctly recognized based on the relationship between the detected specific object and the automatically opened door. For example, whether the user U1's intention was correctly recognized may be determined based on the distance the user U1 walked after the vehicle C1 stopped and before getting into the vehicle C1. For example, if the distance the user U1 walked after the vehicle C1 stopped and before getting into the vehicle C1 is equal to or greater than a threshold (e.g., several meters), it is assumed that the vehicle C1 stopped in a location contrary to the user U1's intention or that a door opened contrary to the user U1's intention. In this case, it can be determined that the user U1's intention was not correctly recognized. On the other hand, if the distance the user U1 walked after the vehicle C1 stopped and before getting into the vehicle C1 is less than the threshold, it is assumed that the vehicle C1 stopped in a location consistent with the user U1's intention or that a door opened consistent with the user U1's intention. In this case, it can be determined that the user U1's intention was correctly recognized.

[0111] Furthermore, for example, whether the intention of the user U1 has been correctly recognized may be determined based on the behavior (e.g., voice, facial expression, reaction) of the user U1 or surrounding users after the vehicle C1 has stopped. For example, if the user U1 utters a specific keyword (e.g., a negative word such as "That's not right"), makes a specific facial expression (e.g., an angry expression), or makes a specific reaction (e.g., an angry reaction) after the vehicle C1 has stopped, it is assumed that the vehicle C1 has stopped in a location contrary to the user U1's intention, or that a door has opened contrary to the user U1's intention. Therefore, in this case, it can be determined that the intention of the user U1 has not been correctly recognized. On the other hand, if the user U1 utters a specific keyword (e.g., a positive word such as "Thank you"), makes a specific facial expression (e.g., a smiling expression), or makes a specific reaction (e.g., a nodding reaction) after the vehicle C1 has stopped, it is assumed that the vehicle C1 has stopped in a location consistent with the user U1's intention, or that a door has opened consistent with the user U1's intention. Therefore, in this case, it can be determined that the intention of the user U1 has been correctly recognized.

[0112] In step S512, the determination unit 126 stores the situation, stopping position, etc. of the user U1 who issued the instruction to move the vehicle C1 received in step S501 in the determination result DB 170 of the storage unit 140.

[0113] In step S513, the determination unit 126 estimates the cause of the inability to correctly recognize the intention of the user U1 who issued the instruction to move the vehicle C1. For example, assume that the user U1 issues an instruction to move the vehicle C1 while carrying a medium-sized item. In this case, the setting unit 123 sets the position of the rear seat directly in front of the user U1 as the stopping position based on the stopping position setting information DB 160, and sets the rear door of the rear seat as the door that automatically opens. If the user U1 places a medium-sized item through the front door on the passenger side after the vehicle C1 stops according to this setting, the cause of the inability to correctly recognize the intention of the user U1 can be estimated to be an incorrect detection of the size of the item, an incorrect setting of the automatic door opening, or the like. Note that these estimation results are merely examples, and other estimation methods may also be used. Furthermore, for example, artificial intelligence (AI) may be used to estimate the cause.

[0114] In step S514, the determination unit 126 stores the cause estimated in step S513, the situation of the user U1 who issued the instruction to move the vehicle C1 received in step S501, the stopping position, etc., in the determination result DB 170 of the storage unit 140. Note that the estimated cause can be stored in the determination result DB 170 in association with the situation, stopping position, etc., of the user U1.

[0115] In this way, the determination result stored in the determination result DB 170 can be used when a movement instruction is subsequently issued by the user U1. For example, assume that the user U1 issues a movement instruction for the vehicle C1 while carrying a medium-sized baggage. In this case, the setting unit 123 sets the position of the rear seat directly in front of the user U1 as the stopping position based on the stopping position setting information DB 160, and sets the rear door of the rear seat as the door that opens automatically. However, it is also assumed that the determination result DB 170 records that, when a movement instruction is issued while carrying a medium-sized baggage, the baggage is to be placed through the front door on the passenger side. In this way, when the setting contents based on the stopping position setting information DB 160 and the setting contents based on the determination result DB 170 differ, the determination result DB 170 is used preferentially. That is, in this case, the setting unit 123 sets the position where the passenger seat is located in front of the user U1 as the parking position and sets the passenger seat front door as the door that opens automatically, based on the determination result DB 170. Note that, when determination results with different contents for the same specific object are stored in the determination result DB 170, the content that takes these into consideration (the most common content) or the latest content may be used.

[0116] The above describes an example of executing a setting process, etc., in which the stopping position of the vehicle C1 and the orientation of the vehicle C1 at the stopping position are set based on the presence or absence of a specific object detected by the detection unit 122. However, the setting process, etc. may be executed by other methods. For example, the setting process, etc. may be executed using artificial intelligence (AI). For example, the user U1 himself / herself, the situation around the user U1, and the behavior of the user U1 after the vehicle C1 has stopped may be learned in advance, and this learning data may be used in the setting process, etc. For example, the stopping position of the vehicle C1 and the orientation of the vehicle C1 at the stopping position may be set using the learning data on the user U1 himself / herself, the situation around the user U1, and the behavior of the user U1 after the vehicle C1 has stopped, and the vehicle C1 may be stopped at the set stopping position and orientation.

[0117] The above example illustrates a case in which the stopping location of the vehicle C1 and the orientation of the vehicle C1 at the stopping location are set based on the presence or absence of a specific object associated with the user U1 and the attributes of that specific object (e.g., a baby, the size of luggage). However, the stopping location of the vehicle C1 and the orientation of the vehicle C1 at the stopping location may also be set based on the type of input (movement instruction) by the user U1 (e.g., a gesture, voice, or electronic device MC1). For example, if the user U1 issues a movement instruction by operating the electronic device MC1, it is assumed that the accuracy of the position of the user U1 and the attributes of the specific object detected based on the movement instruction information is high. Therefore, in this case, it is possible to set the stopping location of the vehicle C1 and the orientation of the vehicle C1 at the stopping location using only the position of the user U1 and the attributes of the specific object detected based on the movement instruction information. Furthermore, for example, if the user U1 issues a movement instruction using a specific gesture, a specific ringtone, or the like, it is assumed that the accuracy of the position of the user U1 and the attributes of the specific object detected based on the movement instruction is relatively low. Therefore, the position of user U1, attributes of specific objects, etc. can be sequentially detected using exterior images, etc. acquired while vehicle C1 is traveling, and the latest detected position of user U1 and the latest detected attributes of the specific object can be used to appropriately correct the stopping position of vehicle C1 and the orientation of vehicle C1 at that stopping position.

[0118] [Example of Effect of the Present Embodiment] As described above, in the present embodiment, when the user U1 issues a movement instruction for the vehicle C1 from outside the vehicle C1 using a gesture, a call, the electronic device MC1, or the like, the vehicle C1 can be automatically driven to the location of the user U1. In this case, the belongings of the user U1 who issued the movement instruction and the surrounding conditions of the user U1 (the number of people, whether or not there is a driver) can be detected, and based on the detection results, the vehicle C1 can be moved close to the user U1, stopped at a location intended by the user U1, and the doors can be automatically opened.

[0119] For example, assume that user U1 who issued the movement instruction is carrying large luggage, and the vehicle C1 is parked near the front door in front of user U1 without setting a stopping position for vehicle C1. In this case, after vehicle C1 has stopped, user U1 may walk to the back door BD1 to place the large luggage on the back door BD1, open the back door BD1 by hand, and place the large luggage on the back door BD1.

[0120] In contrast, in this embodiment, for example, as shown in FIG. 8 , if the user U1 who issued the movement instruction is carrying a large baggage BG3, the vehicle C1 is parked in a position where the back door BD1 is directly in front of the user U1, and the back door BD1 is automatically opened. This eliminates the need for the user U1 to walk to the back door BD1 and manually open the back door BD1 after the vehicle C1 has stopped. This allows the vehicle C1 to park in an appropriate location according to the user U1's intentions and automatically open the appropriate door according to the user U1's intentions. In this way, by understanding and supporting the situation of the user U1 who called the vehicle C1, it is possible to further increase the user's trust in the vehicle C1. Furthermore, movement control can be performed such that the vehicle C1 guides the user U1 to the back door BD1.

[0121] [Example of Executing Processing in Other Devices or Systems] While the above describes an example in which the reception process, detection process, setting process, determination process, control process, etc. are executed in the information processing device 100, all or part of each of these processes may be executed in other devices. In this case, an information processing system is configured by the devices that execute part of each of these processes. For example, at least part of each process may be executed using various information processing devices and various electronic devices, such as in-vehicle devices, devices available to the user (e.g., smartphones, tablet terminals, personal computers, car navigation devices, and IVIs (In-Vehicle Infotainment)), and servers connectable via a predetermined network such as the Internet.

[0122] Furthermore, a part (or all) of the information processing system capable of executing the functions of the information processing device 100 may be provided by an application that can be provided via a predetermined network such as the Internet. This application is, for example, SaaS (Software as a Service).

[0123] [Configuration Example and Effects of the Present Embodiment] The information processing method according to the present embodiment is an information processing method for controlling a vehicle to automatically travel along a travel route to a set destination. This information processing method includes a reception process (step S501) for receiving a movement instruction from a user U1 located outside the vehicle C1, a detection process (step S505) for detecting a specific object in the user U1 and at least a portion of the user U1's surroundings, and a setting process (steps S506 to S508) for setting the user U1's position as a destination and setting a stopping position at the destination based on the detection result of the specific object. Furthermore, the program according to the present embodiment is a program for causing a computer to execute each of these processes. In other words, the program according to the present embodiment is a program for causing a computer to realize each function executable by the information processing device 100.

[0124] This configuration allows the vehicle C1 to be stopped at an appropriate location according to the user U1's intention, based on the user U1's own or surrounding circumstances who issued the movement instruction. By understanding and supporting the situation of the user U1 who called the vehicle C1, the user U1's trust in the vehicle C1 can be further enhanced. Furthermore, movement control can be performed so that the vehicle C1 guides the user U1 to the intended location.

[0125] In the information processing method according to this embodiment, in the setting process (steps S506 and S507), if a specific object is detected, a stopping position is set based on the attributes of the specific object. For example, it is possible to extract a position according to the attributes of the detected specific object from the target position 162 and set it. On the other hand, in the setting process (steps S506 and S508), if a specific object is not detected, a predetermined stopping position is set based on the destination. For example, it is possible to set the stopping position of the vehicle C1 based on the destination so that the driver's seat or passenger seat is located at the position of the user U1.

[0126] According to this configuration, it is possible to stop the vehicle C1 at an appropriate position according to the attributes of the detected specific object.

[0127] The information processing method according to this embodiment further includes an acquisition process (step S505) for acquiring an image of the exterior of the vehicle C1, and in the detection process (step S505), a specific object is detected based on an image of the user U1 contained in the image and an image of the surroundings of the user U1.

[0128] According to this configuration, it is possible to detect a specific object based on an image of the outside of the vehicle C1, thereby improving the accuracy of detecting the specific object.

[0129] In the information processing method according to this embodiment, in the reception process (step S501), a movement instruction is received based on at least one of a specific movement by the user U1, a specific voice uttered by the user U1, and movement instruction information transmitted from the electronic device MC1 carried by the user U1. For example, the movement instruction information can be transmitted by selecting the vehicle call button 411 on the display screen 410 (see FIG. 4).

[0130] According to this configuration, the user U1 can give appropriate movement instructions according to the situation of the waiting location, and therefore can easily execute the movement instructions.

[0131] In the information processing method according to this embodiment, when object information about a specific object is selected by user U1's operation input, electronic device MC1 transmits movement instruction information including the object information. For example, movement instruction information including object information can be transmitted by performing a selection operation to select luggage selection area 412, baby selection area 413, or multiple passengers selection area 414 on display screen 410 (see FIG. 4). In the detection process (step S505), the specific object is detected based on the object information included in the movement instruction information.

[0132] According to this configuration, it is possible to detect a specific object based on object information selected by operation input by the user U1, thereby improving the accuracy of detecting the specific object.

[0133] In the information processing method according to this embodiment, the detection process (step S505) detects an object present in the user U1 and at least a part of the area around the user U1, and if the object is at least one of a baggage larger than a predetermined size and a person with a predetermined attribute, the object is detected as a specific object. For example, each object stored in the specific object 161 (see FIG. 6 ) of the stop position setting information DB 160 can be detected as a specific object.

[0134] According to this configuration, it is possible to set in advance a specific object that is used when setting the stopping position of the vehicle C1 and the orientation of the vehicle C1 at that stopping position.

[0135] In the information processing method according to this embodiment, the setting process (steps S506 to S508) sets the stopping position of the vehicle C1 and the orientation of the vehicle C1 at the stopping position based on the detection result of the specific object. For example, as shown in FIG. 1, if the stopping location of the vehicle C1 is determined, the orientation of the vehicle C1 according to the stopping location (e.g., the orientation with the passenger seat side facing the user U1) is set. On the other hand, as shown in FIG. 2, if the stopping location of the vehicle C1 is not determined, the orientation of the vehicle C1 is appropriately set according to the travel route from the departure point of the vehicle C1 to the stopping position.

[0136] According to this configuration, it is possible to set the orientation of the vehicle C1 at an appropriate stopping position according to the detected specific object.

[0137] In the information processing method according to this embodiment, the vehicle C1 includes an actuator 220 for automatically opening and closing each door. In the setting process (steps S506 to S508), the door that is to be automatically opened after the vehicle C1 is stopped at the stopping position is set based on the detection result of the specific object.

[0138] This configuration allows the vehicle C1 to stop at an appropriate location according to the user U1's intention, based on the user U1 who issued the movement command or the surrounding situation, and automatically open an appropriate door according to the user U1's intention. In this way, by understanding and supporting the situation of the user U1 who called the vehicle C1, it is possible to further increase the user's trust in the vehicle C1. Furthermore, it is possible to perform movement control such that the vehicle C1 guides the user U1 to the location intended by the user U1.

[0139] In the information processing method according to this embodiment, when multiple specific objects are detected in the detection process (step S505), a priority is set for each of the specific objects. For example, the priority is set according to the priority 164 (see FIG. 6 ) in the stopping position setting information DB 160. In the setting process (steps S506 to S508), the stopping position of the vehicle C1 is set based on the position of the specific object with the highest priority among the multiple specific objects.

[0140] According to this configuration, it is possible to set the priority of a specific object in advance when setting the stopping position of the vehicle C1, thereby making it possible to stop the vehicle C1 at an appropriate stopping position according to the priority of the situation of the user U1.

[0141] In the information processing method according to this embodiment, the vehicle C1 includes an actuator 220 for automatically opening and closing each door. In the setting process (steps S506 to S508), the door that is to be automatically opened after the vehicle C1 is stopped at the stopping position is set based on the specific object with the highest priority.

[0142] According to this configuration, it is possible to set the priority of a specific object in advance when setting the stopping position of the vehicle C1, thereby making it possible to set the orientation of the vehicle C1 at an appropriate stopping position according to the priority of the situation of the user U1.

[0143] In the information processing method according to this embodiment, the specific object includes users other than the user U1.

[0144] According to this configuration, when other users are waiting for the vehicle C1 together with the user U1 who issued the movement instruction, the stopping position of the vehicle C1 can be set taking into consideration the other users who are waiting together, thereby making it possible to stop the vehicle C1 at an appropriate position according to the intention of the user U1, based on the situation around the user U1 who issued the movement instruction.

[0145] The information processing method according to this embodiment further includes a determination process (step S511) for determining whether the vehicle C1 is parked at a location that matches the user U1's intention based on the relationship between the door set to be automatically opened and the door into which the specific object detected in the detection process (step S505) is placed, and a storage process (steps S512 and S514) for storing the determination results in the determination process in the storage unit 140. In the setting process (steps S506 to S508), if a specific object is detected, the vehicle C1 is parked at a location based on the determination results of the previous determination process (step S511) regarding the specific object. For example, the vehicle C1's parked location can be set based on the determination result DB 170.

[0146] According to this configuration, the stopping position of the vehicle C1 can be set in consideration of the past behavior history of the user U1, thereby making it possible to stop the vehicle C1 at an appropriate position according to the intention of the user U1 who issued the movement instruction.

[0147] The information processing device 100 is an information processing device that executes control to automatically drive a vehicle C1 along a driving route to a set destination. The information processing device 100 includes a reception unit 121 that receives a movement instruction from a user U1 located outside the vehicle C1, a detection unit 122 that detects a specific object in the user U1 and at least a part of the user U1's surroundings, and a setting unit 123 that sets the position of the user U1 as a destination and sets a stopping position at the destination based on the detection result of the specific object. Note that the information processing device 100 may be replaced by an information processing system configured with multiple devices that can execute each of the processes realized by the information processing device 100.

[0148] This configuration allows the vehicle C1 to be stopped at an appropriate location according to the user U1's intention, based on the user U1's own or surrounding circumstances who issued the movement instruction. By understanding and supporting the situation of the user U1 who called the vehicle C1, the user U1's trust in the vehicle C1 can be further enhanced. Furthermore, movement control can be performed so that the vehicle C1 guides the user U1 to the intended location.

[0149] Note that each processing procedure shown in this embodiment is an example for realizing this embodiment, and the order of some of the processing procedures may be changed within the scope that makes it possible to realize this embodiment, and some of the processing procedures may be omitted or other processing procedures may be added.

[0150] Each process in this embodiment is executed based on a program that causes a computer to execute various processing procedures. This embodiment can also be understood as an embodiment of a program that realizes the function of executing each process and a recording medium that stores the program. For example, an update process for adding a new function to an information processing device can store the program in the storage device of the information processing device. This makes it possible to cause the updated information processing device to execute each process described in this embodiment.

[0151] Although the embodiments of the present invention have been described above, the above embodiments merely show application examples of the present invention, and are not intended to limit the technical scope of the present invention to the specific configurations of the above embodiments.

Claims

1. An information processing method for controlling a vehicle to automatically travel along a route to a set destination, the information processing method including: a reception process for receiving a movement instruction from a user located outside the vehicle; a detection process for detecting a specific object within the user and at least a portion of the user's surroundings; and a setting process for setting the user's position as the destination and setting a stopping position at the destination based on the detection result of the specific object.

2. An information processing method according to claim 1, wherein, in the setting process, if the specific object is detected, the stopping position is set based on the attributes of the specific object, and if the specific object is not detected, a predetermined stopping position is set based on the destination.

3. An information processing method according to claim 1, further comprising an acquisition process for acquiring an image of the exterior of the vehicle, wherein the detection process detects the specific object based on an image of the user and an image of the user's surroundings contained in the image.

4. An information processing method according to claim 1, wherein the reception process receives the movement instruction based on at least one of a specific movement by the user, a specific voice uttered by the user, and movement instruction information transmitted from an electronic device carried by the user.

5. An information processing method according to claim 4, wherein, when object information relating to the specific object is selected by the user's operational input, the electronic device transmits the movement instruction information including the object information, and in the detection process, detects the specific object based on the object information included in the movement instruction information.

6. An information processing method according to claim 1, wherein the detection process detects an object present in the user and at least a portion of the user's surroundings, and if the object is at least one of a piece of luggage larger than a predetermined size and a person belonging to a predetermined attribute, the object is detected as the specific object.

7. An information processing method according to any one of claims 1 to 6, wherein the setting process sets the stopping position and the orientation of the vehicle at the stopping position based on the detection result of the specific object.

8. An information processing method according to any one of claims 1 to 6, wherein the vehicle is equipped with an actuator for automatically opening and closing each door, and the setting process sets the door that will automatically open after the vehicle is stopped at the stopping position based on the detection result of the specific object.

9. An information processing method according to any one of claims 1 to 6, wherein, if a plurality of specific objects are detected in the detection process, a priority is set for each of the specific objects, and in the setting process, the stopping position is set based on the position of the specific object with the highest priority among the plurality of specific objects.

10. An information processing method according to claim 9, wherein the vehicle is equipped with an actuator for automatically opening and closing each door, and the setting process sets the door that will automatically open after the vehicle is stopped at the stopping position based on the specific object with the highest priority.

11. An information processing method according to any one of claims 1 to 6, wherein the specific object includes a user other than the user.

12. An information processing method as set forth in claim 8, further comprising: a determination process for determining whether the stopping position is a position that matches the user's intention based on the relationship between a door that is set to open automatically and the door through which the specific object detected in the detection process is placed inside the vehicle; and a storage process for storing the determination result of the determination process in a storage unit, wherein in the setting process, when the specific object is detected, the stopping position is set based on the determination result of the previous determination process regarding the specific object.

13. An information processing device that controls a vehicle to automatically travel along a route to a set destination, comprising: a reception unit that receives movement instructions from a user located outside the vehicle; a detection unit that detects a specific object within the user and at least a portion of the user's surroundings; and a setting unit that sets the user's position as the destination and sets a stopping position at the destination based on the detection result of the specific object.

Images