Information processing method and information processing device

Abstract

To enable a user to designate not only a target parking position but also a target runway when the user receives parking support.SOLUTION: An information processing device according to the present embodiment is held by a user and generates a target parking position and a target runway of a vehicle from a movement locus of a three-dimensional target object whose movement and rotation are detected, on a display screen in which an image indicating a peripheral situation of the vehicle is displayed.SELECTED DRAWING: Figure 5

Description

[Technical Field]

[0001] The present invention relates to an information processing method and an information processing device. [Background technology]

[0002] Conventionally, there is known a parking assistance device that generates a movement route from the current position of the vehicle to a target parking position designated by a user to assist in parking operation. For example, Patent Document 1 listed below discloses a parking assistance device that generates a target movement route so that there is no deviation between the target parking position and the actual position of the vehicle when it is determined that parking is complete. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2014-227021 Summary of the Invention [Problem to be solved by the invention]

[0004] However, the above-mentioned conventional technology has a problem in that although the user can specify the target parking position, the user cannot specify the route (path) to the target parking position. For example, there are actually multiple "paths from the current position to the target parking position" that the vehicle can travel, but the above-mentioned conventional technology does not allow the user to select a preferred path from the multiple "paths from the current position to the target parking position."

[0005] In one aspect, the present invention has been made in consideration of the above circumstances, and its purpose is to provide an information processing method and an information processing device that allow a user to specify not only a target parking position but also a target driving route when receiving parking assistance. [Means for solving the problem]

[0006] In order to solve the above-mentioned problems, an information processing method according to one aspect of the present invention is an information processing method that causes a processor to execute a process of automatically parking a vehicle in an autonomous driving system that realizes autonomous driving of the vehicle, wherein the processor acquires a movement trajectory of a three-dimensional object that is held by a user and whose movement and rotation are detected on a display screen that displays an image showing the surrounding situation of the vehicle, generates a target parking position and a target driving route from the acquired movement trajectory, and outputs the generated target parking position and the target driving route to the autonomous driving system. [Effects of the Invention]

[0007] According to the present invention, it is possible to provide an information processing method and an information processing device that allow a user to specify not only a target parking position but also a target driving route when receiving parking assistance. [Brief explanation of the drawings]

[0008] [Figure 1] 1 is a block diagram showing a schematic configuration of a vehicle equipped with an information processing device according to an embodiment. [Figure 2] FIG. 10 is a diagram showing a situation in which a user specifies a target parking position and a target driving route for a vehicle by moving and rotating a miniature car on a display. [Figure 3] 2 illustrates a schematic example of a hardware configuration of an information processing device according to an embodiment. [Figure 4] 2 illustrates a schematic example of a software configuration of an information processing device according to an embodiment. [Figure 5] 1 illustrates an example of a processing procedure of an information processing device according to an embodiment. [Figure 6] This figure shows a situation in which a user attempts to automatically park a vehicle at a target parking position on a target driving path by moving and rotating a miniature car on a display with reference to a possible trajectory. [Figure 7] FIG. 10 is a diagram showing a situation in which a user stops a vehicle by stopping the movement and rotation of a miniature car on a display. [Figure 8]10A and 10B are diagrams illustrating a guide force that occurs when the movement trajectory of the miniature car deviates from the possible trajectory selected by the user. [Figure 9] 10A and 10B are diagrams illustrating the guide force that occurs when the movement trajectory of the miniature car does not correspond to a feasible running path for the vehicle. DETAILED DESCRIPTION OF THE INVENTION

[0009] An embodiment according to one aspect of the present invention (hereinafter also referred to as "the present embodiment") will be described below with reference to the drawings. However, the present embodiment described below is merely an example of the present invention in all respects. Needless to say, various improvements and modifications can be made without departing from the scope of the present invention. In other words, when implementing the present invention, specific configurations according to the embodiment may be appropriately adopted. Note that, although data appearing in the present embodiment are described in natural language, more specifically, they are specified using pseudo-language, commands, parameters, machine language, etc. that can be recognized by a computer.

[0010] §1 Application Examples FIG. 1 is a block diagram showing a schematic configuration of a vehicle VH equipped with an information processing device (information processing device 10) according to this embodiment. In this embodiment, the vehicle VH is configured to perform autonomous driving. For example, by autonomously driving along a "target driving path TC specified by a user Us," the vehicle VH can automatically park at a "target parking position TP specified by the user Us." Autonomous driving is also referred to as automatic driving. The vehicle VH may be a so-called automatic driving vehicle, and in particular, may be a fully automatic driving (unmanned driving) vehicle. However, the vehicle VH may also be a vehicle that is capable of switching between automatic driving, including automatic parking, and manual driving, including manual parking. The vehicle VH illustrated in FIG. 1 includes an information processing system 1 including the information processing device 10, a surrounding situation detection device 2, and an autonomous driving system 3.

[0011] The autonomous driving system 3 is a system that realizes autonomous driving (automated driving) of the vehicle VH, and controls, for example, the acceleration, steering, and deceleration (braking) of the vehicle VH. The autonomous driving system 3 may be realized by a so-called vehicle control ECU (Electronic Control Unit). In this embodiment, the autonomous driving system 3 can realize automatic parking of the vehicle VH by communicating with the information processing system 1 (mutually transmitting and receiving information). For example, the information processing system 1 outputs a signal (target signal TS) indicating a target parking position TP and a target route TC to the autonomous driving system 3. The autonomous driving system 3 can automatically park the vehicle VH at the target parking position TP along the target route TC in accordance with the "target signal TS indicating the target parking position TP and the target route TC" output from the information processing system 1. The autonomous driving system 3 and the information processing system 1 may be connected, for example, by a controller area network (CAN) or other in-vehicle LAN, and may transmit and receive information to and from each other.

[0012] The surrounding situation detection device 2 detects (senses) and acquires surrounding information NI indicating the surrounding situation of the vehicle VH, and acquires at least a captured image of the surrounding situation of the vehicle VH. The surrounding situation detection device 2 outputs the detected and acquired surrounding information NI to the information processing system 1, and outputs at least a captured image of the surrounding situation of the vehicle VH to the information processing system 1. In other words, the "surrounding information NI" detected, acquired, and output by the surrounding situation detection device 2 to the information processing system 1 includes "a captured image of the surrounding situation of the vehicle VH." The surrounding situation detection device 2 and the information processing system 1 may be connected, for example, by an in-vehicle LAN, and may transmit and receive information to and from each other.

[0013] The surrounding situation detection device 2 includes at least one camera (image capture device). The surrounding situation detection device 2 may further include an object detection device, a distance measurement device, a communication device, etc. The surrounding situation detection device 2 can detect objects (targets) around (around) the vehicle VH, and can detect the position, attitude (yaw angle), size, speed, acceleration, deceleration, and yaw rate of the objects relative to the vehicle VH. The surrounding situation detection device 2 can detect, for example, the movement (motion) of the objects. Objects detected by the surrounding situation detection device 2 include moving objects such as other vehicles, motorcycles, bicycles, and pedestrians. Objects detected by the surrounding situation detection device 2 also include stationary objects such as lane boundaries, center lines, road markings, medians, guardrails, curbs, highway sidewalls, road signs, traffic lights, crosswalks, construction sites, accident sites, traffic restrictions, parked vehicles, and buildings. Furthermore, the surrounding situation detection device 2 can detect the state and situation of the road and lane on which the vehicle VH is traveling, and can detect, for example, the position of white lines indicating parking spaces around the vehicle VH. The information detected (acquired) by the surrounding situation detection device 2 is output to the information processing system 1, for example, periodically (periodically), as surrounding information NI indicating the surrounding situation of the vehicle VH.

[0014] The camera included in the surrounding situation detection device 2 is a device that recognizes the surrounding situation of the vehicle VH, for example, objects (targets) around the vehicle VH, from captured images of the surrounding situation of the vehicle VH. Such a camera may be, for example, a camera equipped with an imaging element such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor), an ultrasonic camera, an infrared camera, or the like. A plurality of such cameras may be provided on the vehicle VH, and may be located, for example, near the front grille, under the left and right door mirrors, and near the rear bumper of the vehicle VH.

[0015] In this embodiment, the user Us uses a minicar 20 (a three-dimensional object) to specify a target parking position TP and a target route TC for the vehicle VH. The user Us also uses the minicar 20 to control the traveling of the vehicle VH during automatic parking, for example, to stop the traveling, control the traveling speed, or control (correct) the traveling route. Specifically, as illustrated in FIG. 2, the user Us moves and rotates the minicar 20 held by the user Us on a display 410 (display screen) that displays an image IS showing the surrounding situation of the vehicle VH, thereby specifying the target parking position TP and the target route TC and controlling the traveling of the vehicle VH.

[0016] The information processing system 1 identifies a target parking position TP and a target route TC specified by the user Us from the movement trajectory MT of the minicar 20 on the display 410, and outputs a target signal TS indicating the identified target parking position TP and target route TC to the autonomous driving system 3. The information processing system 1 illustrated in FIG. 1 includes an information processing device 10, a minicar 20, a minicar control unit 30, and a display device 40.

[0017] The miniature car 20 is a three-dimensional object held by the user Us, and may be, for example, a miniature (small model) that resembles the shape of the vehicle VH. The miniature car 20 may have magnetic properties. The movement and rotation of the miniature car 20 on the display 410 of the display device 40 is detected by the miniature car control unit 30. The movement of the miniature car 20 refers to, for example, a change (alteration) of the position of the miniature car 20 (for example, a change in the center position). The rotation of the miniature car 20 includes a change (alteration) of the orientation of the miniature car 20 without a change in the position of the miniature car 20. The miniature car 20 differs from a touch pen, which detects only the movement but not the rotation (a change in orientation without a change in position), in that both the movement and the rotation are detected. The user Us can grasp the movement and rotation of the miniature car 20, which is a three-dimensional object, not only visually but also tactilely. In particular, if the tires of the minicar 20 are configured to be rotatable, for example, like the tires of the vehicle VH, when the user Us attempts to move or rotate (change direction of) the minicar 20 in a direction in which the tires of the minicar 20 cannot rotate, the following force acts on the user Us. That is, a physical force that restricts such movement or rotation acts on the user Us (particularly the hand of the user U that is holding the minicar 20). Therefore, the user Us can grasp the movement and rotation of the minicar 20 by touch, without having to see the minicar 20.

[0018] The minicar control unit 30 detects the movement and rotation of the minicar 20 on the display 410, and for example, detects the position and direction of the minicar 20 (for example, the longitudinal direction of the minicar 20) at each of a plurality of points in time. The minicar control unit 30 detects (specifies) the movement and rotation of the minicar 20 from the position and direction of the minicar 20 at each point in time, and may detect, for example, the amount of movement and amount of change in direction of the minicar 20 per unit time. The minicar control unit 30 specifies a movement trajectory MT of the minicar 20 on the display 410 from the detected "position and direction of the minicar 20 (the amount of movement and amount of change in direction of the minicar 20 per unit time)" and outputs the specified movement trajectory MT to the information processing device 10.

[0019] The minicar control unit 30 also generates a physical force (guide force GF) that controls the movement and rotation of the minicar 20 on the display 410. In other words, it generates a guide force GF that guides (induces) the movement and rotation of the minicar 20 by the user Us. For example, the minicar control unit 30 may use the guide force GF to generate or control a magnetic field to control the movement and rotation of the magnetically-charged minicar 20 and guide the movement and rotation of the minicar 20 by the user Us. Specifically, the minicar control unit 30 may perform such guidance by controlling the respective positions of one or more magnets (or coils) provided inside the display 410. The minicar control unit 30 may also perform such guidance by controlling the power supply to each of multiple coils provided inside the display 410. However, the above example is merely one example of the guide force GF generated by the minicar control unit 30. The minicar control unit 30 can generate the guide force GF that controls the movement and rotation of the minicar 20 on the display 410 by any method.

[0020] The minicar control unit 30 generates a guide force GF in accordance with instructions (guide signal GS) from the information processing device 10, and guides the user Us in moving and rotating the minicar 20. The guide signal GS will be described in detail later. The minicar control unit 30 may also generate a force (physical force) that fixes the minicar 20 on the display 410.

[0021] The display device 40 includes a display 410 and a display control unit 420 that controls images to be displayed on the display 410. The display control unit 420 displays an image IS showing the surrounding conditions of the vehicle VH on the display 410. For example, the display control unit 420 generates an image IS by synthesizing and correcting images captured by one or more cameras included in the surrounding condition detection device 2 to capture the surrounding conditions of the vehicle VH, and displays the generated image IS on the display 410. The display control unit 420 may also display one or more possible trajectories AT, etc., generated by the information processing device 10 on the display 410. The feasible trajectories AT will be described in detail below.

[0022] The information processing device 10 controls the acceleration, deceleration (braking), steering, etc. of the vehicle VH in accordance with the operation (at least one of movement and rotation) of the minicar 20 by the user Us on the display 410 of the display device 40. In particular, the information processing device 10 generates a target parking position TP and a target route TC for the vehicle VH from the "movement trajectory MT of the minicar 20 on the display 410 displaying the image IS showing the surrounding situation of the vehicle VH" identified by the minicar control unit 30. The information processing device 10 then outputs the generated target parking position TP and target route TC to the autonomous driving system 3, thereby causing the autonomous driving system 3 to automatically park the vehicle VH at the target parking position TP along the target route TC. In this embodiment, the information processing device 10 outputs a target signal TS indicating the target parking position TP and the target route TC to the autonomous driving system 3, thereby causing the autonomous driving system 3 to perform the above-mentioned automatic parking. Therefore, when the user Us receives parking assistance for the vehicle VH, for example, when the vehicle VH is automatically parked, the information processing device 10 enables the user Us to specify not only the target parking position TP but also the target driving path TC. The information processing system 1, the overview of which has been explained above, and in particular the information processing device 10, will be explained in detail below with reference to Figures 3 to 9.

[0023] §2 Configuration example [Hardware configuration] Fig. 3 schematically illustrates an example of the hardware configuration of an information processing device 10 according to this embodiment. As shown in Fig. 3, the information processing device 10 according to this embodiment is a computer to which a control unit 11, a storage unit 12, a communication interface 13, an external interface 14, an input device 15, an output device 16, and a drive 17 are electrically connected. In Fig. 3, the communication interface and the external interface are referred to as a "communication I / F" and an "external I / F."

[0024] The control unit 11 includes a hardware processor such as a central processing unit (CPU), a random access memory (RAM), and a read-only memory (ROM), and is configured to execute information processing based on programs and various data. The CPU is an example of a processor resource. The storage unit 12 is an example of a memory resource, and is configured, for example, with a hard disk drive or a solid-state drive. In this embodiment, the storage unit 12 stores various information such as an information processing program PG and vehicle information VI.

[0025] The information processing program PG is a program for causing the information processing device 10 to execute information processing (FIG. 5) described below, which causes the autonomous driving system 3 to automatically park the vehicle VH (in particular, causes the vehicle VH to automatically park at the target parking position TP on the target driving path TC). The information processing program PG includes a series of instructions for the information processing.

[0026] The vehicle information VI includes information for calculating (identifying) the routes along which the vehicle VH can travel (move), and in particular, information for identifying the directions in which the vehicle VH can travel. For example, the vehicle information VI includes information related to the axles of the vehicle VH (for example, the wheelbase length) and information related to the maximum steering angle of the vehicle VH.

[0027] The communication interface 13 is, for example, a wired LAN (Local Area Network) module, a wireless LAN module, or the like, and is an interface for performing wired or wireless communication via a network. As described above, the communication interface 13 may be an interface for performing communication via a CAN or other in-vehicle LAN. The information processing device 10 may use this communication interface 13 to perform data communication via a network with other information processing devices (e.g., the surrounding condition detection device 2, the autonomous driving system 3, the minicar control unit 30, the display device 40, etc.). The external interface 14 is, for example, a USB (Universal Serial Bus) port, a dedicated port, or the like, and is an interface for connecting to an external device. The type and number of external interfaces 14 may be selected appropriately depending on the type and number of external devices to be connected.

[0028] For example, the information processing device 10 is connected to another information processing device via at least one of the communication interface 13 and the external interface 14, and performs the following communications with the other information processing device. That is, the information processing device 10 is connected to the surrounding situation detection device 2 and acquires surrounding information NI from the surrounding situation detection device 2. The information processing device 10 is connected to the autonomous driving system 3 and outputs a target signal TS indicating a target parking position TP and a target driving path TC to the autonomous driving system 3. The information processing device 10 is connected to the minicar control unit 30 and (1) acquires a movement trajectory MT of the minicar 20 on the display 410 from the minicar control unit 30 and (2) outputs a guide signal GS to the minicar control unit 30. The information processing device 10 is connected to the display device 40 and causes the display device 40 to display one or more possible trajectories AT.

[0029] The input device 15 is a device for inputting information, such as a mouse or a keyboard. The output device 16 is a device for outputting information, such as a display or a speaker. An operator such as a user can operate the information processing device 10 by using the input device 15 and the output device 16. The output device 16 may be the display 410 of the above-mentioned display device 40, and for example, the information processing device 10 may include the display 410 as the output device 16. However, in the following, to facilitate understanding of the information processing device 10, an example will be described in which the output device 16 and the display 410 are realized as separate devices or hardware.

[0030] The drive 17 is, for example, a CD drive, a DVD drive, or the like, and is a drive device for reading various information, such as programs, stored in a storage medium 91. The storage medium 91 is a medium that stores information, such as programs, electrically, magnetically, optically, mechanically, or chemically, so that a computer or other device, machine, or the like can read the stored information. At least one of the information processing program PG and the vehicle information VI may be stored in the storage medium 91. The information processing device 10 may acquire at least one of the information processing program PG and the vehicle information VI from the storage medium 91. Note that FIG. 3 illustrates a disk-type storage medium, such as a CD or a DVD, as an example of the storage medium 91. However, the type of the storage medium 91 is not limited to a disk-type medium and may be other than a disk-type medium. Examples of storage media other than a disk-type medium include semiconductor memories, such as flash memories. The type of the drive 17 may be selected arbitrarily depending on the type of the storage medium 91.

[0031] Note that, with regard to the specific hardware configuration of the information processing device 10, components may be omitted, replaced, or added as appropriate depending on the embodiment. For example, the processor resources may include multiple hardware processors. The hardware processor may be configured with a microprocessor, an FPGA (field-programmable gate array), a DSP (digital signal processor), or the like. The storage unit 12 may be configured with RAM and ROM included in the control unit 11. At least one of the communication interface 13, the external interface 14, the input device 15, the output device 16, and the drive 17 may be omitted. The information processing device 10 may be configured with multiple computers. In this case, the hardware configurations of the computers may or may not be identical. Furthermore, the information processing device 10 may be an information processing device designed specifically for the service provided, as well as a general-purpose server device, a PC (Personal Computer), or the like.

[0032] [Software configuration] FIG. 4 schematically illustrates an example of the software configuration of the information processing device 10 according to this embodiment. The control unit 11 of the information processing device 10 loads the information processing program PG stored in the storage unit 12 into RAM. The control unit 11 then uses the CPU to interpret and execute instructions included in the information processing program PG loaded into RAM to control each component. As a result, as shown in FIG. 4, the information processing device 10 according to this embodiment operates as a computer including a movement trajectory acquisition unit 110, a conversion unit 120, a possibility determination unit 130, a surrounding situation recognition unit 140, a display trajectory etc. generation unit 150, and a guide control unit 160 as software modules. That is, in this embodiment, each software module of the information processing device 10 is realized by the control unit 11 (CPU).

[0033] The movement trajectory acquisition unit 110 acquires a movement trajectory MT of the minicar 20 on the display 410 from the minicar control unit 30. The movement trajectory acquisition unit 110 outputs the acquired movement trajectory MT to the conversion unit 120. The movement trajectory MT of the minicar 20 may include information indicating the position and direction of the minicar 20 on the display 410 at each of a plurality of points in time.

[0034] The conversion unit 120 generates a target parking position TP and a target route TC for the vehicle VH from the movement trajectory MT acquired by the movement trajectory acquisition unit 110. For example, the conversion unit 120 may generate the target parking position TP and the target route TC from the movement trajectory MT by converting the movement trajectory MT of the minicar 20 into the target route TC for the vehicle VH, which includes the target parking position TP for the vehicle VH. The conversion unit 120 outputs the generated target parking position TP and target route TC to the feasibility determination unit 130.

[0035] The conversion unit 120 may identify the "movement amount and directional change amount of the minicar 20 per unit time" from the "position and direction of the minicar 20 on the display 410 at each of multiple points in time" indicated by the movement trajectory MT. The conversion unit 120 may then generate the "target travel amount TM and target directional change amount TA of the vehicle VH per unit time" from the identified "movement amount and directional change amount of the minicar 20 per unit time." For example, the conversion unit 120 may convert the "movement amount of the minicar 20 per unit time" into the "target travel amount TM of the vehicle VH per unit time." The conversion unit 120 may also convert the "directional change amount of the minicar 20 per unit time" into the "target directional change amount TA of the vehicle VH per unit time." The "target travel distance TM per unit time of the vehicle VH" indicates, for example, the target value of the control amount (control amount per unit time) of at least one of the drive device (accelerator) and braking device (brake) of the vehicle VH. Furthermore, the "target directional change amount TA per unit time of the vehicle VH" indicates, for example, the target value of the control amount (control amount per unit time) of the steering device (steering) of the vehicle VH. The conversion unit 120 may output the generated "target travel distance TM and target directional change amount TA per unit time of the vehicle VH" to the feasibility determination unit 130.

[0036] The feasibility determination unit 130 is an example of the "output unit" of the present invention. The feasibility determination unit 130 determines whether the target parking position TP and the target route TC generated by the conversion unit 120 are feasible in the following two points: That is, the feasibility determination unit 130 determines whether the target parking position TP and the target route TC are feasible in accordance with a first condition related to the vehicle VH (a condition based on the structure, settings, etc. of the vehicle VH) and a second condition related to the surrounding conditions of the vehicle VH (a condition based on the surrounding conditions of the vehicle VH).

[0037] When it is determined that the target parking position TP and the target route TC are feasible for the first and second conditions, the feasibility determination unit 130 outputs a target signal TS indicating the target parking position TP and the target route TC to the autonomous driving system 3. For example, the feasibility determination unit 130 outputs to the autonomous driving system 3 the target signal TS including command values ​​(control variables) related to the steering angle, speed, etc. of the vehicle VH for automatically parking the vehicle VH at the target parking position TP on the target route TC.

[0038] When it is determined that the target parking position TP and the target traveling path TC are feasible for the first and second conditions, the feasibility determination unit 130 may output, as a target signal TS, a command value (control amount) that realizes the "target traveling amount TM per unit time and target directional change amount TA" generated by the conversion unit 120 to the autonomous traveling system 3. The autonomous traveling system 3 can realize (control) the "target traveling amount TM per unit time" and the "target directional change amount TA per unit time" for the vehicle VH in accordance with the target signal TS.

[0039] When it is determined that the target parking position TP and the target route TC are not feasible for at least one of the first condition and the second condition, the feasibility determination unit 130 outputs an instruction to stop the movement of the vehicle VH (for example, a signal instructing braking of the vehicle VH) to the autonomous driving system 3. Furthermore, when it is determined that the target parking position TP and the target route TC are not feasible for at least one of the first condition and the second condition, the feasibility determination unit 130 notifies the result of the determination, for example, to the display trajectory etc. generation unit 150 and the guide control unit 160. Although details will be described later, if the target parking position TP and the target route TC generated from the movement trajectory MT of the minicar 20 are not feasible, the information processing device 10 performs the following process. That is, the information processing device 10 guides (instructs) the user Us to perform at least one of "stopping at least one of the movement and rotation of the minicar 20" and "changing (modifying) at least one of the movement and rotation of the minicar 20." For example, the information processing device 10 guides the user Us in operating the minicar 20 so that the target parking position TP and target running path TC generated from the movement trajectory MT satisfy the first and second conditions described above. By following this guidance, the user Us can easily move and rotate the minicar 20 so that the target parking position TP and target running path TC generated from the movement trajectory MT satisfy the first and second conditions. The information processing device 10 may perform the above-mentioned guidance by generating a guide force GF and notifying the user Us of information (warning) indicating that the movement and rotation of the minicar 20 are inappropriate. The first and second conditions will be described in detail below.

[0040] (First condition) As a first condition, the feasibility determination unit 130 determines whether the target route TC is a route on which the vehicle VH can travel (move), in other words, whether the target route TC is a route that is feasible for the vehicle VH. For example, the feasibility determination unit 130 refers to the storage unit 12 to acquire vehicle information VI, and identifies "routes on which the vehicle VH can travel" from the maximum values ​​of the axles and steering angles of the vehicle VH indicated by the acquired vehicle information VI. Then, the feasibility determination unit 130 determines whether the target route TC is included in the "routes on which the vehicle VH can travel" identified using the vehicle information VI.

[0041] If the target path TC is included in the "paths on which the vehicle VH can travel", the feasibility determination unit 130 determines that the target parking position TP and the target path TC are feasible for the first condition. If the target path TC is not included in the "paths on which the vehicle VH can travel", the feasibility determination unit 130 determines that the target parking position TP and the target path TC are not feasible for the first condition, and notifies the display trajectory etc. generation unit 150 and the guide control unit 160 of the determination result. If the target path TC is not included in the "paths on which the vehicle VH can travel", the information processing device 10 executes the following process. That is, the information processing device 10 prompts the user Us to change the movement trajectory MT so that the target path TC generated from the movement trajectory MT becomes a path on which the vehicle VH can travel, and guides (directs) the movement and rotation of the minicar 20 by the user Us.

[0042] As a first condition, the feasibility determination unit 130 may determine whether the "amount of change in target direction TA per unit time of the vehicle VH" is within the range of directions in which the vehicle VH can travel. For example, the feasibility determination unit 130 identifies the directions in which the vehicle VH can travel (the range of such directions) from the maximum values ​​of the axle and steering angle of the vehicle VH indicated by the vehicle information VI. Then, the feasibility determination unit 130 determines whether the "amount of change in target direction TA per unit time of the vehicle VH" is within the identified "range of directions in which the vehicle VH can travel."

[0043] If the "target direction change amount TA of the vehicle VH per unit time" is within the identified "range of directions in which the vehicle VH can travel," the feasibility determination unit 130 determines that the target parking position TP and the target route TC are feasible for the first condition. If the target direction change amount TA is within the "range of directions in which the vehicle VH can travel," the feasibility determination unit 130 outputs a command value that commands a "steering control amount" corresponding to the target direction change amount TA as a target signal TS to the autonomous driving system 3. In other words, the feasibility determination unit 130 generates a "target direction change amount TA per unit time" (a command value corresponding to the "target direction change amount TA per unit time") to be output to the autonomous driving system 3 within the "range of directions in which the vehicle VH can travel" identified using the vehicle information VI. Therefore, the autonomous driving system 3 can control the steering of the vehicle VH in accordance with the "target direction change amount TA per unit time."

[0044] If the target direction change amount TA is not within the "range of directions in which the vehicle VH can travel," the feasibility determination unit 130 determines that the target parking position TP and the target route TC are not feasible for the first condition. If the target direction change amount TA is not within the "range of directions in which the vehicle VH can travel," the information processing device 10 guides (instructs) the user Us to move and rotate the minicar 20 so that the target direction change amount TA is included in the "range of directions in which the vehicle VH can travel."

[0045] (Second condition) As a second condition, the feasibility determination unit 130 determines whether "the vehicle VH approaches an object other than the vehicle VH (hereinafter also referred to as an "obstacle Ob") within a predetermined distance DT." For example, when the vehicle VH is automatically parked on the target traveling path TC to the target parking position TP, the feasibility determination unit 130 determines whether the vehicle VH approaches the obstacle Ob within a predetermined distance DT. Furthermore, when the vehicle VH is driven and the direction of the vehicle VH is changed in accordance with the operation (movement, rotation) of the minicar 20 by the user Us, the feasibility determination unit 130 determines whether the vehicle VH approaches the obstacle Ob within a predetermined distance DT. The "obstacle Ob" may be rephrased as, for example, an object other than the vehicle VH that may become an obstacle during automatic parking of the vehicle VH. The possibility determination unit 130 determines, from the "position of the obstacle Ob, etc." notified by the surrounding situation recognition unit 140, whether the distance between the vehicle VH and the obstacle Ob is equal to or less than a predetermined distance DT (whether it will become equal to or less than the predetermined distance DT).

[0046] If the distance between the vehicle VH and the obstacle Ob is greater than the predetermined distance DT, the feasibility determination unit 130 determines that the target parking position TP and the target route TC are feasible for the second condition. If the distance between the vehicle VH and the obstacle Ob is equal to or less than the predetermined distance DT (becomes equal to or less than the predetermined distance DT), the feasibility determination unit 130 determines that the target parking position TP and the target route TC are not feasible for the second condition. If the distance between the vehicle VH and the obstacle Ob becomes equal to or less than the predetermined distance DT, the information processing device 10 may guide (instruct) the user Us to "stop at least one of the movement and rotation of the minicar 20." Furthermore, if the distance between the vehicle VH and the obstacle Ob is equal to or less than the predetermined distance DT, the information processing device 10 may instruct the user Us to "change (correct) at least one of the movement and rotation of the minicar 20." For example, the information processing device 10 may guide the user Us to change (correct) the movement trajectory MT of the minicar 20 so that the distance between the vehicle VH and the obstacle Ob becomes greater than a predetermined distance DT.

[0047] The surrounding situation recognition unit 140 acquires surrounding information NI indicating the surrounding situation of the vehicle VH from the surrounding situation detection device 2, and in particular acquires captured images of the surrounding situation of the vehicle VH by each of one or more cameras included in the surrounding situation detection device 2. The surrounding situation recognition unit 140 recognizes the surrounding situation of the vehicle VH from the acquired surrounding information NI (one or more captured images), and recognizes (detects), for example, the position of an obstacle Ob that may be an obstacle during automatic parking of the vehicle VH. The surrounding situation recognition unit 140 may recognize, as obstacles Ob, objects present around the target parking position TP of the vehicle VH (e.g., vehicles other than the vehicle VH), objects on the target traveling path TC of the vehicle VH, and objects present around the target traveling path TC (e.g., road cones, walls, etc.). The surrounding situation recognition unit 140 may recognize people (e.g., pedestrians approaching the vehicle VH or attempting to cross the target path TC) present on the target path TC of the vehicle VH as obstacles Ob. The surrounding situation recognition unit 140 outputs the recognized "surrounding situation of the vehicle VH (particularly, the position of the obstacle Ob, etc.)" to the feasibility determination unit 130 and the display trajectory etc. generation unit 150.

[0048] The display trajectory etc. generation unit 150 generates various images to be superimposed on the image IS showing the surrounding conditions of the vehicle VH displayed on the display 410 of the display device 40. The display trajectory etc. generation unit 150 generates, for example, a trajectory (possible trajectory AT) that serves as a reference when the user Us moves and rotates the minicar 20 as an image to be superimposed on the image IS. The display trajectory etc. generation unit 150 also generates, for example, a trajectory (executed trajectory RT) corresponding to the "path along which the vehicle VH is actually traveling while performing automatic parking" as an image to be superimposed on the image IS. As described above, the information processing device 10 controls the acceleration, deceleration (braking), steering, etc. of the vehicle VH in accordance with the operation (movement and rotation) of the minicar 20 by the user Us, thereby causing the vehicle VH to perform automatic parking. Therefore, the display trajectory etc. generation unit 150 displays the executed trajectory RT corresponding to the "path along which the vehicle VH is traveling while performing the automatic parking" superimposed on the image IS. Furthermore, the display trajectory etc. generation unit 150 generates an image to be superimposed on the image IS to guide the user Us in their operation on the minicar 20 when the operation by the user Us on the minicar 20 is inappropriate. For example, when at least one of the first condition and the second condition described above is not satisfied, the display trajectory etc. generation unit 150 superimposes an image to guide the operation by the user Us on the image IS so that both the first condition and the second condition are satisfied. In the example shown in FIG. 4, the display trajectory etc. generation unit 150 includes a possible trajectory generation unit 152 and an actual trajectory generation unit 154.

[0049] The feasible trajectory generation unit 152 generates one or more feasible trajectories AT. For example, the feasible trajectory generation unit 152 identifies the current position of the vehicle VH (e.g., the position of the vehicle VH before automatic parking is performed) from an image IS showing the surrounding conditions of the vehicle VH, and sets a target parking position TP. The feasible trajectory generation unit 152 may set the target parking position TP based on an operation by the user Us to specify the target parking position TP. The feasible trajectory generation unit 152 may set the target parking position TP to the position where the vehicle VH can be parked that is closest to the current position of the vehicle VH. Next, the feasible trajectory generation unit 152 acquires vehicle information VI by referring to the storage unit 12. Furthermore, the feasible trajectory generation unit 152 acquires information showing the surrounding conditions of the vehicle VH (e.g., the position of an obstacle Ob) from the surrounding condition recognition unit 140. Then, the feasible trajectory generation unit 152 identifies one or more routes that the vehicle VH can take from its current position to the target parking position TP, based on the maximum values ​​of the axle and steering angles of the vehicle VH indicated in the acquired vehicle information VI. Furthermore, the feasible trajectory generation unit 152 uses the "position of the obstacle Ob" notified by the surrounding situation recognition unit 140 to identify the following "one or more routes that the vehicle VH can take from its current position to the target parking position TP" from the above-mentioned "one or more routes that the vehicle VH can take from its current position to the target parking position TP." In other words, the feasible trajectory generation unit 152 identifies "one or more routes that the vehicle VH can take from its current position to the target parking position TP" on which the distance between the obstacle Ob and the vehicle VH is greater than a predetermined distance DT. That is, the feasible trajectory generation unit 152 identifies a route from the current position of the vehicle VH to the target parking position TP that satisfies the above-mentioned first condition (the vehicle VH is drivable) and the second condition (the vehicle VH does not approach the obstacle Ob within a predetermined distance DT). The feasible trajectory generation unit 152 generates one or more feasible trajectories AT from the identified "one or more routes that the vehicle VH can take from the current position of the vehicle VH to the target parking position TP." For example, the feasible trajectory generation unit 152 generates one or more feasible trajectories AT by converting each of the one or more "routes that the vehicle VH can take from the current position of the vehicle VH to the target parking position TP" into one or more feasible trajectories AT.

[0050] The execution trajectory generation unit 154 generates an execution trajectory RT corresponding to the "path along which the vehicle VH is traveling while automatically parking." For example, the execution trajectory generation unit 154 acquires information relating to the steering angle, speed, position, etc. of the vehicle VH while automatically parking from the autonomous driving system 3 that executes the automatic parking of the vehicle VH. Then, the execution trajectory generation unit 154 identifies the "path along which the vehicle VH is traveling while automatically parking" from the acquired information relating to the steering angle, speed, position, etc. of the vehicle VH. The execution trajectory generation unit 154 generates the execution trajectory RT from the identified "path along which the vehicle VH is traveling while automatically parking," and generates the execution trajectory RT by, for example, converting the "path along which the vehicle VH is traveling while automatically parking" into the execution trajectory RT.

[0051] The guide control unit 160 outputs a guide signal GS to the minicar control unit 30, causing the minicar control unit 30 to generate a physical force (guide force GF) that controls the movement and rotation of the minicar 20 on the display 410.

[0052] For example, if the feasibility determination unit 130 determines that "the target route TC does not satisfy the first condition (the vehicle VH is drivable (a drivable route)")," the guide control unit 160 causes the minicar control unit 30 to generate the following guide force GF. That is, the guide control unit 160 generates a guide force GF that guides the movement and rotation of the minicar 20 by the user Us and encourages a change in the movement trajectory MT so that the target route TC of the vehicle VH generated from the movement trajectory MT of the minicar 20 becomes a route on which the vehicle VH is drivable. For example, the guide control unit 160 generates a guide force GF that guides the movement and rotation of the minicar 20 by the user Us so that the movement trajectory MT of the minicar 20 follows (e.g., coincides with) "any of one or more possible trajectories AT" generated by the possible trajectory generation unit 152. As described above, the possible trajectory generation unit 152 identifies "one or more routes on which the vehicle VH can travel" based on the maximum values ​​of the axle and steering angles of the vehicle VH, and generates one or more possible trajectories AT from the identified "one or more routes on which the vehicle VH can travel (more precisely, feasible)." The guide control unit 160 then generates a guide force GF that guides the movement and rotation of the minicar 20 by the user Us so that the movement trajectory MT of the minicar 20 follows one of the one or more possible trajectories AT. For example, the guide control unit 160 generates a guide force GF that guides the movement and rotation of the minicar 20 by the user Us so that the movement trajectory MT follows the possible trajectory AT selected by the user from among the one or more possible trajectories AT. The guide control unit 160 may consider the possible trajectory AT closest to the movement trajectory MT (or the executed trajectory RT) of the minicar 20 as the "possible trajectory AT selected by the user."

[0053] For example, if the possibility determination unit 130 determines that the above-mentioned second condition (the vehicle VH does not approach the obstacle Ob within the predetermined distance DT) is not satisfied, the guide control unit 160 generates the following guide force GF. That is, the guide control unit 160 generates a guide force GF that prompts the user Us to at least one of "stop at least one of the movement and rotation of the minicar 20" and "change (correct) the movement trajectory MT of the minicar 20."

[0054] When the distance between the vehicle VH and the obstacle Ob becomes equal to or less than the predetermined distance DT, the guide control unit 160 may generate a guide force GF that stops the movement and rotation of the minicar 20 by the user Us, i.e., a guide force GF that fixes the position and direction of the minicar 20. As described above, the information processing device 10 controls the acceleration, deceleration (braking), steering, etc. of the vehicle VH in accordance with the operation (at least one of movement and rotation) of the minicar 20 by the user Us on the display 410 of the display device 40. Therefore, when the distance between the vehicle VH and the obstacle Ob becomes equal to or less than the predetermined distance DT, the guide control unit 160 generates a guide force GF that fixes the position and direction of the minicar 20 on the display 410, i.e., a guide force GF that stops the vehicle VH from traveling and changing direction. By generating such a guide force GF, the information processing device 10 can prevent the vehicle VH from coming into contact with the obstacle Ob by continuing to travel and change direction when the distance between the vehicle VH and the obstacle Ob becomes equal to or less than the predetermined distance DT.

[0055] If the distance between the vehicle VH and the obstacle Ob is equal to or less than the predetermined distance DT, the guide control unit 160 may generate a guide force GF that guides the user Us in moving and rotating the minicar 20 and prompts the user Us to change the movement trajectory MT so that the distance between the vehicle VH and the obstacle Ob becomes greater than the predetermined distance DT. For example, the guide control unit 160 generates a guide force GF that guides the user Us in moving and rotating the minicar 20 so that the movement trajectory MT of the minicar 20 follows "one of one or more possible trajectories AT" generated by the feasible trajectory generation unit 152. As described above, the feasible trajectory generation unit 152 identifies, from among "one or more routes on which the vehicle VH can travel," routes on which the distance between the obstacle Ob and the vehicle VH is greater than the predetermined distance DT as "one or more routes on which the vehicle VH can travel." Then, the feasible trajectory generation unit 152 generates one or more possible trajectories AT from the identified "one or more routes on which the vehicle VH can travel." Therefore, the guide control unit 160 generates a guide force GF that guides the movement and rotation of the minicar 20 by the user Us so that the movement trajectory MT of the minicar 20 follows one of the one or more possible trajectories AT. For example, the guide control unit 160 generates a guide force GF that guides the movement and rotation of the minicar 20 by the user Us so that the movement trajectory MT follows the possible trajectory AT selected by the user from one or more possible trajectories AT. The guide control unit 160 may consider the possible trajectory AT that is closest to the movement trajectory MT (or the execution trajectory RT) of the minicar 20 as the "possible trajectory AT selected by the user."

[0056] As described above, the information processing device 10 (the feasibility determination unit 130) determines whether both the first condition (the vehicle VH is drivable) and the second condition (the vehicle VH does not approach the obstacle Ob within a predetermined distance DT) are satisfied. If both the first condition and the second condition are satisfied, the information processing device 10 causes the vehicle VH to travel and change the direction of the vehicle VH in accordance with the operation (movement, rotation) of the minicar 20 by the user Us. For example, the information processing device 10 causes the autonomous driving system 3 to automatically park the vehicle VH along the target driving path TC generated from the movement trajectory MT at the target parking position TP generated from the movement trajectory MT. For example, the information processing device 10 causes the autonomous driving system 3 to control the acceleration, steering, deceleration (braking), etc. of the vehicle VH to achieve the "target travel distance TM and target directional change amount TA per unit time."

[0057] If at least one of the first condition and the second condition is not satisfied, the information processing device 10 guides the user Us in moving and rotating the minicar 20 so that both the first condition and the second condition are satisfied. For example, the information processing device 10 guides (guides) the user Us in moving and rotating the minicar 20 so that the target traveling path TC does not deviate from a traveling path "on which the vehicle VH can travel and on which the vehicle VH does not approach the obstacle Ob within a predetermined distance DT." By following such guidance, the user Us can easily move and rotate the minicar 20 along a movement trajectory MT corresponding to the traveling path "on which the vehicle VH can travel and on which the vehicle VH does not approach the obstacle Ob within a predetermined distance DT."

[0058] The information processing device 10 may be considered to determine whether "the vehicle VH can be parked (particularly, whether it can be safely parked) at the target parking position TP generated from the movement trajectory MT along the target travel path TC generated from the movement trajectory MT" based on the first and second conditions described above. If "the vehicle VH cannot be parked at the target parking position TP generated from the movement trajectory MT along the target travel path TC generated from the movement trajectory MT," the information processing device 10 guides (guides) the movement and rotation of the minicar 20 by the user Us so that the target travel path TC becomes a travel path "that allows the vehicle VH to be parked (particularly, allows safe parking) at the target parking position TP." By following such guidance, the user Us can easily move and rotate the minicar 20 along the movement trajectory MT corresponding to the travel path "that allows the vehicle VH to be parked (particularly, allows safe parking) at the target parking position TP."

[0059] §3 Example of operation FIG. 5 is a flowchart showing an example of a processing procedure of the information processing device 10 according to this embodiment. The processing procedure described below is an example of a processing procedure of an information processing method PM that causes a processor (e.g., the CPU of the information processing device 10) to execute the process of "causing the autonomous driving system 3 to automatically park the vehicle VH (particularly, causing the vehicle VH to automatically park on the target path TC at the target parking position TP)." However, the processing procedure described below is merely an example, and each step may be changed as much as possible. Furthermore, steps may be omitted, replaced, or added to the processing procedure described below as appropriate depending on the embodiment.

[0060] (Step S110) In step S110, the control unit 11 operates as a movement trajectory acquisition unit 110 and acquires the movement trajectory MT of the minicar 20, and in particular, acquires the movement trajectory MT of the minicar 20 on the display 410 on which an image IS showing the surrounding conditions of the vehicle VH is displayed.

[0061] (Step S120) In step S120, the control unit 11 operates as the conversion unit 120 and generates a target parking position TP and a target running path TC of the vehicle VH from the movement trajectory MT acquired in step S110. For example, the control unit 11 may generate the target parking position TP and the target running path TC from the movement trajectory MT by converting the movement trajectory MT of the minicar 20 into the target running path TC of the vehicle VH, which includes the target parking position TP of the vehicle VH.

[0062] (Step S130) In step S130, the control unit 11 operates as the feasibility determination unit 130, and outputs the target parking position TP and the target route TC generated in step S120 to the autonomous driving system 3. For example, the control unit 11 outputs to the autonomous driving system 3 a target signal TS including command values ​​(control variables) related to the steering angle, speed, etc. of the vehicle VH for automatically parking the vehicle VH at the target parking position TP on the target route TC.

[0063] Up to now, the information processing method PM for causing a processor to execute the process of "causing the autonomous driving system 3 to automatically park the vehicle VH" has been described using Fig. 5. Next, specific examples of automatic parking of the vehicle VH, etc., realized by the information processing method PM will be described using Figs. 6 to 9.

[0064] (The correspondence between the movement, rotation, and stopping of a miniature car and the running, direction change, and braking of a vehicle) In the example shown in FIG. 6, an image IS showing the surroundings of the vehicle VH shows the vehicle VH and obstacles Ob(1) and Ob(2), which are vehicles other than the vehicle VH. The control unit 11 (feasible trajectory generation unit 152) displays one or more feasible trajectories AT superimposed on the image IS, and in the illustrated example, feasible trajectories AT(1), AT(2), and AT(3) are displayed. Each of the feasible trajectories AT(1), AT(2), and AT(3) is a trajectory corresponding to a "path that the vehicle VH can travel from the current position of the vehicle VH to the target parking position TP." In other words, each feasible trajectory AT is a trajectory corresponding to a "path that the vehicle VH can travel from the current position of the vehicle VH to the target parking position TP" where the distance between the vehicle VH and each of the obstacles Ob(1) and Ob(2) is greater than a predetermined distance DT.

[0065] The user Us uses the possible trajectories AT(1), AT(2), and AT(3) as references to move and rotate the minicar 20 on the display 410, that is, change the position of the minicar 20 and change the direction of the minicar 20. For example, the user Us selects a preferred possible trajectory AT from the possible trajectories AT(1), AT(2), and AT(3), and moves and rotates the minicar 20 on the display 410 so as to follow the selected possible trajectory AT.

[0066] The movement and rotation of the minicar 20 by the user Us is detected by the minicar control unit 30. That is, the minicar control unit 30 detects the movement and rotation of the minicar 20 by the user Us, and identifies the movement trajectory MT of the minicar 20 on the display 410 from the detected “movement and rotation of the minicar 20.”

[0067] The control unit 11 (conversion unit 120) generates a target parking position TP and a target route TC for the vehicle VH from the movement trajectory MT identified by the minicar control unit 30. Then, the control unit 11 (possibility determination unit 130) outputs the target parking position TP and the target route TC (in this embodiment, a target signal TS indicating the target parking position TP and the target route TC) to the autonomous driving system 3. The autonomous driving system 3 automatically parks the vehicle VH according to the target parking position TP and the target route TC output from the information processing device 10, and specifically, automatically parks the vehicle VH at the target parking position TP on the target route TC. Therefore, the information processing device 10 can cause the autonomous driving system 3 to automatically park the vehicle VH at the target parking position TP corresponding to the movement trajectory MT on the target route TC corresponding to the movement trajectory MT of the minicar 20 on the display 410.

[0068] In particular, in this embodiment, the control unit 11 (conversion unit 120) generates a "target travel distance TM and target directional change amount TA of the vehicle VH" corresponding to the "movement and rotation of the minicar 20" detected by the minicar control unit 30. That is, the control unit 11 generates a "target travel distance TM and target directional change amount TA of the vehicle VH per unit time" from the "movement distance and directional change amount of the minicar 20 per unit time." Specifically, the control unit 11 generates a "target travel distance TM of the vehicle VH per unit time" from the "movement distance of the minicar 20 per unit time." Furthermore, the control unit 11 generates a "target directional change amount TA of the vehicle VH per unit time" from the "directional change amount of the minicar 20 per unit time."

[0069] Then, the control unit 11 (possibility determination unit 130) outputs a target signal TS including, for example, control variables of the drive device and braking device of the vehicle VH (command values ​​related to the speed of the vehicle VH, etc.) that realize the "target travel distance TM per unit time of the vehicle VH" to the autonomous driving system 3. The control unit 11 also outputs a target signal TS including, for example, control variables of the steering device of the vehicle VH (command values ​​related to the steering angle of the vehicle VH, etc.) that realize the "target directional change amount TA per unit time of the vehicle VH" to the autonomous driving system 3.

[0070] This allows the control unit 11 (information processing device 10) to control the acceleration, deceleration (braking), steering, etc. of the vehicle VH in accordance with the operation (movement and rotation) of the minicar 20 by the user Us. Therefore, the user Us can move the minicar 20 on the display 410, move the minicar 20 while rotating it, or stop the movement and rotation, thereby causing the vehicle VH to travel, change the traveling direction of the vehicle VH, or stop the vehicle VH.

[0071] In the example shown in Fig. 7, the user Us selects the possible trajectory AT(2) from among the possible trajectories AT(1), AT(2), and AT(3) illustrated in Fig. 6, and moves and rotates the minicar 20 on the display 410 so that the minicar 20 follows the selected possible trajectory AT(2). That is, the user Us moves and rotates the minicar 20 on the display 410 so that the minicar 20 follows the possible trajectory AT(2) in order to automatically park the vehicle VH at the target parking position TP between the obstacles Ob(1) and Ob(2). Then, in accordance with the above-described operations (movement and rotation) of the minicar 20 by the user Us, the control unit 11 (information processing device 10) controls the acceleration, deceleration (braking), steering, etc. of the vehicle VH.

[0072] The image IS illustrated in FIG. 7 shows a pedestrian (obstacle Ob(3)) approaching the vehicle VH. When the user Us notices the presence of the obstacle Ob(3), the user Us stops the movement of the minicar 20 (or the movement and rotation of the minicar 20). Then, as described above, the control unit 11 (information processing device 10) stops the vehicle VH in accordance with the stopping of the movement of the minicar 20 on the display 410, thereby halting the automatic parking of the vehicle VH midway.

[0073] When the obstacle Ob(3) no longer exists, for example, when the user Us confirms that the obstacle Ob(3) is no longer displayed in the image IS, the user Us resumes the movement and rotation of the minicar 20. In response to this, the control unit 11 (information processing device 10) resumes automatic parking of the vehicle VH, and, for example, causes the vehicle VH to travel or change the traveling direction of the vehicle VH in accordance with the movement and rotation of the minicar 20.

[0074] As described above with reference to FIGS. 6 and 7 , the user Us can control the acceleration, deceleration (braking), steering, and the like of the vehicle VH by moving and rotating the minicar 20 on the display 410. The user Us can accurately and easily grasp the surroundings of the vehicle VH (surrounding circumstances) from the “image IS showing the surroundings of the vehicle VH” displayed on the display 410, and can accurately and easily recognize, for example, the presence or absence of an obstacle Ob around the vehicle VH. In other words, the user Us can accurately and easily recognize the presence or absence of an obstacle Ob around the vehicle VH from the image IS, even while controlling the acceleration, deceleration (braking), steering, and the like of the vehicle VH. Furthermore, when the user Us recognizes the presence of an obstacle Ob, the user Us can easily and quickly stop the vehicle VH by simply stopping the movement of the minicar 20 on the display 410, even during automatic parking of the vehicle VH.

[0075] Therefore, the user Us can specify the target parking position TP and target driving route TC of the vehicle VH by moving and rotating the minicar 20 on the display 410, and can also control the acceleration, deceleration (braking), steering, etc. of the vehicle VH. For example, the user Us can make the vehicle VH run by moving the minicar 20 on the display 410, and can stop the vehicle VH by stopping the movement of the minicar 20. Furthermore, the user Us can make the vehicle VH run in a desired direction by moving and rotating the minicar 20 on the display 410. In other words, the user Us can intuitively and easily control the vehicle VH by operating the minicar 20.

[0076] Furthermore, the user Us can move and rotate the minicar 20 along a trajectory obtained by modifying the selected possible trajectory AT, thereby automatically parking the vehicle VH at the target parking position TP and target running path TC corresponding to the modified trajectory (movement trajectory MT). Therefore, the information processing device 10 can automatically park the vehicle VH in a more flexible manner in response to the wishes of the user Us than simply "preparing a plurality of possible trajectories AT in advance and automatically parking the vehicle VH in accordance with the possible trajectory AT selected by the user."

[0077] (Guiding force, etc.) As described above, the user Us moves and rotates the minicar 20 on the display 410, for example, so as to follow the selected possible trajectory AT. However, in this embodiment, the user Us does not have to move and rotate the minicar 20 on the display 410 along the possible trajectory AT, and may move and rotate the minicar 20 without following the possible trajectory AT. The user Us may move and rotate the minicar 20 as appropriate according to the user Us's preferences, and generate a movement trajectory MT that modifies the possible trajectory AT. For example, the movement trajectory MT of the minicar 20 on the display 410 does not have to match any of one or more possible trajectories AT.

[0078] However, if the user Us moves and rotates the minicar 20 without restriction, the vehicle VH may be unable to achieve the "acceleration, deceleration (braking), and steering" corresponding to such movement and rotation, for example, structurally. For example, if the user Us moves and rotates the minicar 20 without restriction, a situation may arise in which the target path TC generated from the movement trajectory MT of the minicar 20 is no longer a path along which the vehicle VH can travel (move) (i.e., the first condition described above is not satisfied). Furthermore, even if such "acceleration, deceleration (braking), and steering" is possible for the vehicle VH, for example, structurally, if such "acceleration, deceleration (braking), and steering" is realized, the vehicle VH may approach the obstacle Ob within a predetermined distance DT. For example, if the user Us moves and rotates the minicar 20 without any restrictions, a situation may arise in which the vehicle VH, which runs and changes direction in response to the operation by the user Us, approaches the obstacle Ob within a predetermined distance DT (i.e., the second condition described above is not satisfied).

[0079] Therefore, the control unit 11 (information processing device 10) guides the user Us to move and rotate the minicar 20 so that the movement and rotation of the minicar 20 by the user Us are appropriate and do not cause the above-mentioned problems. Specifically, the control unit 11 guides the user Us to move and rotate the minicar 20 so that the target parking position TP and the target running path TC generated from the movement trajectory MT of the minicar 20 satisfy both the first and second conditions described above.

[0080] For example, the control unit 11 (possibility determination unit 130) determines whether the target parking position TP and the target path TC are feasible for the first and second conditions described above. If it is determined that the target parking position TP and the target path TC are not feasible for at least one of the first and second conditions, the control unit 11 (display trajectory etc. generation unit 150) notifies (warns) the user Us of information indicating that "the movement and rotation of the minicar 20 by the user Us is inappropriate." For example, the control unit 11 warns the user Us that "the movement trajectory MT of the minicar 20 (in other words, the target path TC generated from the movement trajectory MT) is inappropriate." The notification (warning) of such information may be performed by displaying text indicating the information on the display 410, by flashing a screen or the like displayed on the display 410, or by generating a warning sound. The control unit 11 may display information on the display 410 prompting (guiding) the user Us to perform at least one of the following: "stopping at least one of the movement and rotation of the minicar 20" and "changing (modifying) at least one of the movement and rotation of the minicar 20."

[0081] Furthermore, when it is determined that the target parking position TP and the target running path TC are not feasible for at least one of the first and second conditions, the control unit 11 (guide control unit 160) causes the minicar control unit 30 to generate a guide force GF. That is, the control unit 11 generates a physical force (guide force GF) that restricts the movement and rotation of the minicar 20 by the user Us, and guides the user Us so that the movement and rotation of the minicar 20 by the user Us is appropriate and satisfies the first and second conditions.

[0082] The information processing device 10 can make the operation (movement, rotation) of the minicar 20 by the user Us appropriate by at least one of generating a guide force GF and notifying the user Us of information indicating that "the movement and rotation of the minicar 20 are inappropriate." The information processing device 10 can, for example, guide the operation of the minicar 20 by the user Us so that the target parking position TP and the target running path TC generated from the movement trajectory MT satisfy the first and second conditions described above.

[0083] Below, the guidance of "movement and rotation of the minicar 20 by the user Us" performed by the information processing device 10 will be explained using Figures 8 and 9, in particular the "notification of a warning (in other words, guidance information)" and "generation of a guide force GF" performed by the information processing device 10.

[0084] The control unit 11 (guide control unit 160) may generate a guide force GF that guides the movement and rotation of the minicar 20 by the user Us so that the movement trajectory MT of the minicar 20 follows the possible trajectory AT selected by the user Us. In the example shown in FIG. 8, the user Us selects the possible trajectory AT(2) from the possible trajectories AT(1), AT(2), and AT(3) illustrated in FIG. 6. Therefore, the control unit 11 generates guide forces GF(1) and GF(2) that guide the movement and rotation of the minicar 20 by the user Us so that the movement trajectory MT of the minicar 20 follows the possible trajectory AT(2). As illustrated in FIG. 8, the control unit 11 (display trajectory etc. generation unit 150) may superimpose the direction and magnitude of the guide force GF that guides the operation (movement, rotation) of the minicar 20 by the user Us on an image IS showing the surrounding conditions of the vehicle VH. The control unit 11 may identify the possible trajectory AT selected by the user Us by accepting an "explicit operation by the user Us to select one possible trajectory AT from one or more possible trajectories AT (e.g., possible trajectories AT(1), AT(2), AT(3))." The control unit 11 may identify the possible trajectory AT that is closest to the movement trajectory MT (or the execution trajectory RT) of the minicar 20 as the "possible trajectory AT selected by the user."

[0085] In other words, the control unit 11 (guide control unit 160) may generate a guide force GF so that "the user Us can move and rotate the minicar 20 along the possible trajectory AT selected by the user Us." If the user Us can move and rotate the minicar 20 smoothly, that is, without being restricted by the guide force GF, the movement trajectory MT generated by the movement and rotation of the minicar 20 will be appropriate (valid). In other words, if the user Us can move and rotate the minicar 20 smoothly, the target parking position TP and target running path TC generated from the movement trajectory MT corresponding to the movement and rotation of the minicar 20 will satisfy the first and second conditions described above.

[0086] Furthermore, since the user Us can grasp the guide force GF through touch, he or she can move and rotate the minicar 20 along an appropriate (reasonable) movement trajectory MT without staring at the display 410, for example, while directly observing the situation (surrounding conditions) around the vehicle VH.

[0087] In the example shown in FIG. 9, the control unit 11 (guide control unit 160) generates a guide force GF when the movement and rotation of the minicar 20 by the user Us (i.e., the movement trajectory MT) deviates by a predetermined distance or more from the possible trajectory AT(2) selected by the user Us. As described above, the possible trajectory AT(2) is a trajectory (the possible trajectory AT) corresponding to a route that satisfies the first condition (the vehicle VH is navigable) and the second condition (the vehicle VH does not approach the obstacle Ob within the predetermined distance DT). The control unit 11 then generates the guide force GF to guide the movement and rotation of the minicar 20 by the user Us so that the target parking position TP and the target route TC generated from the movement trajectory MT of the minicar 20 satisfy both the first and second conditions described above. The control unit 11 generates a guide force GF when the target parking position TP and the target route TC generated from the movement trajectory MT do not satisfy at least one of the first and second conditions, for example, when the movement trajectory MT of the minicar 20 deviates from the possible route AT by a predetermined distance or more. In the example shown in FIG. 9, the control unit 11 generates the guide force GF so that the "movement trajectory MT of the minicar 20," which once deviated from the possible route AT(2), approaches the possible route AT(2) again. By generating the guide force GF that moves the movement trajectory MT closer to the possible route AT, the control unit 11 guides the operation of the minicar 20 by the user Us so that the target parking position TP and the target route TC generated from the movement trajectory MT satisfy the first and second conditions.

[0088] As illustrated in Figure 9, the control unit 11 (display trajectory etc. generation unit 150) may superimpose the direction and magnitude of the guide force GF that guides the user Us to operate (move, rotate) the minicar 20 on an image IS showing the surrounding conditions of the vehicle VH.

[0089] Furthermore, if the target parking position TP and the target traveling path TC do not satisfy at least one of the first condition and the second condition, the control unit 11 (possibility determination unit 130) outputs an instruction to stop the movement of the vehicle VH to the autonomous traveling system 3. For example, if the movement trajectory MT of the minicar 20 deviates from the possible trajectory AT by more than a predetermined distance, the control unit 11 outputs a signal to the autonomous traveling system 3 instructing the vehicle VH to brake (i.e., an instruction to stop the movement of the vehicle VH). For example, if the user Us performs a sudden operation on the minicar 20 (for example, an operation to suddenly move or turn the minicar 20), the control unit 11 outputs an instruction to stop the movement of the vehicle VH to the autonomous traveling system 3. For example, if the target parking position TP and the target traveling path TC do not satisfy the second condition (the vehicle VH does not approach the obstacle Ob by less than a predetermined distance DT), the control unit 11 outputs an instruction to stop the movement of the vehicle VH to the autonomous traveling system 3. Therefore, the user Us can automatically park the vehicle VH at the target parking position TP on the target route TC that is safe and drivable for the vehicle VH, by operating (moving, rotating) the minicar 20.

[0090] As described above with reference to FIGS. 8 and 9 , the user Us is guided by the guide force GF to operate the minicar 20 so that the target path TC generated from the movement trajectory MT of the minicar 20 is a drivable and safe path for the vehicle VH. In other words, the user Us is guided by the guide force GF to move and rotate the minicar 20 so that "the target path TC generated from the movement trajectory MT of the minicar 20 satisfies the first and second conditions." Therefore, the user Us can move and rotate the minicar 20 so that the movement trajectory MT of the minicar 20, or more precisely, the target path TC generated from the movement trajectory MT, satisfies the first and second conditions. For example, the user Us selects one of one or more possible paths AT displayed superimposed on the image IS showing the surrounding conditions of the vehicle VH, and moves and rotates the minicar 20 on the display 410 so that the selected possible path AT follows. When the movement trajectory MT of the minicar 20 deviates (strays from) the possible trajectory AT selected by the user Us, the control unit 11 (guide control unit 160) generates a guide force GF so that the movement trajectory MT approaches the selected possible trajectory AT again. Therefore, the user Us can grasp the guide force GF by touch and easily correct the movement trajectory MT of the minicar 20 to an appropriate one.

[0091] [Features] As described above, the information processing device 10 according to this embodiment includes a movement trajectory acquisition unit 110, a conversion unit 120, and a feasibility determination unit 130 (output unit). The movement trajectory acquisition unit 110 acquires a movement trajectory MT of the miniature car 20, which is held by the user Us and whose movement and rotation are detected on a display 410 (display screen) on which an image IS showing the surrounding situation of the vehicle VH is displayed. The conversion unit 120 generates a target parking position TP and a target traveling path TC from the movement trajectory MT acquired by the movement trajectory acquisition unit 110. The feasibility determination unit 130 outputs the target parking position TP and the target traveling path TC generated by the conversion unit 120 to the autonomous driving system 3, which realizes autonomous driving of the vehicle VH.

[0092] Furthermore, the information processing method PM according to this embodiment is an information processing method that causes a processor (for example, the control unit 11 (particularly, the CPU) of the information processing device 10) to execute a process of automatically parking the vehicle VH in an autonomous driving system 3 that realizes autonomous driving of the vehicle VH. The processor executes steps S110, S120, and S130 illustrated in FIG. 5. That is, in step S110, the processor acquires a movement trajectory MT of the minicar 20 on a display 410 that displays an image IS showing the surrounding situation of the vehicle VH. The minicar 20 is a three-dimensional object that is held by a user Us and moved and rotated on the display 410. The movement and rotation of the minicar 20 on the display 410 are detected by the minicar control unit 30. In step S110, the processor acquires a movement trajectory MT that indicates the movement and rotation of the minicar 20 detected by the minicar control unit 30. In step S120, the processor generates a target parking position TP and a target route TC for the vehicle VH from the movement trajectory MT acquired in step S110. In step S130, the processor outputs the target parking position TP and the target route TC generated in step S120 to the autonomous driving system 3.

[0093] According to this configuration, the information processing device 10 (information processing method PM) generates a target parking position TP and a target route TC for the vehicle VH from a movement trajectory MT of the minicar 20 held by the user Us and moved and rotated on the display 410. The information processing device 10 (information processing method PM) then outputs the generated target parking position TP and target route TC to the autonomous driving system 3 that realizes autonomous driving of the vehicle VH. Therefore, the information processing device 10 (information processing method PM) enables the user Us to specify not only the target parking position TP but also the target route TC when receiving parking assistance for the vehicle VH, for example, when automatically parking the vehicle VH.

[0094] Therefore, the user Us can specify not only the target parking position TP but also the target running path TC for the vehicle VH, and thus have the freedom to select the parking trajectory (target running path TC) when automatically parking the vehicle VH. Furthermore, the user Us can automatically park the vehicle VH by intuitively operating the minicar 20. In particular, according to a questionnaire survey regarding driving skills, 29% of respondents answered that they are "not good at parking," making "parking" the driving skill that most people feel uncomfortable with. Therefore, the information processing device 10 (information processing method PM), which allows the user Us to specify not only the target parking position TP but also the target running path TC when automatically parking the vehicle VH, is extremely useful, especially for the approximately 20 to 30% of users who are not good at parking.

[0095] §4 Variations Although the embodiments of the present invention have been described above in detail, the above description is merely illustrative of the present invention in every respect. It goes without saying that various improvements and modifications can be made without departing from the scope of the present invention. For example, the following modifications are possible. Note that, in the following, the same reference numerals are used for components similar to those in the above embodiment, and descriptions of the same points as those in the above embodiment are omitted where appropriate. The following modifications can be combined as appropriate.

[0096] In the above embodiment, an example has been described in which the information processing device 10, the surrounding situation detection device 2, and the autonomous driving system 3 are each configured as separate computers. However, the configuration of the information processing device according to this embodiment is not limited to this example and may be determined appropriately depending on the embodiment. For example, the information processing device 10 and the surrounding situation detection device 2 may be integrated into a single computer. Furthermore, the information processing device 10 and the autonomous driving system 3 may be integrated into a single computer. Similarly, the information processing device 10 and the minicar control unit 30 may be integrated into a single computer, or the minicar control unit 30 and the display device 40 may be integrated into a single computer, or the information processing device 10, the minicar control unit 30, and the display device 40 may be integrated into a single computer. In addition, for example, at least one of the information processing device 10, the surrounding situation detection device 2, and the autonomous driving system 3 may be configured as multiple computers. [Explanation of symbols]

[0097] 3...Autonomous driving system, 10...Information processing device, 20...Miniature car (three-dimensional object), 110...movement trajectory acquisition unit, 120...conversion unit, 130...possibility determination unit (output unit), 410...Display (display screen), DT...Predetermined distance, GF...Guiding force (physical force), IS...Image showing the vehicle's surroundings, MT...Trajectory, Ob...Obstacle, PM...information processing method, TA...target direction change amount, TC...target running route, TM...target running amount, TP...target parking position, Us...user, VH...vehicle,

Claims

1. An information processing method for causing a processor to execute a process for automatically parking a vehicle in an autonomous driving system that realizes autonomous driving of the vehicle, comprising: The processor: Acquire a movement trajectory of a three-dimensional object that is held by a user and whose movement and rotation are detected on a display screen that displays an image showing the surrounding situation of the vehicle; Generate a target parking position and a target driving route from the acquired movement trajectory; outputting the generated target parking position and target driving route to the autonomous driving system; Information processing methods.

2. the movement trajectory includes a position and an orientation of the three-dimensional object at each of a plurality of time points; The processor further comprises: Identifying the amount of movement and amount of change in direction of the three-dimensional object per unit time from the movement trajectory; generating a target travel distance and a target directional change distance per unit time of the vehicle from the identified travel distance and directional change distance per unit time; outputting the generated target travel distance and target direction change amount per unit time to the autonomous driving system; The information processing method according to claim 1 .

3. The processor further comprises: Identifying a direction in which the vehicle can travel based on information relating to the axle and maximum steering angle of the vehicle; generating the target direction change amount per unit time within the specified range of travelable directions; The information processing method according to claim 2 .

4. The processor further comprises: The movement and rotation of the three-dimensional object by the user is guided so that the target path generated from the movement trajectory does not deviate from a path along which the vehicle can travel and which does not approach an obstacle within a predetermined distance. The information processing method according to any one of claims 1 to 3.

5. The processor further comprises: determining whether the vehicle can be parked at the target parking position on the target traveling path generated from the movement trajectory; If it is determined that the vehicle cannot be parked at the target parking position along the target traveling path, the movement and rotation of the three-dimensional object by the user is guided so that the target traveling path generated from the movement trajectory becomes a traveling path that allows the vehicle to be parked at the target parking position. The information processing method according to any one of claims 1 to 3.

6. The processor further comprises: generating a physical force that restricts the user from moving and rotating the three-dimensional object; and notification of information indicating that the movement and rotation of the three-dimensional object by the user is invalid; and guiding the user to move and rotate the three-dimensional object by performing at least one of the following steps. The information processing method according to any one of claims 1 to 3.

7. a movement trajectory acquisition unit that acquires a movement trajectory of a three-dimensional object that is held by a user and whose movement and rotation are detected on a display screen that displays an image showing the surrounding situation of the vehicle; a conversion unit that generates a target parking position and a target driving route from the movement trajectory acquired by the movement trajectory acquisition unit; an output unit that outputs the target parking position and the target driving route generated by the conversion unit to an autonomous driving system that realizes autonomous driving of the vehicle; Equipped with Information processing device.

Images