Vehicle control method and vehicle control device

Abstract

To suppress reduction in accuracy at the time of reproduction travel caused by a moving object around the vehicle, related to a recording type automatic operation.SOLUTION: A vehicle control method makes a display device display a vehicle image corresponding to a vehicle and a peripheral region corresponding to the periphery of the vehicle around the vehicle image before registering a teacher route, eliminates data corresponding to the peripheral region in a first three-dimensional situation when an operation device receives an operation for selecting the peripheral region, and registers the teacher route. The vehicle image displayed on the display device includes: a first side surface part along a travel direction of the vehicle image; a second side surface part located along the travel direction and also opposite to the first side surface part; one end part with respect to the travel direction; and the other end part opposite to the one end part with respect to the travel direction. The peripheral region displayed on the display device is arranged along the first side surface part.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to a vehicle control method and a vehicle control device. [Background technology]

[0002] Generally, parking spaces in homes are often narrow, making parking difficult. For this reason, there is a demand for vehicle control that enables automated driving during parking and exiting.

[0003] In such a situation, in addition to fixed objects such as walls, there may be moving objects such as other vehicles and bicycles around the parking space. In a record-type autonomous driving system that realizes autonomous driving (replay driving) such as parking and leaving based on the driving route of the supervised driving, if the presence or absence of surrounding three-dimensional objects differs between the supervised driving and the replayed driving, there is a risk that the accuracy of the autonomous driving system will decrease, such as a deviation in self-position estimation. For example, Patent Document 1 discloses a technology aimed at appropriately setting the stopping position of a moving body in a place that may be used for purposes other than its original purpose. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Publication No. 2019-196091 Summary of the Invention [Problem to be solved by the invention]

[0005] However, there was room for improvement in the handling of surrounding moving objects; for example, if a fixed object was treated as a moving object, or if a moving object was treated as a fixed object, the accuracy of automated driving could be reduced.

[0006] One of the problems that the present disclosure aims to solve is, in relation to recording-type autonomous driving, to suppress a decrease in accuracy during playback driving caused by surrounding moving objects. [Means for solving the problem]

[0007] A vehicle control method according to the present disclosure can be executed by a vehicle control device mounted on a vehicle that includes an operation device that receives operation from a passenger, a sensor that acquires the status of external three-dimensional objects, a display device that is visible to the passenger, and a movement control device that controls at least steering, and during a teacher driving in which the vehicle drives from a predetermined position to a parking target position by operation of the passenger, a teacher route is registered based on a first three-dimensional object status of an external three-dimensional object acquired by the sensor, and based on the teacher route and a second three-dimensional object status acquired by the sensor, the vehicle is autonomously driven from the predetermined position to the parking target position by causing the movement control device to control at least the steering, Before registering the teacher route, the display device displays a vehicle image corresponding to the vehicle and a surrounding area corresponding to the periphery of the vehicle around the vehicle image, the vehicle image displayed on the display device has a first side portion along the traveling direction of the vehicle image, a second side portion along the traveling direction and located opposite the first side portion, one end portion along the traveling direction, and another end portion opposite the one end portion along the traveling direction, the surrounding area displayed on the display device is arranged along the first side portion, and when the operating device receives an operation to select the surrounding area, it deletes the data corresponding to the surrounding area in the first three-dimensional object situation and registers the teacher route. [Effects of the Invention]

[0008] According to the present disclosure, in relation to recording-type autonomous driving, it is possible to suppress a decrease in accuracy during playback driving caused by surrounding moving objects. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 is a block diagram showing an example of the overall configuration of a vehicle according to an embodiment. [Figure 2] FIG. 2 is a schematic diagram showing an example of the arrangement of the cameras. [Figure 3]FIG. 3 is a schematic diagram showing an example of the external configuration of a vehicle. [Figure 4] FIG. 4 is a diagram showing an example of the configuration near the driver's seat of the vehicle according to this embodiment. [Figure 5] FIG. 5 is a block diagram showing an example of a hardware configuration of the vehicle control device according to the embodiment. [Figure 6A] FIG. 6A is a diagram illustrating an example of the teacher running mode according to the embodiment. [Figure 6B] FIG. 6B is a diagram showing an example of a display screen displayed in the map editing process according to the embodiment. [Figure 7] FIG. 7 is a flowchart showing an example of the flow of the map editing process executed by the vehicle control device according to the embodiment. [Figure 8] FIG. 8 is a diagram showing an example of a display screen displayed in the map editing process according to the embodiment. [Figure 9] FIG. 9 is a diagram showing another example of the display screen displayed in the map editing process according to the embodiment. [Figure 10] FIG. 10 is a diagram showing another example of the display screen displayed in the map editing process according to the embodiment. [Figure 11] FIG. 11 is a diagram showing another example of the display screen displayed in the map editing process according to the embodiment. [Figure 12] FIG. 12 is a diagram showing another example of the display screen displayed in the map editing process according to the embodiment. [Figure 13] FIG. 13 is a diagram showing another example of the display screen displayed in the map editing process according to the embodiment. [Figure 14] FIG. 14 is a diagram showing another example of the display screen displayed in the map editing process according to the embodiment. [Figure 15] FIG. 15 is a diagram showing another example of the display screen displayed in the map editing process according to the embodiment. [Figure 16] FIG. 16 is a diagram showing another example of the display screen displayed in the map editing process according to the embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, embodiments of a vehicle control device and a vehicle control method that can be executed by the vehicle control device according to the present disclosure will be described with reference to the drawings.

[0011] In the description of the present disclosure, components having the same or substantially the same functions as those described above with respect to the previously-mentioned drawings may be given the same reference numerals, and descriptions thereof may be omitted as appropriate. Furthermore, even when the same or substantially the same parts are shown, the dimensions and proportions may be different depending on the drawing. Furthermore, for example, in order to ensure the visibility of the drawings, reference numerals may be given to only the main components in the description of each drawing, and reference numerals may not be given to components having the same or substantially the same functions as those described above with respect to the previously-mentioned drawings.

[0012] In the description of the present disclosure, components having the same or substantially the same functions may be distinguished by adding an alphanumeric character to the end of the reference symbol. Alternatively, when multiple components having the same or substantially the same functions are not distinguished, they may be collectively described by omitting the alphanumeric character at the end of the reference symbol.

[0013] (First embodiment) FIG. 1 is a block diagram showing an example of the overall configuration of a vehicle 1 according to an embodiment.

[0014] 1, a vehicle control device 10 according to the present disclosure is mounted on a vehicle 1. The vehicle 1 has at least a movement control device 12, a sensor 14, a camera 16, an operation device 20, and a display device 22. In the example of FIG. 1, the vehicle 1 further has a storage device 18.

[0015] The vehicle control device 10 is connected to each of the movement control device 12, sensor 14, camera 16, storage device 18, operation device 20, and display device 22 via I / F 11D (see Figure 5) so as to be able to send and receive data or signals.

[0016] The mobility control device 12 is an example of a mobility control device that controls at least the steering of the vehicle 1. Specifically, the mobility control device 12 is a means for realizing the driving, braking, and turning movements required for the vehicle 1 to travel. For example, the mobility control device 12 is configured to include a drive motor, a power transmission mechanism, a brake device, a steering device, etc., and an electronic vehicle control device that controls them. The mobility control device 12 travels the vehicle 1, for example, by generating power using the drive motor and transmitting the power to the wheels via the power transmission mechanism. The power transmission mechanism is, for example, a propeller shaft, a differential gear, a drive shaft, etc.

[0017] Controlling at least the steering means that the mobile control device 12 controls at least one of the driving, braking, and turning motions required for the running of the vehicle 1. In other words, controlling the steering means that the mobile control device 12 controls at least one of the turning direction by steering the steering wheel, the vehicle speed and acceleration by accelerator steering, and the deceleration and stopping by braking.

[0018] Specifically, the movement control device 12 includes an auxiliary control device 12A, a brake control device 12B, an engine control device 12C, and a power steering control device 12D. The brake control device 12B, the engine control device 12C, and the power steering control device 12D can be collectively referred to as an actuator control unit that controls the operation of the vehicle 1.

[0019] The auxiliary control device 12A is a control device that monitors the transmission status of the vehicle control device 10 and operates as a backup to execute appropriate degeneration control in the event of a failure of the vehicle control device 10. Note that if safety can be ensured by providing a degeneration control function within the vehicle control device 10 even in the event of a failure of the vehicle control device 10, degeneration control is not necessary.

[0020] The brake control device 12B is a control device that controls the brakes of the vehicle 1. The brake control is sometimes referred to as braking force control. For example, the brake control device 12B controls the brakes of the vehicle 1 in accordance with the intensification and relaxation of the brake pedal operation by the passenger. Intensification of the brake pedal operation by the passenger specifically means the passenger depressing the brake pedal. Furthermore, the brake control device 12B controls the brakes in accordance with the surrounding image during autonomous driving.

[0021] The engine control device 12C is a control device that controls an engine that generates a driving force for the vehicle 1. The power steering control device 12D is a control device that controls the power steering of the vehicle 1.

[0022] The sensors 14 are various sensors mounted on the vehicle 1 and detect the traveling state of the vehicle 1 and the state around the vehicle 1. The state around the vehicle 1 includes the status of external three-dimensional objects.

[0023] The sensor 14 includes at least a camera 16. The sensor 14 also includes at least one of a LiDAR (Light detection and ranging), radar, sonar, and ultrasonic sensor. The sensor 14 includes, for example, an accelerator opening sensor that detects an accelerator opening, a steering angle sensor that detects a steering angle of a steering device, an acceleration sensor that detects acceleration acting in the longitudinal direction of the vehicle 1, a torque sensor that detects torque acting on a power transmission mechanism between the wheels of the vehicle 1 and the drive motor, a vehicle speed sensor that detects the vehicle speed of the vehicle 1, a wheel speed sensor, a GPS (Global Positioning System), and the like. The sensor 14 outputs sensor information obtained by detection to the vehicle control device 10.

[0024] Here, the three-dimensional objects around the vehicle 1 that can be acquired by the sensor 14 include fixed objects that are objects around the vehicle 1 that do not move or are considered not to move, and moving objects that are objects that can be moved. Fixed parts may be, for example, fixed installations (objects) such as walls or pillars. Moving objects may be, for example, portable installations such as flower pots, pedestrians, other vehicles, and other objects. Furthermore, other vehicles may be various types of moving objects such as passenger cars, freight vehicles, buses, motorcycles, bicycles, kick scooters, etc., and may be moving objects with a power source or human-powered moving objects.

[0025] The camera 16 is a sensor mounted on the vehicle 1 and monitors the environment surrounding the vehicle 1. The camera 16 has an imaging element that captures images of the environment surrounding the vehicle 1. The environment surrounding the vehicle 1 includes the status of external three-dimensional objects. In this embodiment, the camera 16 captures images of the environment surrounding the vehicle 1 and sequentially outputs the captured images to the vehicle control device 10. In this embodiment, the camera 16 outputs a captured video consisting of a plurality of captured images in time series obtained by capturing images of the environment surrounding the vehicle 1 in time series to the vehicle control device 10. Hereinafter, the captured video may be simply referred to as video. In addition, in this embodiment, the camera 16 is also used to detect three-dimensional objects (objects) present around the vehicle 1 and estimate the location of the vehicle 1 from the positional relationship between the vehicle 1 and the objects present around the vehicle 1.

[0026] The positions, number of cameras 16, and shooting directions are adjusted in advance so that the cameras 16 can capture images of the surroundings of the vehicle 1. For example, the vehicle 1 is provided with four cameras 16 arranged so as to capture images in four directions, namely, the front, rear, left, and right directions of the vehicle 1. Note that the number of cameras 16 provided on the vehicle 1 is not limited to four.

[0027] The storage device 18 stores various types of data. In this embodiment, data such as teacher route data 18A and map data 18B are stored in the storage device 18. Details of the teacher route data 18A and the map data 18B will be described later.

[0028] The storage device 18 is, for example, an auxiliary storage device such as a hard disk drive (HDD), a solid state drive (SSD), or a flash memory. At least a part of the data included in the storage device 18 may be stored in an external storage device such as a server device that is provided outside the vehicle 1 and communicably connected to the vehicle control device 10.

[0029] The operation device 20 accepts operations by the passenger (user) of the vehicle 1 and acquires the results of the operation. The operation device 20 includes operation mechanisms related to driving operations, such as a steering wheel, a shift lever for changing gears in the transmission, an accelerator pedal, a brake pedal, a turn signal lever, and a push switch, as well as input devices such as a keyboard, a touch panel, and a switch. The steering angle of the steering device is adjusted by the passenger's operation of the steering operation unit. At least one of the keyboard, the touch panel, and the switch functions as an automatic parking instruction unit that accepts an instruction to start automatic parking. The steering wheel is an example of a steering operation unit. The steering wheel may also be referred to as a steering wheel.

[0030] The shift lever is an example of a forward / reverse operation unit. The forward / reverse operation unit switches the vehicle 1 at least between forward and reverse. The brake pedal is an example of a brake operation unit. The brake operation unit is an operation unit for a brake that suppresses the speed of the vehicle 1. In other words, the brake operation unit receives an instruction to decelerate the vehicle 1. The operation device 20 may constitute a part of an HMI (Human Machine Interface) or an IVI (In-Vehicle Infotainment).

[0031] The display device 22 is a display that outputs various images. The display device 22 is installed in a position where it can be seen by passengers of the vehicle 1. Examples of the display include a liquid crystal display (LCD), an organic electro-luminescence (EL) display, and a projector. The display device 22 may be a touch panel display that is an integral configuration of the display device 22 and the operation device 20. The display device 22 is an example of at least one of an HMI and an IVI.

[0032] The display device 22 is not limited to having only one display area, but may have multiple display areas, for example.

[0033] The vehicle 1 may also include multiple display devices 22. For example, the display device 22 may include a first display unit and a second display unit, where the first display unit and the second display unit are displays that output various images. The first display unit and the second display unit are display devices 22 configured as separate units. The first display unit and the second display unit may be located in different positions within the vehicle 1. For example, the first display unit may function as an IVI, and the second display unit may function as part of the instrument panel of the vehicle 1.

[0034] The operation device 20 and the display device 22 may be integrally configured as a touch panel display in which the operation device 20 as a touch panel is superimposed on the display area of ​​the display device 22. Each of the operation device 20 and the display device 22 is an example of an HMI.

[0035] The operation device 20 may include, for example, a microphone and may be configured to be able to accept operations via voice input from a passenger (user) of the vehicle 1. The vehicle 1 may also be provided with a speaker that outputs notification sounds, warning sounds, and voices instead of or in addition to the display device 22. In this case, notifications in the information processing according to the present disclosure may be realized by outputting voices or the like. Operations such as user selection and specification in response to notifications may also be realized by voice input.

[0036] FIG. 2 is an explanatory diagram of an example of the arrangement of the sensor 14 and the camera 16.

[0037] Vehicle 1 is provided with, for example, four cameras 16 (camera 16A to camera 16D) so that the external situation of vehicle 1 can be acquired in at least four directions, for example, in front, behind, to the right, and to the left of vehicle 1.

[0038] Specifically, for example, camera 16 includes camera 16A, camera 16B, camera 16C, and camera 16D. Camera 16A is disposed at the front of vehicle 1 and photographs the area in front of vehicle 1. Camera 16A is an example of a first camera. Camera 16A may be referred to as a front camera. Camera 16B is disposed at the right side of vehicle 1 and photographs the area to the right of vehicle 1. Camera 16B is an example of a second camera. Camera 16C is disposed at the left side of vehicle 1 and photographs the area to the left of vehicle 1. Camera 16C is an example of a third camera. Camera 16D is disposed at the rear of vehicle 1 and photographs the area to the rear of vehicle 1. Camera 16D is an example of a fourth camera. Camera 16D may be referred to as a rear camera, rear camera, etc.

[0039] The number of cameras 16 provided on the vehicle 1 is not limited to four. It is also preferable that the placement positions and number of sensors included in the sensor 14, such as lidar, radar, sonar, and ultrasonic sensors, that detect objects are adjusted in advance so as to be able to acquire the external conditions on the right side, left side, front, and rear of the vehicle 1. For example, as shown in FIG. 2, the sensor 14 includes sensors 14A to 14F. These sensors 14A to 14F are arranged on the vehicle 1 so as to be able to acquire the external conditions on the right side, left side, front, and rear of the vehicle 1. The object-detecting sensors 14, such as lidar, radar, sonar, and ultrasonic sensors, may be arranged only in the rear of the vehicle 1.

[0040] Next, the configuration of the vehicle 1 will be described.

[0041] FIG. 3 is a schematic diagram showing an example of the external configuration of the vehicle 1. As shown in FIG.

[0042] The vehicle 1 includes a vehicle body 2 and two pairs of wheels 23 arranged along a predetermined direction on the vehicle body 2. The two pairs of wheels 23 include a pair of front tires 23F and a pair of rear tires 23R (see also FIG. 2). Note that FIGS. 2 and 3 show an example in which the vehicle 1 includes four wheels 23. However, the number of wheels 23 provided on the vehicle 1 is not limited to this.

[0043] Next, the configuration of the vicinity of the driver's seat of the vehicle 1 of this embodiment will be described.

[0044] FIG. 4 is a diagram showing an example of the configuration of the vicinity of the driver's seat 24A of the vehicle 1 according to this embodiment.

[0045] The vehicle 1 has a driver's seat 24A and a passenger seat 24B. In front of the driver's seat 24A are provided a windshield 25, a dashboard 26, a steering wheel 20A, operation buttons 20B, and a display device 22. In addition, a shift lever 20C, which is a lever for changing gears in the transmission, is provided near the driver's seat 24A.

[0046] The steering wheel 20A, the operation buttons 20B, and the shift lever 20C are examples of the operation device 20.

[0047] Steering wheel 20A is provided in front of driver's seat 24A and can be operated by a passenger. The rotation angle of steering wheel 20A, i.e., the steering angle, is electrically or mechanically linked to the change in orientation of front tires 23F, which are steered wheels. Note that rear tires 23R may be steered wheels, or both front tires 23F and rear tires 23R may be steered wheels.

[0048] The operation button 20B is a button that can accept operation by a passenger. The operation button 20B may include a direction indicator. The position of the operation button 20B is not limited to the example shown in FIG. 4, and the operation button 20B may be provided on the steering wheel 20A, for example. Although one operation button 20B is illustrated in FIG. 4, a plurality of operation buttons 20B may be provided. When the display device 22 also functions as a touch panel, the display device 22 may be an example of the operation device 20.

[0049] Returning to Figure 1, we continue the explanation.

[0050] The vehicle control device 10 is an electronic control unit that controls each part of the vehicle 1 in an integrated manner.

[0051] The vehicle control device 10 uses sensor information and images received from the sensors 14 and the cameras 16 to control the movement control device 12 so as to optimize the driving conditions of the vehicle 1. The vehicle control device 10 also controls the movement control device 12 to cause the vehicle 1 to drive autonomously.

[0052] Here, the hardware configuration of the vehicle control device 10 of this embodiment will be described.

[0053] FIG. 5 is a block diagram showing an example of a hardware configuration of the vehicle control device 10 according to the embodiment.

[0054] The vehicle control device 10 has at least one processor (e.g., CPU 11A) and at least one memory (e.g., RAM 11C), and has a hardware configuration using a normal computer. For example, as shown in FIG. 5, the vehicle control device 10 has a CPU (Central Processing Unit) 11A, a ROM (Read Only Memory) 11B, a RAM (Random Access Memory) 11C, and an I / F (Interface) 11D. The CPU 11A, ROM 11B, RAM 11C, and I / F 11D are connected to each other via a bus 11E so as to be able to communicate with each other. Note that each component of the vehicle control device 10 may be realized by a combination of two or more components.

[0055] The CPU 11A is a calculation device that controls the overall processing of the vehicle control device 10. The RAM 11C stores data necessary for various processes by the CPU 11A. The ROM 11B stores programs and the like that realize various processes by the CPU 11A. The I / F 11D is an interface that is connected to an external device or external terminal via a communication line or the like and transmits and receives data to and from the connected external device or external terminal.

[0056] The vehicle control device 10 may be realized by a computer such as an ECU (Electronic Control Unit) provided inside the vehicle 1, a DCU (Domain Control Unit) such as a CDC (Cockpit Domain Controller) integrating multiple ECUs, or an OBU (On Board Unit). In this case, the vehicle control device 10 may transmit and receive information to and from other ECUs mounted on the vehicle 1 or devices connected to the vehicle 1 via an in-vehicle network including a CAN (Controller Area Network), Ethernet (registered trademark), or USB (Universal Serial Bus (registered trademark)) within the vehicle 1, or may communicate with an information processing device outside the vehicle 1 via a network such as the Internet. Furthermore, the vehicle control device 10 according to the embodiment may be realized by a single ECU or may be realized by or in cooperation with other ECUs, DCUs, or OBUs that control each function of the vehicle 1, such as power steering control, accelerator control, brake control, other charging control, and autonomous driving control.

[0057] The vehicle control device 10 is not limited to the CPU 11A, and may include various processors such as a GPU (Graphics Processing Unit), an ASIC (Application Specific Integrated Circuit), and an FPGA (Field Programmable Gate Array).

[0058] The vehicle control device 10 may also have various recording media and recording devices such as a hard disk drive (HDD), a solid state drive (SSD), and a flash memory as auxiliary storage devices.

[0059] In addition, the program executed by the vehicle control device 10 according to this embodiment may be provided by being recorded in an installable or executable file format on a computer-readable recording medium (Computer Program Product) such as a CD-ROM, FD, CD-R, or DVD.

[0060] The program executed by the vehicle control device 10 according to the present embodiment may be stored on a computer connected to a network such as the Internet and provided by being downloaded via the network. The program executed by the vehicle control device 10 according to the present embodiment may be provided or distributed via a network such as the Internet.

[0061] The program executed by the vehicle control device 10 according to this embodiment may be provided by being pre-installed in a ROM or the like.

[0062] Next, the functional configuration of the vehicle control device 10 of this embodiment will be described.

[0063] For example, at least one processor (e.g., CPU 11A) of the vehicle control device 10 loads a program stored in an auxiliary storage device (e.g., ROM 11B) into a main storage device (e.g., RAM 11C) and executes the loaded program, thereby realizing each function of the vehicle control device 10. Note that at least one processor of the vehicle control device 10 may be configured as a dedicated hardware circuit.

[0064] For example, in the vehicle control device 10, the CPU 11A realizes at least one function including the function of the control unit 11. The control unit 11 controls each unit of the vehicle 1 in an integrated manner.

[0065] The control unit 11 controls, for example, the steering, acceleration, deceleration, forward movement and reverse movement of the vehicle 1 using the movement control device 12, but it is also possible to control only the steering using the movement control device 12 and have the passenger operate the acceleration, deceleration and forward movement.

[0066] The control unit 11 according to this embodiment is configured to be able to switch the driving mode between a teacher driving mode and an autonomous driving mode based on input operations by the user to the operation device 20. Note that the driving modes that can be executed by the vehicle 1 may include various driving modes other than the teacher driving mode and the autonomous driving mode.

[0067] The supervised driving mode is a mode for registering a supervised route for autonomous driving of the vehicle 1. The supervised route is a route obtained by supervised driving from a predetermined position to a target parking position. In the supervised driving mode, the vehicle 1 is controlled to travel by the driving operation of the user. That is, in the supervised driving mode, the control unit 11 controls the movement control device 12 so that the vehicle travels in accordance with the driving operation of the user.

[0068] The autonomous driving mode is a mode in which the vehicle 1 drives autonomously. In this embodiment, the autonomous driving mode refers to a mode in which the vehicle 1 drives replayedly along a teacher route. In the autonomous driving mode, the control unit 11 controls at least steering and controls the mobile control device 12 so that the vehicle 1 drives along the teacher route. In the autonomous driving mode, the vehicle 1 is automatically controlled to drive by the vehicle control device 10 without any driving operation by the user.

[0069] FIG. 6A is a diagram illustrating an example of the teacher running mode according to the embodiment.

[0070] In the supervised driving mode, the user drives the vehicle from a predetermined position P1 to a parking target position P2. The parking target position P2 is, for example, a parking lot, but is not limited to this. The predetermined position P1 may be any position of the user in real space.

[0071] The travel route R traveled during teacher driving is treated as a teacher route R1, and teacher route data 18A for the teacher route R1 is stored in the storage device 18. During teacher driving, the user may drive the vehicle 1 so that it travels from the parking target position P2 toward the predetermined position P1, or from the predetermined position P1 toward the parking target position P2. When the vehicle 1 travels from the parking target position P2 toward the predetermined position P1 in teacher driving mode, the control unit 11 may create teacher route data 18A for the teacher route R1, which is the opposite of the traveling direction during teacher driving for the travel route R. When the vehicle 1 travels from the predetermined position P1 toward the parking target position P2 during teacher driving, the control unit 11 may create teacher route data 18A for the teacher route R1, which is the same as the traveling direction during teacher driving for the travel route R. The creation of the teacher route data 18A will be described in detail later.

[0072] In the autonomous driving mode, the control unit 11 controls at least the steering along the driving route R obtained by the supervised driving, and causes the vehicle 1 to autonomously drive to the parking target position P2. Note that in the autonomous driving mode, the control unit 11 controls the steering and the front and rear acceleration / deceleration of the vehicle 1, but at least a part of the front and rear acceleration / deceleration control may be performed by the driver's operation.

[0073] Next, the control by the control unit 11 in each of the driving modes, ie, the teacher driving mode and the autonomous driving mode, will be described in detail.

[0074] [Teacher driving mode] First, the control of the control unit 11 in the teacher running mode will be described in detail.

[0075] When the control unit 11 receives a signal indicating an instruction to start the teacher-driven driving mode by a user operating the operation device 20, the control unit 11 switches the driving mode to the teacher-driven driving mode. Then, when the control unit 11 is in the teacher-driven driving mode, the control unit 11 executes the following processing.

[0076] The control unit 11 acquires sensor information indicating the traveling state of the vehicle 1 from the sensor 14. Then, the control unit 11 estimates the current position of the vehicle 1 based on the temporal changes in the sensor values ​​represented by the sensor information. For example, the control unit 11 calculates the amount of movement of the vehicle 1 from a reference position, such as the traveling start position when the teacher traveling mode was started, based on the temporal changes in the vehicle speed and yaw rate represented by the sensor values, and estimates the current position of the vehicle 1 based on the amount of movement.

[0077] Note that the accuracy of estimating the current position based on the amount of movement may be low. Therefore, the control unit 11 may correct the estimated current position based on an image of the surroundings of the vehicle 1 acquired by the camera 16, and use the corrected result as the current position.

[0078] The control unit 11 sequentially stores the current position of the vehicle 1 estimated sequentially as the vehicle 1 travels in the storage device 18. In detail, the control unit 11 defines the travel route R during teacher travel, which is represented by a group of current positions sequentially estimated from the time when an instruction to start the teacher travel mode is received until the time when an instruction to end the teacher mode is received, as a teacher route R1, and stores teacher route data 18A representing the teacher route R1 in the storage device 18.

[0079] The teacher route data 18A is composed of a group of driving information for each position, which is the current position that is sequentially estimated during teacher driving. The driving information includes, for example, an INDEX, a driving position, a bearing, a driving direction, and reference driving information. The INDEX is identification information for the driving information. The driving position is the estimated position of the vehicle 1. The bearing indicates the orientation of the vehicle 1 at that position. The driving direction indicates the direction in which the vehicle 1 is driving at that position, and is expressed, for example, as forward or reverse. The reference driving information is information that indicates the driving state at that position. The reference driving information is, for example, information such as the steering angle and vehicle speed detected at each position during teacher driving.

[0080] The map data 18B is map data in which a plurality of feature points (for example, feature point 404a in FIG. 9) around the vehicle 1 when traveling along the teacher route R1 are registered. When the vehicle 1 is traveling along the teacher route R1, the control unit 11 creates the map data 18B for estimating the current position of the vehicle 1 from the image captured by the camera 16. The method for estimating the current position of the vehicle 1 from the image may be a SLAM (Simultaneous Localization and Mapping) method or the like.

[0081] A feature point is a characteristic point obtained by image analysis of an image captured by the camera 16 during teacher driving. For example, a feature point is a portion of an object (e.g., a tree, wall, or pillar) that can serve as a landmark in the actual scene, from which a characteristic image pattern is obtained by analyzing the captured image. Such a portion is, for example, an edge portion of the object. The map data 18B includes a plurality of feature points, and each feature point is registered by being assigned an identification number so that it can be identified.

[0082] A feature point is represented by feature point data including a three-dimensional position and a feature amount. The three-dimensional position of a feature point is the three-dimensional position of the feature point in real space and is represented, for example, by a three-dimensional Cartesian coordinate system (X, Y, Z). The feature amount of a feature point is a characteristic amount of the feature point represented by image analysis of the captured image. The feature amount of a feature point is, for example, brightness or density on the captured image, SIFT (Scale Invariant Feature Transform) feature amount, SURF (Speeded Up Robust Features) feature amount, etc.

[0083] In the map data 18B, one feature point is registered for each identical three-dimensional position. Note that, for the same three-dimensional position, the map data 18B may register multiple feature points for each photographing position and photographing direction of the three-dimensional position by the camera 16. Furthermore, the feature point data of the feature points registered in the map data 18B may further include image data of an object having the feature point.

[0084] During supervised driving, the control unit 11 identifies the coordinates of feature points in the actual scene, for example, based on stereo photogrammetry. Specifically, the control unit 11 reads multiple captured images taken at different times and associates the same feature points that appear in the multiple captured images. The control unit 11 then estimates the virtual position of the vehicle 1 at the time the multiple captured images were taken and identifies the virtual coordinates of the feature points in the actual scene using the principle of triangulation. The control unit 11 then performs bundle adjustment, for example, using the virtual position of the vehicle 1 and the virtual coordinates of the feature points in the actual scene as reference information, to calculate the formal position of the vehicle 1 and the formal coordinates of the feature points in the actual scene so as to minimize the reprojection error when each feature point in the actual scene is projected onto all captured images. The control unit 11 then stores, in the storage device 18, map data 18B in which feature points are registered, represented by feature point data including the formal coordinates of the feature points in the actual scene as three-dimensional positions.

[0085] The three-dimensional positions of the feature points registered in the map data 18B may be positions measured in advance using a LiDAR (Light Detection and Ranging) or a stereo camera, instead of using the SLAM method. However, from the viewpoint of suppressing a decrease in the accuracy of position estimation, the SLAM method may be used.

[0086] The control unit 11 may generate one piece of map data 18B that includes the entire teacher route R1, or may create multiple pieces of partial map data that are at least partially non-overlapping along the teacher route R1 as the map data 18B.

[0087] As described above, the control unit 11 executes the above process in the supervised driving mode. Therefore, in the supervised driving mode, the control unit 11 generates and stores in the storage device 18: supervised route data 18A of the supervised route R1 obtained by supervised driving from a predetermined position P1 to a parking target position P2; and map data 18B in which the three-dimensional positions of each of a plurality of feature points around the vehicle 1 while the vehicle 1 was traveling along the supervised route R1 and the feature amounts of each feature point are registered. In other words, the control unit 11 registers the supervised route based on the status of external three-dimensional objects (first three-dimensional object status), including data on the feature points of the three-dimensional objects, acquired by the sensor 14 and / or the camera 16 during supervised driving in which the vehicle 1 travels from a predetermined position P1 to a parking target position P2 in response to an operation by the vehicle occupant.

[0088] The control unit 11 then executes the map editing process before registering the teacher route data 18A and the map data 18B as teacher data for one teacher route R1, i.e., before registering the teacher route, such as when teacher driving in teacher driving mode is completed. More preferably, the control unit 11 executes the map editing process after the vehicle 1 in teacher driving has arrived at the parking target position P2 and before registering the teacher route.

[0089] In the supervised driving mode, when the vehicle 1 is driven to the parking target position P2 by the user, there may be moving objects, which are three-dimensional objects with a certain degree of possibility of moving, located in the surrounding area A1 to the left of the parking target position P2, the surrounding area A2 to the right of the parking target position P2, and the surrounding area A3 behind the parking target position P2. Such moving objects may not be present in the autonomous driving mode.

[0090] Therefore, in the map editing process, the control unit 11 according to the present disclosure causes the display device 22 to display a vehicle image corresponding to the host vehicle and a surrounding area corresponding to the periphery of the vehicle image. This surrounding area is displayed around the vehicle image.

[0091] FIG. 6B is a diagram showing an example of the display screen 300 displayed in the map editing process according to the embodiment.

[0092] The display screen 300 includes a vehicle image 401 showing the host vehicle and a surrounding area S (S1 to S3). Specifically, the vehicle image 401 and the surrounding area S are displayed superimposed on an external image 402.

[0093] The vehicle image 401 may be an icon of a synthesized image, or may be an image of the vehicle 1 captured in the past. The external image 402 is an image based on at least one image of the surroundings of the vehicle 1 captured by, for example, the camera 16. The external image 402 may also be an image that has been subjected to image processing including viewpoint conversion, or may be an overhead image (top-view image) that is a composite of images captured by multiple cameras 16. A known method may be used to generate the overhead image.

[0094] The vehicle image 401 includes a first side surface F1, a second side surface F2, one end F3, and another end F4.

[0095] The first side surface F1 is a side surface of the vehicle image 401 that is aligned with the traveling direction (direction of arrow D) of the vehicle image 401. The traveling direction of the vehicle image 401 is the traveling direction of the vehicle 1 represented by the vehicle image 401. In other words, the first side surface F1 is a region that corresponds to a side surface of the vehicle image 401 that is aligned with the traveling direction of the vehicle 1 represented by the vehicle image 401.

[0096] The second side surface portion F2 is a side surface portion in the vehicle image 401 that is along the traveling direction (direction of arrow D) of the vehicle image 401 and is located opposite the first side surface portion F1. In other words, the second side surface portion F2 is a region corresponding to the side surface portion opposite the first side surface portion F1 among the side surfaces in the vehicle image 401 that are along the traveling direction of the vehicle 1 represented by the vehicle image 401.

[0097] The one end F3 is one end in the traveling direction (direction of arrow D) of the vehicle image 401. In detail, the one end F3 is a region in the vehicle image 401 that corresponds to the side surface of the vehicle 1 represented by the vehicle image 401 on the one end side in the traveling direction (direction of arrow D).

[0098] The other end F4 is the other end in the traveling direction (arrow D direction) of the vehicle image 401. In detail, the other end F4 is a region in the vehicle image 401 that corresponds to the side surface on the other end side of the one end F3 in the traveling direction (arrow D direction) of the vehicle 1 represented by the vehicle image 401.

[0099] The surrounding area S displayed on the display device 22 is an area arranged along the first side surface F1 of the displayed vehicle image 401.

[0100] Specifically, the surrounding region S displayed on the display device 22 includes a first surrounding region S1 and a second surrounding region S2. The surrounding region S may further include a third surrounding region S3 in addition to the first surrounding region S1 and the second surrounding region S2. Fig. 6B shows an example in which the surrounding region S includes the first surrounding region S1, the second surrounding region S2, and the third surrounding region S3.

[0101] The first surrounding area S1 is the surrounding area S arranged along the first side surface F1 of the vehicle image 401 displayed on the display device 22. The first surrounding area S1 is an area corresponding to the surrounding area A2 (see FIG. 6A) on the right side of the vehicle 1 in real space. The first surrounding area S1 displayed on the display device 22 has a rectangular shape. One of the long sides of the rectangle of the first surrounding area S1 is arranged along the first side surface F1 of the vehicle image 401.

[0102] The second surrounding area S2 is the surrounding area S arranged along the second side surface F2 of the vehicle image 401 displayed on the display device 22. The second surrounding area S2 is an area corresponding to the surrounding area A1 (see FIG. 6A) on the left side of the vehicle 1 in real space. The second surrounding area S2 displayed on the display device 22 has a rectangular shape. One of the long sides of the rectangle of the second surrounding area S2 is arranged along the second side surface F2 of the vehicle image 401.

[0103] The third surrounding area S3 is a surrounding area S arranged along one end F3 of the vehicle image 401 displayed on the display device 22. The third surrounding area S3 is an area corresponding to the surrounding area A3 (see FIG. 6A) behind the vehicle 1 in real space. The third surrounding area S3 displayed on the display device 22 has a rectangular shape. One of the long sides of the rectangle of the third surrounding area S3 is arranged along one end F3 of the vehicle image 401.

[0104] Furthermore, in the map editing process, when the operation device 20 receives a user operation to select the surrounding area S, the control unit 11 deletes data corresponding to the surrounding area S in the situation related to external objects (first three-dimensional object situation) acquired during the teacher driving, and then registers the teacher route. The data corresponding to the surrounding area S includes data on feature points of external three-dimensional objects corresponding to the surrounding area S.

[0105] The control unit 11 may accept any of a user operation to select only the first surrounding region S1, a user operation to select only the second surrounding region S2, and a user operation to select only the third surrounding region S3. The control unit 11 may also accept a user operation to select two or more of the first surrounding region S1, the second surrounding region S2, and the third surrounding region S3. For example, the control unit 11 may accept any of a user operation to select the first surrounding region S1 and the second surrounding region S2, a user operation to select the first surrounding region S1 and the third surrounding region S3, a user operation to select the second surrounding region S2 and the third surrounding region S3, and a user operation to select the first surrounding region S1, the second surrounding region S2, and the third surrounding region S3.

[0106] Then, the control unit 11 registers the teacher route after deleting data corresponding to the surrounding area S for which the user operation to select was accepted from among the first surrounding area S1, the second surrounding area S2, and the third surrounding area S3, in the situation regarding external objects (first three-dimensional object situation) acquired during teacher driving.

[0107] This prevents the situation regarding external objects (second three-dimensional object situation) acquired in the autonomous driving mode from differing from the situation regarding external objects (first three-dimensional object situation) registered in the map data 18B, thereby preventing a decrease in the accuracy of the vehicle position estimation. For example, if an object such as another vehicle is present near the side of the vehicle, the other vehicle's image will occupy a larger area within the angle of view of the camera 16, which is installed to capture the side, and this will have a significant impact on the acquisition of the situation regarding surrounding objects during teacher driving and playback driving. Under such circumstances, by performing a map editing process that deletes data corresponding to the surrounding area S and registers a teacher route, it is possible to prevent an increase in the impact of the presence of another vehicle during teacher driving but the absence of another vehicle during playback driving, and to prevent a situation in which feature points cannot be acquired sufficiently near the parking target position P2, which requires particularly careful control.

[0108] [Autonomous driving mode] Next, the control of the control unit 11 in the autonomous driving mode will be described in detail.

[0109] When the control unit 11 receives a signal indicating an instruction to start the autonomous driving mode by a user's operation of the operation device 20, the control unit 11 switches the driving mode to the autonomous driving mode. Then, when in the autonomous driving mode, the control unit 11 executes the following processing.

[0110] The control unit 11 reads the teacher route data 18A and the map data 18B from the storage device 18, and controls the mobile control device 12 so that the mobile device 12 performs autonomous traveling along the teacher route R1 represented by the teacher route data 18A.

[0111] The control unit 11 estimates the current position of the vehicle 1 based on the map data 18B and the captured image of the surroundings of the vehicle 1 acquired by at least one camera 16.

[0112] For example, the control unit 11 first compares characteristic points extracted from the image captured by the camera 16 with characteristic points stored in the map data 18B using pattern matching, feature amount search, etc. Then, the control unit 11 randomly selects several (e.g., 3 to 6) characteristic points from the characteristic points extracted from the image captured by the camera 16 that can be compared with the characteristic points stored in the map data 18B.

[0113] Then, the control unit 11 estimates the current position of the vehicle 1 in real space based on the positions of these several characteristic points in the captured image and the three-dimensional positions in real space of characteristic points registered in the map data 18B that correspond to the several characteristic points. At this time, the control unit 11 estimates the current position of the vehicle 1 by solving the PnP problem using a known method such as Lambda Twist (for example, literature: Mikael Persson et al. "Lambda Twist: An Accurate Fast Robust Perspective Three Point (P3P) Solver.", ECCV 2018, pp. 334-349, published in 2018, http: / / openaccess.thecvf.com / content_ECCV_2018 / papers / Mikael_Persson_Lambda_Twist_An_ECCV_2018_paper.pdf).

[0114] When comparing the characteristic points extracted from the image captured by the camera 16 with the characteristic points stored in the map data 18B, the control unit 11 may, for example, calculate the current position of the vehicle 1 as a tentative position based on the amount of movement of the vehicle 1 described above, and, using the tentative position as a reference, narrow down the characteristic points stored in the map data 18B to be compared with the characteristic points extracted from the image captured by the camera 16.

[0115] Through these processes, the control unit 11 estimates current position information representing the current position of the vehicle 1 based on the map data 18B and the captured images of the surroundings of the vehicle 1 acquired by at least one camera 16, and the current position information includes the two-dimensional position (X coordinate, Y coordinate) of the vehicle 1 in real space and information related to the orientation of the vehicle 1.

[0116] The control unit 11 then controls the movement control device 12 so that the estimated current position of the vehicle 1 is a position on the teacher route R1 represented by the teacher route data 18A, thereby causing the vehicle 1 to autonomously travel along the teacher route R1 from a predetermined position P1 toward the parking target position P2. The control unit 11 then stops the vehicle 1 at the parking target position P2.

[0117] When the vehicle 1 is traveling autonomously along the travel route R, the control unit 11 feedback controls the movement control device 12 so that the vehicle 1 moves along the teacher route R1 based on the estimated current position of the vehicle 1 and each position on the teacher route R1 represented by the teacher route data 18A.

[0118] In this way, in the autonomous driving mode, the control unit 11 controls at least the steering of the vehicle 1 to make the vehicle 1 autonomously drive from a predetermined position P1 to a parking target position P2 based on the teacher route and the situation of the external three-dimensional object (second three-dimensional object situation) including data on the characteristic points of the three-dimensional object acquired by the sensor 14 and / or the camera 16.

[0119] An example of the operation of the vehicle 1 according to the embodiment will be described below with reference to the drawings. Note that the processing described below is an example, and it is possible to change the processing order, delete some processing, or add other processing. Also, in the following description, explanations of previously mentioned content will be omitted as appropriate.

[0120] Fig. 7 is a flowchart showing an example of the flow of map editing processing executed by the vehicle control device 10 according to the embodiment. Fig. 8 is a diagram showing an example of a display screen displayed in the map editing processing according to the embodiment. Fig. 8 shows operation screens 410 to 450 as examples of display screens in the map editing processing. Each of the operation screens 410 to 450 shown in Fig. 8 or each notification displayed on each of these operation screens is an example of a display that prompts the user to select a surrounding area S in which a three-dimensional object with a possibility of movement greater than a certain level is located.

[0121] The flow of FIG. 7 is executed in the teacher running mode.

[0122] The vehicle control device 10 determines whether the shift position is "P" (S101). If the shift position is not "P" (S101: No), the vehicle control device 10 repeats the processing of S101 until it is determined that the shift position is "P".

[0123] Here, determining whether the shift position is "P" is an example of determining whether the teacher driving has been completed. The determination of whether the teacher driving has been completed is not limited to the shift position, and may be realized by determining whether an operation indicating that the teacher driving has been completed has been performed by the user, such as displaying a completion button on the display screen of the display device 22 during teacher driving and pressing the completion button. Alternatively, the determination of whether the teacher driving has been completed may be realized by determining whether a predetermined time has elapsed since the vehicle 1 stopped. This predetermined time may be, for example, determined in advance and stored in the internal memory of the vehicle control device 10.

[0124] If the shift position is "P" (S101: Yes), the vehicle control device 10 displays the operation screen 410 for checking surrounding moving objects (S102).

[0125] 8, operation screen 410 includes vehicle image 401 showing the host vehicle and external image 402 showing surrounding areas A1 to A3 of the host vehicle. Specifically, on operation screen 410, vehicle image 401 is displayed superimposed on external image 402.

[0126] The vehicle image 401 may be an icon of a synthesized image, or may be an image of the vehicle 1 captured in the past. The external image 402 is an image based on at least one image captured around the vehicle 1 by, for example, the camera 16. The external image 402 may be an image that has been subjected to image processing including viewpoint conversion, or may be a bird's-eye view image (top-view image) synthesized from at least one image.

[0127] Furthermore, if there is an object such as a moving object in the surrounding areas A1 to A3, the operation screen 410 may further include a moving object image 403 that shows the surrounding moving object.

[0128] 8, the operation screen 410 includes a notification 411 such as "Is there a parked vehicle next to you?" The notification 411 includes "Yes" and "No" operation buttons 413 and 415 for accepting user operations.

[0129] The vehicle control device 10 determines whether or not there is a moving object in the vicinity (S103). For example, when the "Yes" operation button 413 is selected on the operation screen 410, the vehicle control device 10 determines that there is a moving object in the vicinity. For example, when the "No" operation button 415 is selected on the operation screen 410, the vehicle control device 10 determines that there is no moving object in the vicinity.

[0130] If there is no moving object in the vicinity (S103: No), the flow in FIG. 7 ends.

[0131] If there is a moving object in the vicinity (S103: Yes), the vehicle control device 10 displays the operation screen 420 for specifying the position of the moving object in the vicinity (S104).

[0132] 8, operation screen 420 includes, in addition to vehicle image 401 and external image 402, notification 421 such as "Where is the parked vehicle located?". Notification 421 includes "left" and "right" operation buttons 423, 425 that accept user operations. Note that the operation buttons may be any buttons that specify any of the left, right, and rear areas corresponding to surrounding areas A1 to A3, and there may be three of them.

[0133] The vehicle control device 10 determines whether the position of a surrounding moving object has been designated (S105). For example, the vehicle control device 10 determines that the position of a surrounding moving object has been designated when either the operation button 423 or 425 is selected on the operation screen 420. If the position of a surrounding moving object has not been designated (S105: No), the vehicle control device 10 waits until a designated operation is performed.

[0134] If the position of the surrounding moving object is specified (S105: Yes), the vehicle control device 10 displays the operation screen 430 for specifying the type of the surrounding moving object (S106).

[0135] As shown in FIG. 8, operation screen 430 includes, in addition to vehicle image 401 and external image 402, notification 431 such as "What type of parked vehicle?" Notification 431 includes operation buttons 433 to 435 for "car," "motorcycle," and "bicycle" that accept user operations. Note that the operation buttons may be buttons that specify the type of surrounding moving object, and may also be buttons that indicate other types. Furthermore, when control unit 11 determines the type of surrounding moving object using a known image analysis method or the like, it may highlight an operation button (for example, operation button 433) among operation buttons 433 to 435 that indicates the determined type of moving object.

[0136] The vehicle control device 10 determines whether or not the type of surrounding moving object has been designated (S107). For example, the vehicle control device 10 determines that the type of surrounding moving object has been designated when any of the operation buttons 433 to 435 is selected on the operation screen 430. If the type of surrounding moving object has not been designated (S107: No), the vehicle control device 10 waits until a designated operation is performed.

[0137] If the type of surrounding moving object is specified (S107: Yes), the vehicle control device 10 displays the operation screen 440 for checking the range of surrounding moving objects (S108).

[0138] It should be noted that steps S102, S103, S104, S105, S106, and S107 may be omitted, i.e., after Yes in S101, the process may transition directly to S108.

[0139] 8, operation screen 440 includes, in addition to vehicle image 401 and external image 402, a surrounding area 405 corresponding to a moving object whose position and type are specified, and a notification 441 such as "Does the frame fit the vehicle?" The displayed surrounding area 405 is an example of the first surrounding area S1 and surrounding area S described above. Notification 441 includes "Yes" and "No" operation buttons 443, 445 for accepting user operations.

[0140] For example, the surrounding area 405 may be a display showing an area of ​​the surrounding areas A1 to A3 of the host vehicle where a moving object, which is a three-dimensional object with a higher probability of moving than a certain level, is located. The surrounding area 405 is displayed around the vehicle image 401, as shown in Fig. 8. The operation screen 440 of Fig. 8 illustrates a case where a moving object is present in the surrounding area A2 of the surrounding areas A1 to A3 of the host vehicle, and therefore the surrounding area 405 corresponding to the surrounding area A2 to the right of the host vehicle is displayed. Similarly, on the operation screen 440, when a moving object is present in the surrounding area A1 or the surrounding area A3, the surrounding area 405 corresponding to the surrounding area A1 to the left of the host vehicle and the surrounding area A1 behind the host vehicle are displayed.

[0141] For example, the surrounding area 405 corresponding to the surrounding area A1 on the left side of the host vehicle is arranged along the left side of the vehicle image 401. The displayed surrounding area 405 corresponding to the surrounding area A1 on the left side of the host vehicle is an example of the second surrounding area S2.

[0142] For example, the surrounding area 405 corresponding to the surrounding area A2 on the right side of the vehicle is arranged along the right side of the vehicle image 401. The displayed surrounding area 405 corresponding to the surrounding area A2 on the right side of the vehicle is an example of the first surrounding area S1.

[0143] For example, the surrounding area 405 corresponding to the surrounding area A3 behind the host vehicle is arranged along the rear of the vehicle image 401. The displayed surrounding area 405 corresponding to the surrounding area A3 behind the host vehicle is an example of the third surrounding area S3.

[0144] Note that FIG. 8 and the following figures show, as an example, a display form in which the displayed surrounding area 405 (surrounding area S) indicates an area among the surrounding areas A1 to A3 of the vehicle where a moving object, which is a three-dimensional object with a higher probability of moving than a certain level, is located. However, the displayed surrounding area 405 (surrounding area S) may also be an area where no moving three-dimensional object, such as a vehicle, is located. FIG. 8 and the following figures show, as an example, a display form in which the surrounding area 405 (first surrounding area S1), which is an area where a moving object is located, is displayed. However, regardless of whether or not a moving object is located, at least one of the second surrounding area S2 and the third surrounding area S3 may also be displayed.

[0145] The size of the surrounding area 405 may be slightly larger than the detected moving object, considering that feature points may not be obtained over the entire outer shape of the surrounding moving object due to noise, etc.

[0146] Note that the surrounding area 405 may be a display of a frame surrounding the detected moving object, as exemplified in Fig. 8. Furthermore, since the required accuracy decreases as the distance from the host vehicle increases, the surrounding area 405 may be a part of a frame, such as a U-shape (angled U-shape) that opens in a direction away from the host vehicle. Furthermore, the frame of the surrounding area 405 is not limited to a rectangle, and may have other shapes.

[0147] Furthermore, the surrounding area 405 may be represented by a moving object image 403 corresponding to a vehicle other than the vehicle 1. This moving object image 403 is an example of a second vehicle image. The moving object image 403 may be an icon of a composite image, or may be an image of an actually captured moving object that is highlighted by marking or the like. The moving object image 403 does not need to include the entire detected moving object, but may instead display a portion of it. The icon serving as the moving object image 403 may not strictly match the size of the surrounding area 405. For example, the icon may be smaller or larger than the surrounding area 405. The icon serving as the moving object image 403 may also represent the surrounding area 405.

[0148] The vehicle control device 10 determines whether to adjust the range of surrounding moving objects (S109). For example, when the "No" operation button 445 is selected on the operation screen 440, the vehicle control device 10 determines to adjust the range of surrounding moving objects. For example, when the "Yes" operation button 443 is selected on the operation screen 440, the vehicle control device 10 determines not to adjust the range of surrounding moving objects.

[0149] 7 proceeds to the processing of S113. That is, when the "No" operation button 445 is selected on the operation screen 440 (S109: No), this corresponds to selecting the range of the moving object. Note that this range of the moving object can also be regarded as a predetermined surrounding area S corresponding to the periphery of the vehicle image 401.

[0150] When the range of surrounding moving objects is to be adjusted (S109: Yes), the vehicle control device 10 displays the operation screen 450 for adjusting the range of surrounding moving objects (S110).

[0151] 8, operation screen 450 includes surrounding area 405 and notification 451 such as "Please adjust the frame." Notification 451 includes operation buttons 453 and 454 for instructing to extend / retract the frame of surrounding area 405 in the front-to-rear direction of vehicle 1, operation button 455 for instructing to move the frame of surrounding area 405 in the front-to-rear or left-to-right directions of vehicle 1, and operation button 456 for instructing to complete adjustment.

[0152] The vehicle control device 10 displays the frame of the surrounding area 405 by modifying it in accordance with the user's operation on the operation buttons 453 to 455 (S111). Thereafter, the vehicle control device 10 determines whether or not the adjustment is complete (S112). For example, the vehicle control device 10 determines that the adjustment is complete when the "Complete" operation button 456 is selected on the operation screen 450. For example, the vehicle control device 10 determines that the adjustment is not complete when the operation buttons 453 to 455 are operated on the operation screen 450.

[0153] If the adjustment is not completed (S112: No), the vehicle control device 10 repeats the processes of S111 to S112. On the other hand, if the adjustment is completed (S112: Yes), the flow in FIG. 7 proceeds to the process of S113. That is, when the "Complete" operation button 456 is selected on the operation screen 450 (S112: Yes), this corresponds to selecting the range of this moving object. Note that it is also possible to regard this range of the moving object as a predetermined surrounding area S corresponding to the periphery of the vehicle image 401.

[0154] The vehicle control device 10 deletes surrounding moving objects from the map data 18B based on the range of the frame of the surrounding area 405 (S113). As a result, it is possible to register a teacher route based on the status of external three-dimensional objects (first three-dimensional object status) acquired during teacher driving, from which data corresponding to the surrounding area 405 indicating the moving objects has been deleted. After S113, the process transitions to the end. Alternatively, after S113, the process may return to the start without transitioning to the end. Furthermore, if a forced cancel button, which is a hard button provided on the operating device 20, is operated between S101 and S113 after the start, the process may forcibly transition to the end or forcibly transition to the start.

[0155] The data to be deleted may be data at a predetermined height that is predetermined and stored in the internal memory within the frame range of the surrounding area 405. This predetermined height may be a height equal to or greater than the vehicle itself, such as 2 m.

[0156] In this way, the vehicle control device 10 according to the present disclosure realizes a UI (user interface) for specifying the presence and location of surrounding moving objects when the teacher driving for registering teacher data is completed. Furthermore, when the presence and location of surrounding moving objects are specified, the vehicle control device 10 according to the present disclosure registers map data 18B from which the moving objects that cannot exist in a fixed manner have been removed.

[0157] Therefore, according to the configuration of the present disclosure, since map data 18B based on fixed and unchanging feature points can be registered, it is possible to suppress a decrease in accuracy during playback driving caused by surrounding moving objects in record-type automated driving. This suppression of accuracy decrease during playback driving contributes to suppressing misalignment of the parking position during automatic parking playback.

[0158] The data to be deleted may be, for example, data corresponding to a range according to the specified position and / or type, which is predetermined and stored in the internal memory when at least one of the position and type of the surrounding moving object is specified. With this configuration, it is possible to suppress a decrease in accuracy during playback driving caused by surrounding moving objects in record-type autonomous driving by using simpler processing, i.e., while reducing the user's effort.

[0159] (Second embodiment) Here, differences from the first embodiment will be mainly described, and overlapping descriptions will be omitted as appropriate.

[0160] Fig. 9 is a diagram showing another example of a display screen displayed in the map editing process according to the embodiment. Fig. 9 shows operation screens 460 to 500 as display screens in the map editing process. The operation screens 460 to 500 correspond to the operation screens 410 to 450 shown in Fig. 8, respectively.

[0161] As shown in FIG. 9, the operation screens 460 to 500 may include a display of a moving object image 403 as well as feature points 404a (point cloud data) detected during instructor travel.

[0162] The display mode of the feature points 404b in the surrounding area 405, such as color, brightness, size, and shape, may be different from that of the other feature points 404a.

[0163] Even with this configuration, the same effects as in the first embodiment can be obtained.

[0164] (Third embodiment) Here, differences from the first embodiment will be mainly described, and overlapping descriptions will be omitted as appropriate.

[0165] Fig. 10 is a diagram showing another example of a display screen displayed in the map editing process according to the embodiment. Fig. 10 shows operation screens 510 to 520 as display screens in the map editing process. The operation screen 510 corresponds to the operation screens 410 to 430 shown in Fig. 8. The operation screen 520 corresponds to the operation screens 440 to 450 shown in Fig. 8.

[0166] The vehicle control device 10 displays the operation screen 510 in the same manner as the operation screen 410.

[0167] 10, the operation screen 510 includes a vehicle image 401 of the vehicle itself and a moving object image 403 of a detected surrounding object. This moving object image 403 may be a composite image icon or an image of an actually captured moving object marked in a selectable manner. The operation screen 510 also includes a notification 511 such as "Please touch the screen if there is a parked vehicle nearby." The notification 511 includes a "No vehicle" operation button 513 that accepts a user operation.

[0168] When the "no vehicle" operation button 513 is operated, the vehicle control device 10 ends the flow of Fig. 7. On the other hand, when the moving object image 403 is operated, the vehicle control device 10 displays the operation screen 520.

[0169] 10, in addition to the vehicle image 401 and the moving object image 403, the operation screen 520 includes a surrounding area 405, a notification 521 such as "You can adjust the frame by dragging the four corners or the center," and an operation icon 525 for adjusting the range of the surrounding area 405. The notification 521 includes a "Done" operation button 523 that accepts a user operation.

[0170] The vehicle control device 10 displays the frame of the surrounding area 405 on the operation screen 520 by deforming it in accordance with a user operation on an operation icon 525 .

[0171] When a user operates the operation button 523 on the operation screen 520, the vehicle control device 10 deletes surrounding moving objects from the map data 18B based on the range of the frame of the surrounding area 405 at that time.

[0172] According to this configuration, it is possible to suppress a decrease in accuracy during playback driving caused by surrounding moving objects in recording-type automatic driving, while using simpler processing, i.e., reducing the user's effort.

[0173] (Fourth embodiment) Here, differences from the third embodiment will be mainly described, and overlapping descriptions will be omitted as appropriate.

[0174] In the map editing process according to the first embodiment, the moving object image 403 may be displayed as an image representing the surrounding area 405 corresponding to at least one of the surrounding areas A1 to A3 of the parking target position P2, regardless of whether it was detected during instruction driving.

[0175] Fig. 11 is a diagram showing another example of a display screen displayed in the map editing process according to the embodiment. Fig. 11 shows operation screens 560 to 570 as display screens in the map editing process. Each of the operation screens 560 to 570 corresponds to the operation screen 510 shown in Fig. 10.

[0176] The vehicle control device 10 displays the operation screen 560 in the same manner as the operation screen 510.

[0177] 11, the operation screen 560 includes a vehicle image 401 of the vehicle itself, multiple moving object images 403a, 403b, and 403c, and a notification 561. The notification 561 includes a "Done" operation button 563. The moving object image 403a is an example of a displayed second surrounding area S2 corresponding to the left surrounding area A1. The moving object image 403b is an example of a displayed first surrounding area S1 corresponding to the right surrounding area A2. The moving object image 403c is an example of a displayed third surrounding area S3 corresponding to the rear surrounding area A3.

[0178] When the "Complete" operation button 563 is operated without selecting any of the moving object images 403, the vehicle control device 10 ends the flow in Fig. 7. On the other hand, when any of the moving object images 403 is operated, the vehicle control device 10 displays the operation screen 570.

[0179] An operation screen 570a, which is an example of the operation screen 570, highlights 407a the moving object image 403b selected by the user from the moving object images 403a to 403c.

[0180] An operation screen 570b as another example of the operation screen 570 displays a check mark 407b for the moving object image 403b selected by the user from among the moving object images 403a to 403c.

[0181] When the "Complete" operation button 563 is operated while any of the moving object images 403 is selected, the vehicle control device 10 displays an operation screen including surrounding areas 405 corresponding to each of the selected moving objects, similar to the operation screen 520 of Figure 10.

[0182] Even with this configuration, the same effects as those of the above-described embodiment can be obtained.

[0183] (Fifth embodiment) Here, differences from the fourth embodiment will be mainly described, and overlapping descriptions will be omitted as appropriate.

[0184] Fig. 12 is a diagram showing another example of a display screen displayed in the map editing process according to the embodiment. Fig. 12 shows operation screens 610 to 620 as display screens in the map editing process. The operation screens 610 to 620 correspond to the operation screens 560 to 570 shown in Fig. 11, respectively.

[0185] 12, operation screen 610 includes vehicle image 401 of the host vehicle, multiple moving object images 403a, 403b, and 403c, a notification 611 such as "Please select whether there are any parked vehicles next to or behind you," and a selection icon 615. Notification 611 includes an operation button 613 for "Done." Notification 611 corresponds to notification 561 in FIG. 11.

[0186] The vehicle control device 10 accepts the selection of the moving object image 403 in response to a user operation on the selection icon 615. When any one of the selection icons 615a, 615b, and 615c is operated and in the "Yes" state, the vehicle control device 10 displays the operation screen 620.

[0187] An example of the operation screen 620, operation screen 620a, corresponds to operation screen 570a in FIG. 11, and highlights 407a the moving object image 403b corresponding to the selection icon 615b for which "Yes" has been selected by the user among the moving object images 403a to 403c.

[0188] Operation screen 570b, another example of operation screen 570, corresponds to operation screen 570b of Figure 11, and displays a check mark 407b for moving object image 403b corresponding to selection icon 615b for which "Yes" has been selected by the user among moving object images 403a to 403c.

[0189] Even with this configuration, the same effects as those of the above-described embodiment can be obtained.

[0190] (Sixth embodiment) Here, differences from the fifth embodiment will be mainly described, and overlapping descriptions will be omitted as appropriate.

[0191] Fig. 13 is a diagram showing another example of a display screen displayed in the map editing process according to the embodiment. Fig. 13 shows operation screens 660 to 670 as display screens in the map editing process. The operation screens 660 to 670 correspond to the operation screens 610 to 620 shown in Fig. 12, respectively.

[0192] 13, operation screen 660 includes check boxes 665 instead of selection icons 615. Notification 661 corresponds to notification 611 in Fig. 12. Also, operation button 663 of notification 661 corresponds to operation button 613 in Fig. 12.

[0193] The vehicle control device 10 accepts the selection of the moving object image 403 in response to a user operation on the check box 665. When any one of the check boxes 665a, 665b, and 665c is operated, the vehicle control device 10 displays an operation screen 670.

[0194] 12, and checkboxes 665b of the moving object images 403a to 403c operated by the user are set to a "checked" state. At this time, highlighting 407a and checkmarks 407b may also be displayed, similar to FIG. 12, etc.

[0195] Even with this configuration, the same effects as those of the above-described embodiment can be obtained.

[0196] (Seventh embodiment) Here, differences from the fourth to sixth embodiments will be mainly described, and overlapping descriptions will be omitted as appropriate.

[0197] Fig. 14 is a diagram showing another example of a display screen displayed in the map editing process according to the fourth embodiment. Fig. 14 illustrates an operation screen 580 as a display screen in the map editing process. When all of the moving object images 403a to 403c are selected by the user on the operation screen 570a of Fig. 11, the vehicle control device 10 highlights 407a all of the moving object images 403a to 403c and displays a notification 581. Note that similar processing may be performed on the operation screen 570b of Fig. 11. Here, the case where all of the moving object images 403a to 403c are selected by the user is an example of a case where the area of ​​the selected surrounding region 405 is larger than a predetermined area.

[0198] FIG. 15 is a diagram showing another example of a display screen displayed in the map editing process according to the fifth embodiment. FIG. 15 illustrates an operation screen 630 as a display screen in the map editing process. The vehicle control device 10 displays a notification 581 when all of the selection icons 615a to 615 are selected by the user on the operation screen 620a of FIG. 12 to indicate "Yes." At this time, all of the moving object images 403a to 403c may be highlighted 407a. Note that similar processing may be performed on the operation screen 620b of FIG. 12. Here, the case where all of the selection icons 615a to 615 are selected by the user to indicate "Yes" is an example of a case where the area of ​​the selected surrounding region 405 is larger than a predetermined area.

[0199] FIG. 16 is a diagram showing another example of a display screen displayed in the map editing process according to the sixth embodiment. FIG. 16 illustrates an operation screen 680 as a display screen in the map editing process. The vehicle control device 10 displays a notification 581 when all of the check boxes 665a to 665b on the operation screen 670 of FIG. 13 are selected by the user to become "checked." At this time, all of the moving object images 403a to 403c may be highlighted 407a. Note that similar processing may be performed on the operation screen 620b of FIG. 12. Here, the case where all of the check boxes 665a to 665b are selected by the user to become "checked" is an example of a case where the area of ​​the selected surrounding region 405 is larger than a predetermined area.

[0200] Notification 581 is a notification that registration is not possible, such as "Registration is not possible if there are parked vehicles in three places on the left, right, and rear." This notification 581 is a notification based on the fact that, due to the presence of moving objects on three sides, if the surrounding moving objects are deleted from map data 18B based on the ranges of the frames of surrounding area 405 on three sides, map data 18B sufficient for self-position estimation cannot be generated.

[0201] If the area of ​​the selected surrounding area 405 is larger than a predetermined area, the vehicle control device 10 does not register the teacher route. In this case, the vehicle control device 10 may issue a notification urging the user to try teacher driving again.

[0202] The notification 581 is performed when, for example, three sides are selected as described above, but may also be performed when the range of the frame of the surrounding area 405 of one or two selected sides is adjusted by a user operation and becomes larger than a predetermined area. The size of the predetermined area is, for example, predetermined and stored in an internal memory.

[0203] According to this configuration, it is possible to suppress a decrease in the accuracy of self-location estimation that accompanies the deletion of data corresponding to moving objects.

[0204] The techniques according to the above-described embodiments can be combined as appropriate.

[0205] In each of the above-mentioned embodiments, "whether it is A or not" refers to at least one of "it is A" and "it is not A." In other words, in each of the above-mentioned embodiments, the determination of "whether it is A or not" may be realized by determining only "it is A," or by determining only "it is not A," or by determining both of these.

[0206] According to at least one of the embodiments described above, in relation to recording-type automatic driving, it is possible to suppress a decrease in accuracy during playback driving caused by surrounding moving objects.

[0207] Although several embodiments of the present invention have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These embodiments can be implemented in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their modifications are included within the scope and spirit of the invention, as well as within the scope of the invention described in the claims and their equivalents. [Explanation of symbols]

[0208] 1 vehicle 10 Vehicle control device 11 Control section 11A CPU 11B ROM 11C RAM 11D I / F 12 Movement control device 14 Sensors 16 Camera (sensor) 18 Storage device 20 Operating device 22 Display device

Claims

1. The present invention can be implemented by a vehicle control device mounted on a vehicle that includes an operation device that accepts operations by a passenger, a sensor that acquires the status of an external three-dimensional object, a display device that is visible to the passenger, and a movement control device that controls at least steering, registering a teacher route based on a first three-dimensional object status of an external three-dimensional object acquired by the sensor during teacher travel in which the vehicle travels from a predetermined position to a parking target position by operation of the passenger; a vehicle control method for causing the movement control device to control at least the steering and autonomously drive the vehicle from the predetermined position to the parking target position based on the teacher path and a second external three-dimensional object state acquired by the sensor, before registering the teacher route, displaying on the display device a vehicle image corresponding to the vehicle and a surrounding area corresponding to the periphery of the vehicle around the vehicle image, The vehicle image displayed on the display device is a first side surface portion along a traveling direction of the vehicle image, a second side surface portion along the traveling direction and located opposite the first side surface portion, one end portion in the traveling direction, and another end portion opposite the one end portion in the traveling direction, the surrounding area displayed on the display device is disposed along the first side surface portion, When the operation device receives an operation to select the surrounding area, data corresponding to the surrounding area in the first three-dimensional object situation is deleted, and the teacher path is registered. Vehicle control method.

2. 2. The vehicle control method according to claim 1, the surrounding area displayed on the display device comprises a first surrounding area and a second surrounding area; the first surrounding area is disposed along the first side portion of the vehicle image displayed on the display device, The second surrounding area is disposed along the second side portion of the vehicle image displayed on the display device. Vehicle control method.

3. 3. The vehicle control method according to claim 2, the surrounding area displayed on the display device further comprises a third surrounding area; the third surrounding area is disposed along the one end of the vehicle image displayed on the display device, Vehicle control method.

4. 2. The vehicle control method according to claim 1, the surrounding area displayed on the display device has a rectangular shape; One of the long sides of the rectangle is arranged along the first side portion. Vehicle control method.

5. 2. The vehicle control method according to claim 1, before registering the teacher route, displaying on the display device the vehicle image and the surrounding area, and displaying a message prompting the user to select the surrounding area in which a three-dimensional object having a higher possibility of movement than a certain level is located; Vehicle control method.

6. 2. The vehicle control method according to claim 1, after the vehicle in the teacher driving mode has arrived at the parking target position and before the teacher route is registered, the vehicle image and the surrounding area are displayed on the display device. Vehicle control method.

7. 2. The vehicle control method according to claim 1, The vehicle image is a composite image. Vehicle control method.

8. 2. The vehicle control method according to claim 1, the sensor includes an image pickup element for acquiring an image of an external three-dimensional object; before registering the teacher route, an external image based on the image is displayed on the display device, and the vehicle image and the surrounding area are displayed superimposed on the external image. Vehicle control method.

9. 2. The vehicle control method according to claim 1, the vehicle image is a first vehicle image; The surrounding area is represented by a second vehicle image corresponding to a vehicle different from the vehicle. Vehicle control method.

10. 2. The vehicle control method according to claim 1, the operation device accepts the operation of selecting the surrounding area, and if the area of ​​the selected surrounding area is larger than a predetermined area, the teacher path is not registered. Vehicle control method.

11. The present invention can be implemented by a vehicle control device mounted on a vehicle that includes an operation device that accepts operations by a passenger, a sensor that acquires the status of an external three-dimensional object, a display device that is visible to the passenger, and a movement control device that controls at least steering, registering a teacher route based on a first three-dimensional object status of an external three-dimensional object acquired by the sensor during teacher travel in which the vehicle travels from a predetermined position to a parking target position by operation of the passenger; causing the movement control device to control at least the steering and cause the vehicle to autonomously travel from the predetermined position to the parking target position based on the teacher path and the state of a second external three-dimensional object acquired by the sensor; before registering the teacher route, displaying on the display device a vehicle image corresponding to the vehicle and a surrounding area corresponding to the periphery of the vehicle around the vehicle image, The vehicle image displayed on the display device is a first side surface portion along a traveling direction of the vehicle image, a second side surface portion along the traveling direction and located opposite the first side surface portion, one end portion in the traveling direction, and another end portion opposite the one end portion in the traveling direction, the surrounding area displayed on the display device is disposed along the first side surface portion, When the operation device receives an operation to select the surrounding area, data corresponding to the surrounding area in the first three-dimensional object situation is deleted, and the teacher path is registered. Vehicle control device.

12. The vehicle control device according to claim 11, the surrounding area displayed on the display device comprises a first surrounding area and a second surrounding area; the first surrounding area is disposed along the first side portion of the vehicle image displayed on the display device, The second surrounding area is disposed along the second side portion of the vehicle image displayed on the display device. Vehicle control device.

13. The vehicle control device according to claim 12, the surrounding area displayed on the display device further comprises a third surrounding area; the third surrounding area is disposed along the one end of the vehicle image displayed on the display device, Vehicle control device.

14. The vehicle control device according to claim 11, the surrounding area displayed on the display device has a rectangular shape; One of the long sides of the rectangle is arranged along the first side portion. Vehicle control device.

15. The vehicle control device according to claim 11, before registering the teacher route, displaying on the display device the vehicle image and the surrounding area, and displaying a message prompting the user to select the surrounding area in which a three-dimensional object having a higher possibility of movement than a certain level is located; Vehicle control device.

16. The vehicle control device according to claim 11, after the vehicle in the teacher driving mode has arrived at the parking target position and before the teacher route is registered, the vehicle image and the surrounding area are displayed on the display device. Vehicle control device.

17. The vehicle control device according to claim 11, The vehicle image is a composite image. Vehicle control device.

18. The vehicle control device according to claim 11, the sensor includes an image pickup element for acquiring an image of an external three-dimensional object; before registering the teacher route, an external image based on the image is displayed on the display device, and the vehicle image and the surrounding area are displayed superimposed on the external image. Vehicle control device.

19. The vehicle control device according to claim 11, the vehicle image is a first vehicle image; The surrounding area is represented by a second vehicle image corresponding to a vehicle different from the vehicle. Vehicle control device.

20. The vehicle control device according to claim 11, the operation device accepts the operation of selecting the surrounding area, and if the area of ​​the selected surrounding area is larger than a predetermined area, the teacher path is not registered. Vehicle control device.

Images