Vehicle control method and vehicle control device

Abstract

A vehicle control unit (10) is configured to be mounted on a vehicle (1) which includes an operating device (20) for receiving passenger operations, a camera (16) for acquiring surrounding images, a display device (22) for visual confirmation by the passenger, and a movement control unit (12) for controlling steering. While the vehicle is traveling in the direction of travel according to the passenger's driving, the vehicle control unit (10) detects a target parking position of the vehicle (1) based on the surrounding images acquired by the camera (16), and causes the display device (22) to display a screen that overlays the target parking position and the corresponding reversing start position onto the surrounding images. The target parking position is set along a direction having a predetermined angle relative to the direction of travel and is located either to the left or right of the centerline of the vehicle (1) along the direction of travel. The reversing start position displayed on the screen is located either to the left or right of the centerline of the vehicle (1).

Description

Technical Field

[0001] This disclosure relates to a vehicle control method and a vehicle control device. Background Technology

[0002] Generally speaking, parking spaces in houses are often narrow and sometimes difficult to navigate, thus creating a high demand for parking and exiting parking spaces using autonomous driving.

[0003] A vehicle control device for implementing such autonomous driving is known. For example, when parking the vehicle in a parking space, the vehicle moves forward and stops after passing the parking space, then reverses from that stop position to park in the parking space. As a system to assist in such parking, a system is disclosed in which, when a passenger drives the vehicle to the stop position (i.e., the reversing start position), and at that reversing start position the gear is shifted into reverse according to the passenger's operation, the vehicle control device detects a target parking position and autonomously drives the vehicle to the detected target parking position.

[0004] Existing technical documents

[0005] Patent documents

[0006] Patent Document 1: Japanese Patent Application Publication No. 2012-066709

[0007] Patent Document 2: Japanese Patent Application Publication No. 2022-069161 Summary of the Invention

[0008] The problem the invention aims to solve

[0009] However, in the existing technology, there are sometimes situations where it is difficult to provide appropriate parking assistance when starting from a reverse position where it is difficult to autonomously drive towards the target parking position.

[0010] The purpose of this disclosure is to provide a vehicle control method and vehicle control device that can provide appropriate parking assistance.

[0011] Solution for solving the problem

[0012] The vehicle control method disclosed herein is executed by a vehicle control device mounted on a vehicle. The vehicle includes an operating device for receiving passenger operations, a camera for acquiring surrounding images, a display that the passenger can visually confirm, and at least a motion control device for controlling steering. In the vehicle control method, while the vehicle is traveling in the direction of travel according to the passenger's driving, a target parking position of the vehicle is detected based on the surrounding images acquired by the camera. The display then shows a display that superimposes the target parking position and the corresponding reversing start position onto the surrounding images. The screen shows that after the vehicle moves forward according to the driver's direction of travel and reaches the reversing start position, the steering is controlled to reverse the vehicle, and the vehicle moves autonomously from the reversing start position to the target parking position. The detected target parking position is set along a direction with a predetermined angle relative to the direction of travel and is located either to the left or right of the vehicle's centerline along the direction of travel. The reversing start position displayed on the screen is located either to the left or right of the vehicle's centerline.

[0013] The effects of the invention

[0014] The vehicle control method and vehicle control device disclosed herein can provide appropriate parking assistance. Attached Figure Description

[0015] Figure 1 This is a block diagram illustrating an example of the overall structure of a vehicle.

[0016] Figure 2 This is a schematic diagram illustrating an example of camera configuration.

[0017] Figure 3 This is a schematic diagram illustrating an example of the structure of a vehicle's exterior.

[0018] Figure 4 This is a diagram showing an example of the structure near the driver's seat of the vehicle according to this embodiment.

[0019] Figure 5 This is an explanatory diagram of an example of parking assistance in this embodiment.

[0020] Figure 6 This is a schematic diagram of an example of a display screen shown by a display device.

[0021] Figure 7 This is a flowchart illustrating an example of the information processing flow performed by the control department.

[0022] Figure 8AThis is an illustrative diagram illustrating an example of a parking assistance implementation.

[0023] Figure 8B This is an illustrative diagram illustrating an example of a parking assistance implementation.

[0024] Figure 9A This is a schematic diagram of an example of a display screen.

[0025] Figure 9B This is a schematic diagram of an example of a display screen.

[0026] Figure 9C This is a schematic diagram of an example of a display screen.

[0027] Figure 9D This is a schematic diagram of an example of a display screen.

[0028] Figure 9E This is a schematic diagram of an example of a display screen.

[0029] Figure 9F This is a schematic diagram of an example of a display screen.

[0030] Figure 9G This is a schematic diagram of an example of a display screen.

[0031] Figure 10 This is a flowchart illustrating an example of the information processing flow performed by the control department.

[0032] Figure 11A This is an illustrative diagram illustrating an example of a parking assistance implementation.

[0033] Figure 11B This is a schematic diagram of an example of a display screen.

[0034] Figure 12A This is an illustrative diagram illustrating an example of a parking assistance implementation.

[0035] Figure 12B This is a schematic diagram of an example of a display screen.

[0036] Figure 13 This is an explanatory diagram of the parking assistance implementation method.

[0037] Figure 14A This is an illustrative diagram illustrating an example of a parking assistance implementation.

[0038] Figure 14B This is a schematic diagram of an example of a display screen.

[0039] Figure 15 This is an explanatory diagram of the parking assistance implementation method.

[0040] Figure 16 This is a flowchart illustrating an example of the information processing flow performed by the control department.

[0041] Figure 17A This is an illustrative diagram illustrating an example of a parking assistance implementation.

[0042] Figure 17B This is an illustrative diagram illustrating an example of a parking assistance implementation.

[0043] Figure 18A This is an illustrative diagram illustrating an example of a parking assistance implementation.

[0044] Figure 18B This is a schematic diagram of an example of a display screen.

[0045] Figure 19A This is an illustrative diagram illustrating an example of a parking assistance implementation.

[0046] Figure 19B This is a schematic diagram of an example of a display screen.

[0047] Figure 20 This is an explanatory diagram of the parking assistance implementation method.

[0048] Figure 21A This is a schematic diagram of an example of a display screen.

[0049] Figure 21B This is a schematic diagram of an example of a display screen.

[0050] Figure 21C This is an example illustration of a 30P display screen.

[0051] Figure 22 This is an illustrative diagram illustrating an example of a parking assistance implementation.

[0052] Figure 23A This is a schematic diagram of an example of a display screen.

[0053] Figure 23B This is a schematic diagram of an example of a display screen.

[0054] Figure 23C This is a schematic diagram of an example of a display screen.

[0055] Figure 24 This is a flowchart illustrating an example of the information processing flow performed by the control department.

[0056] Figure 25 This is a flowchart illustrating an example of the information processing flow performed by the control department.

[0057] Figure 26A This is an illustrative diagram illustrating an example of a parking assistance implementation.

[0058] Figure 26B This is an illustrative diagram illustrating an example of a parking assistance implementation.

[0059] Figure 27A This is an illustrative diagram illustrating an example of a parking assistance implementation.

[0060] Figure 27B This is an illustrative diagram illustrating an example of a parking assistance implementation.

[0061] Figure 28 This is a flowchart illustrating an example of the information processing flow performed by the control department.

[0062] Figure 29A This is an illustrative diagram illustrating an example of a parking assistance implementation.

[0063] Figure 29B This is a schematic diagram of an example of a display screen.

[0064] Figure 30 This is an illustrative diagram illustrating an example of a parking assistance implementation.

[0065] Figure 31 This is a flowchart illustrating an example of the information processing flow performed by the control department.

[0066] Figure 32 This is a block diagram illustrating an example of the hardware structure of a vehicle control device. Detailed Implementation

[0067] Hereinafter, embodiments of the vehicle control method and vehicle control device involved in this disclosure will be described with reference to the accompanying drawings.

[0068] (First Implementation)

[0069] Figure 1 This is a block diagram illustrating an example of the overall structure of vehicle 1 according to this embodiment.

[0070] The vehicle 1 is equipped with a vehicle control device 10, a motion control device 12, a sensor 14, a camera 16, a storage device 18, an operating device 20, and a display device 22.

[0071] The motion control device 12, sensor 14, camera 16, storage device 18, operating device 20, and display device 22 are connected to the vehicle control device 10 in a manner capable of sending and receiving data or signals. That is, the vehicle control device 10 is configured to be communicatively connected to at least the operating device 20, camera 16, display device 22, and motion control device 12.

[0072] The motion control device 12 controls at least the steering of the vehicle 1. The motion control device 12 is a component that enables the driving, braking, and turning movements required for the vehicle 1 to move. For example, the motion control device 12 is configured to include a drive motor, a power transmission mechanism, a braking device, and a steering device, as well as an electronic vehicle control device that controls them. The motion control device 12 generates power using, for example, a drive motor, and transmits power to the wheels via the power transmission mechanism, thereby causing the vehicle 1 to move. The power transmission mechanism may be, for example, a drive shaft, a differential gear, or a drive shaft.

[0073] At least steering control means that the mobile control device 12 controls at least one of the driving, braking, and turning movements required for the movement of the vehicle 1. That is, steering control means that the mobile control device 12 controls at least one of the turning direction generated by steering, the vehicle speed or acceleration generated by throttle steering, and the deceleration or stopping generated by braking. The mobile control device 12 can also control the acceleration and deceleration of the vehicle 1. Acceleration and deceleration control means that the mobile control device 12 controls at least one of the acceleration and deceleration of the vehicle 1.

[0074] In detail, the motion control device 12 includes an auxiliary control device 12A, a brake control device 12B, an engine control device 12C, and a power steering control device 12D. The brake control device 12B, the engine control device 12C, and the power steering control device 12D can be collectively referred to as the actuator control unit for controlling the movement of the vehicle 1.

[0075] The auxiliary control device 12A is a control device that monitors the transmission status of the vehicle control device 10 and performs appropriate degraded control as a backup when the vehicle control device 10 fails. Furthermore, if safety can be ensured by having degraded control functionality within the vehicle control device 10 even in the event of a vehicle control device 10 failure, then degraded control is not required.

[0076] Braking control device 12B is a control device for braking vehicle 1. Braking control is sometimes referred to as braking force control. For example, braking control device 12B performs braking control of vehicle 1 based on the strengthening and weakening of the passenger's operation of the brake pedal. Specifically, strengthening the passenger's operation of the brake pedal means that the passenger presses the brake pedal. In addition, braking control device 12B performs braking control based on surrounding images when driving autonomously.

[0077] Engine control unit 12C is a control unit that controls the engine that generates the driving force of vehicle 1. Power steering control unit 12D is a control unit that controls the power steering of vehicle 1.

[0078] Sensor 14 is mounted on vehicle 1 to acquire at least the external conditions of vehicle 1. In detail, sensor 14 is a variety of sensors that detect the driving state of vehicle 1 and the external conditions of vehicle 1.

[0079] Sensor 14 includes at least camera 16. Additionally, sensor 14 includes at least one of LiDAR (Light Detection and Ranging), radar, sonar, and ultrasonic sensors. Sensor 14 may include, for example, an accelerator pedal opening sensor for detecting accelerator pedal opening, a steering angle sensor for detecting the steering angle of the steering mechanism, an acceleration sensor for detecting acceleration acting in the longitudinal direction of vehicle 1, a torque sensor for detecting the torque acting on the power transmission mechanism between the wheels and drive motor of vehicle 1, a vehicle speed sensor for detecting the vehicle speed, wheel speed sensors, and GPS (Global Positioning System). Sensor 14 outputs the detected sensor information to vehicle control unit 10.

[0080] Camera 16 is a surrounding sensor mounted on vehicle 1 and monitoring the surrounding environment of vehicle 1. Camera 16 has an imaging element. In this embodiment, camera 16 captures images of the area around vehicle 1 and outputs the obtained surrounding image data to vehicle control device 10. The surrounding image data is photographic image data of the area around vehicle 1. Hereinafter, the surrounding image data will sometimes be referred to simply as surrounding image. In addition, in this embodiment, camera 16 is also suitable for detecting objects present in the area around vehicle 1 and estimating the location of vehicle 1 based on the positional relationship between vehicle 1 and the objects present in the area around vehicle 1.

[0081] The position, number, and shooting direction of the cameras 16 are pre-adjusted so that the cameras 16 can capture images of the area around the vehicle 1. For example, four cameras 16 are installed on the vehicle 1, configured to capture images in four directions: forward, backward, left, and right. Furthermore, the number of cameras 16 installed on the vehicle 1 is not limited to four.

[0082] Storage device 18 stores various types of data. In this embodiment, storage device 18 stores data such as map data 18B. Map data 18B is data that encodes and represents part or all of the conditions within the actual space. For example, map data used in a navigation system may be pre-stored in storage device 18 as map data 18B. Additionally, teaching path data 18A may also be stored in storage device 18. Teaching path data 18A is data composed of groups of driving information that are sequentially estimated as the current position during teaching driving.

[0083] Storage device 18 may be an auxiliary storage device such as an HDD (Hard Disk Drive), SSD (Solid State Drive), or flash memory. Alternatively, at least a portion of the data contained in storage device 18 may be stored in an external storage device such as a server device located outside the vehicle 1 and communicatively connected to the vehicle control device 10.

[0084] The operating device 20 includes a steering wheel, a gearshift lever for changing gears in the transmission, an accelerator pedal, a brake pedal, a turn indicator lever, push-button switches, and other operating mechanisms related to driving operations, as well as input devices such as a keyboard, touch panel, and switches. The steering angle of the steering system is adjusted by the passenger's operation of the steering control unit. At least one of the keyboard, touch panel, and switches functions as an automatic parking indicator unit that receives the start instruction for automatic parking. The steering wheel is an example of the steering control unit. The steering wheel is sometimes referred to as the driving wheel.

[0085] The gearshift lever is an example of a forward / reverse control unit. The forward / reverse control unit is used to switch between at least forward and reverse movement of vehicle 1. The brake pedal is an example of a braking control unit. The braking control unit is an operation unit for braking to suppress the speed of vehicle 1. That is, the braking control unit receives instructions to decelerate vehicle 1.

[0086] The operating device 20 may also be part of at least one of HMI (Human Machine Interface) and IVI (In-Vehicle Infotainment).

[0087] Alternatively, the operating device 20 may be configured to include a microphone, capable of accepting operations performed by the occupants (users) of vehicle 1 via voice input. Furthermore, in vehicle 1, a speaker for outputting notification tones, warning tones, and sounds may be provided instead of the display device 22, or in addition to the display device 22. In this case, notifications in the information processing involved in this disclosure can also be implemented through the output of sound, etc. Additionally, user operations such as selecting and specifying notifications can also be implemented through voice input.

[0088] Display device 22 is a display that outputs various images. Display device 22 is positioned in a location that allows passengers of vehicle 1 to visually view and confirm it. Examples of displays include liquid crystal displays (LCDs), organic EL (electro-luminescence) displays, projectors, etc. It can also be a touch panel display that integrates display device 22 and operating device 20. Display device 22 is an example of at least one of HMI and IVI.

[0089] The display device 22 is not limited to having only one display area. For example, the display device 22 may also have multiple display areas.

[0090] Alternatively, vehicle 1 may also have multiple display devices 22. For example, the display device 22 may include a first display unit and a second display unit, which are displays that output various images. The first display unit and the second display unit are separate display devices 22. The first display unit and the second display unit may also be arranged in different locations within vehicle 1. For example, the first display unit may function as an IVI (In-Vehicle Display), and the second display unit may function as part of the dashboard of vehicle 1.

[0091] Figure 2 This is an illustrative diagram illustrating an example of the configuration of sensor 14 and camera 16.

[0092] For example, four cameras 16 (cameras 16A to 16D) are installed in vehicle 1, so that the external conditions of vehicle 1 in at least four directions, namely front, rear, right and left, can be acquired.

[0093] Specifically, for example, camera 16 includes camera 16A, camera 16B, camera 16C, and camera 16D. Camera 16A is positioned at the front of vehicle 1, capturing images of the front of vehicle 1. Camera 16A is an example of a first camera. Camera 16A is sometimes referred to as a front-facing camera. Camera 16B is positioned at the right side of vehicle 1, capturing images of the right side of vehicle 1. Camera 16B is an example of a second camera. Camera 16C is positioned at the left side of vehicle 1, capturing images of the left side of vehicle 1. Camera 16C is an example of a third camera. Camera 16D is positioned at the rear of vehicle 1, capturing images of the rear of vehicle 1. Camera 16D is an example of a fourth camera. Camera 16D is sometimes referred to as a rear-facing camera or rear-end camera, etc.

[0094] Furthermore, the number of cameras 16 installed on vehicle 1 is not limited to four. Additionally, regarding the sensors 14 included for object detection, such as LiDAR, radar, sonar, and ultrasonic sensors, their placement and number are preferably pre-adjusted to enable the acquisition of external conditions to the right, left, front, and rear of vehicle 1. For example, as... Figure 2 As shown, sensor 14 includes sensors 14A to 14F. These sensors 14A to 14F are configured on vehicle 1 in a manner that enables the acquisition of external conditions to the right, left, front, and rear of vehicle 1. Furthermore, sensors 14 used for object detection, such as LiDAR, radar, sonar, and ultrasonic sensors, may also be configured only at the rear of vehicle 1.

[0095] Next, the structure of vehicle 1 will be described.

[0096] Figure 3 This is a schematic diagram illustrating an example of the structure of the exterior of vehicle 1.

[0097] Vehicle 1 has a body 2 and two pairs of wheels 23 arranged along a predetermined direction on the body 2. The two pairs of wheels 23 have a pair of front tires 23F and a pair of rear tires 23R (see also...) Figure 2 In addition, in Figure 2 and Figure 3 The example shown is a vehicle 1 with four wheels 23. However, the number of wheels 23 provided on vehicle 1 is not limited to this.

[0098] Next, the structure near the driver's seat of vehicle 1 in this embodiment will be described.

[0099] Figure 4 This is a diagram showing an example of the structure near the driver's seat 24A of the vehicle 1 in this embodiment.

[0100] Vehicle 1 has a driver's seat 24A and a passenger seat 24B. A windshield 25, an instrument panel 26, a steering wheel 20A, operation buttons 20B, and a display device 22 are provided in front of the driver's seat 24A. Additionally, a gearshift lever 20C, serving as a shift lever for the transmission, is provided near the driver's seat 24A.

[0101] The steering wheel 20A, operation buttons 20B, and gear lever 20C are examples of the operating device 20.

[0102] The steering wheel 20A is located in front of the driver's seat 24A and can be operated by the passenger. The rotation angle of the steering wheel 20A, i.e., the steering angle, is electrically or mechanically linked to the change in the orientation of the front tire 23F, which serves as the steering wheel. Alternatively, the steering wheel may be the rear tire 23R, or both the front tire 23F and the rear tire 23R may be steering wheels.

[0103] Operation button 20B is a button capable of receiving operations from the passenger. Additionally, operation button 20B may include a direction indicator. The location of operation button 20B is not limited to... Figure 4 The example shown could also be set on steering wheel 20A. Additionally, in Figure 4 The diagram shows one operation button 20B, but multiple operation buttons 20B can also be provided. When the display device 22 also functions as a touch panel, the display device 22 can also be an example of the operation device 20.

[0104] Return to Figure 1 Let's continue with the explanation.

[0105] The vehicle control device 10 is an electronic control unit that controls all parts of the vehicle 1 in a unified manner.

[0106] The vehicle control unit 10 uses sensor information and surrounding images received from the sensor 14 and camera 16 respectively to control the motion control unit 12 to optimize the driving state of the vehicle 1. Furthermore, the vehicle control unit 10 controls the motion control unit 12 to enable the vehicle 1 to drive autonomously.

[0107] The vehicle control device 10 includes a control unit 11. Part or all of the control unit 11 may be a software structure implemented through the cooperation of a processor and various programs stored in a memory. Alternatively, part or all of the control unit 11 may be a hardware structure implemented by dedicated circuitry or the like.

[0108] The control unit 11 provides unified control over all parts of the vehicle 1. In the vehicle control device 10, the CPU 11A, described later, performs at least one function, including the function of the control unit 11. The control unit 11 provides unified control over all parts of the vehicle 1.

[0109] The control unit 11 controls the steering, acceleration, deceleration, forward movement, and reverse movement of the vehicle 1, for example, through the motion control device 12. However, the motion control device 12 can also be used to control only the steering, allowing the passenger to operate the acceleration, deceleration, and forward movement.

[0110] In this embodiment, the control unit 11 performs information processing for parking assistance of the vehicle 1. In this embodiment, the following manner is envisioned and described: after the vehicle 1 is driven forward to pass the target parking position according to the driver's driving, the vehicle 1 is stopped according to the driver's driving, and the vehicle 1 reverses from the stopped position, i.e., the reversing start position, and moves towards the target parking position by autonomous driving.

[0111] Figure 5 This is an explanatory diagram of an example of parking assistance in this embodiment.

[0112] While the vehicle 1 is moving forward in the direction of travel (refer to the direction of arrow D, hereinafter sometimes referred to as the direction of travel D) according to the driver of the passenger, the control unit 11 detects the target parking position P1 of the vehicle 1 based on the surrounding images acquired by the camera 16.

[0113] In detail, the control unit 11 uses known methods to perform image analysis on the surrounding images acquired by the camera 16 to determine the parking spaces contained in the surrounding images. Then, the control unit 11 determines a space within the determined parking spaces that is suitable for the current position of the vehicle 1 as the target parking space S1.

[0114] The control unit 11 estimates the current position of vehicle 1 based on the time changes of sensor values ​​represented by sensor information obtained from sensor 14. For example, the control unit 11 calculates the amount of movement of vehicle 1 from a reference position, such as the starting position, based on the time changes of vehicle speed and yaw rate represented by sensor values, and estimates the current position of vehicle 1 based on the amount of movement. Alternatively, the control unit 11 may also use the result obtained by correcting the estimated current position based on the surrounding images of vehicle 1 acquired by camera 16 as the current position.

[0115] Furthermore, the control unit 11 determines the target parking space S1 as the space within the predetermined parking space contained in the surrounding images, which is estimated based on the relationship with the current position of vehicle 1, such that vehicle 1 can move forward and stop within that space, then reverse and park autonomously in a one-way manner. Parking autonomously in a one-way manner means parking from the reversing start position P2 to the target parking space S1 without changing the direction of vehicle 1. Details regarding the reversing start position P2 will be described later.

[0116] The control unit 11 detects the position within the predetermined target parking space S1 as the target parking position P1. The target parking position P1 is represented by two-dimensional coordinates along the horizontal direction in the actual space. The control unit 11 can detect the position coordinates of a specific location within the target parking space S1 as the target parking position P1. For example, the control unit 11 can also detect the position coordinates of the location estimated to be the center of vehicle 1 when it is parked in a manner that includes it within the target parking space S1 as the target parking position P1.

[0117] More specifically, the target parking position P1 detected by the control unit 11 is set along a direction having a predetermined angle θ relative to the travel direction D of the vehicle 1, and is located at a position on either the left LS1 or the right RS1 of the centerline C of the vehicle 1 along the travel direction D.

[0118] The specified angle θ refers to the angle between the direction of travel D of vehicle 1 when the target parking position P1 is detected and the virtual line X parallel to the long side of the target parking space S1 containing the target parking position P1. The virtual line parallel to the long side of the target parking space S1 is a virtual line parallel to the long side of the rectangular target parking space S1. In other words, the virtual line X parallel to the long side of the target parking space S1 is a line parallel to the entire length of vehicle 1 (the direction orthogonal to the vehicle width direction) already parked in the target parking space S1.

[0119] The specified angle θ is simply the angle formed by the intersection of the virtual line X and the direction of travel D of vehicle 1. The specified angle θ can be any of the following: between 10 and 170 degrees, 20 and 160 degrees, 30 and 150 degrees, 40 and 140 degrees, 50 and 130 degrees, 60 and 120 degrees, 70 and 110 degrees, and 80 and 100 degrees. An angle between 70 and 110 degrees is the most typical example of a specified angle θ. Figure 5 The example shown is the case where the specified angle θ is 90 degrees.

[0120] Setting the target parking position P1 along a direction with a specified angle θ means that the long side of the target parking space S1 containing the target parking position P1 is arranged parallel to a virtual line, which is a line obtained by tilting the vehicle 1's direction of travel D to the left LS1 or the right RS1 relative to the center line C by a specified angle θ.

[0121] The centerline C along the direction of travel D is a virtual line that passes through the center of the vehicle 1 in the width direction and is parallel to the direction of travel D of the vehicle 1.

[0122] The control unit 11 determines the following space within the predetermined parking space contained in the surrounding image as the target parking space S1: this space is estimated based on the relationship with the current position of the vehicle 1, allowing the vehicle 1 to move forward and stop within this space and then reverse, parking autonomously in a one-way manner. Furthermore, this space is positioned along a direction having a predetermined angle θ relative to the vehicle 1's direction of travel D, and is located at a position either LS1 to the left or RS1 to the right of the vehicle 1's centerline C along the direction of travel D. Then, the control unit 11 detects the position within the determined target parking space S1 as the target parking position P1.

[0123] When the control unit 1 detects a target parking position P1 while the vehicle 1 is moving forward according to the driver's direction, the display device 22 displays a screen that overlays the target parking position P1 and the corresponding reversing start position P2 onto the surrounding image. The reversing start position P2 displayed on the screen is located on the opposite side of the centerline C of the vehicle 1, between the left side LS1 and the right side RS1. That is, when the target parking position P1 is located at the position LS1 to the left of the centerline C of the vehicle 1, the corresponding reversing start position P2 is located at the position RS2 to the right of the centerline C. Similarly, when the target parking position P1 is located at the position RS1 to the right of the centerline C of the vehicle 1, the corresponding reversing start position P2 is located at the position LS1 to the left of the centerline C.

[0124] The control unit 11 can also cause the display device 22 to display the display screen when the first predetermined condition is met during the process of the vehicle 1 moving forward according to the driver's driving.

[0125] In detail, when the control unit 11 detects the target parking position P1, it determines whether a first predetermined condition is met. The first predetermined condition means that the vehicle speed of 1 is below a predetermined speed, and it has detected entering the parking area based on map data 18B or has received instruction input from the passenger.

[0126] The specified speed included in the first specified condition is any speed lower than the maximum speed for forward travel when parking in any parking space existing in the actual space RS. Alternatively, this specified speed can be appropriately changed according to the operation instructions of the operating device 20 by the passenger or the user who provides the vehicle 1. The control unit 11 uses the vehicle speed detection result included in the sensor information obtained from the sensor 14 to determine whether the vehicle speed of the vehicle 1 is lower than or equal to the specified speed.

[0127] The parking area included in the first specified condition, based on map data 18B, is the parking area for vehicle 1 represented by map data 18B. The parking area, for example, is a parking lot represented by map data 18B. The control unit 11 detects whether vehicle 1 has entered the parking area by determining whether its current position is within the parking area represented by map data 18B. The current position of vehicle 1 is estimated through the above processing.

[0128] The instruction input included in the first specified condition refers to an instruction input indicating a desire to park in any available parking space included in the actual space RS. The control unit 11 can determine whether it has accepted the instruction input by, for example, identifying whether the operation device 20 has received an instruction input signal indicating a desire to park in any parking space included in the actual space RS.

[0129] If the control unit 11 determines that the first predetermined condition is met, it derives the reversing start position P2 corresponding to the detected target parking position P1. Furthermore, the control unit 11 can also further derive the orientation of the vehicle 1 at the reversing start position P2.

[0130] The reversing start position P2 is a position suitable for vehicle 1 to reverse towards the predetermined target parking position P1 and park at the reversing start position P2. The position suitable for parking at the reversing start position P2 refers to a position in the actual space RS that allows the vehicle to reverse autonomously and park to the target parking space S1 in a one-way motion without requiring the vehicle 1 to change direction.

[0131] The control unit 11 only needs to derive the reversing start position P2 and the orientation of vehicle 1 at that reversing start position P2. The reversing start position P2 is a position where the vehicle can park in the predetermined target parking space S1 in a single stroke by autonomous driving based on the reversing method, and is located on the opposite side (either the left LS1 or the right RS1) of the centerline C of vehicle 1, which is different from the side where the detected target parking position P1 is located. For example, the control unit 11 can derive the reversing start position P2 and the orientation of vehicle 1 by using a learning model that takes the current position of vehicle 1 and the target parking position P1 as inputs and outputs the reversing start position P2 and the orientation of vehicle 1 at the reversing start position P2, which satisfy the above conditions.

[0132] Then, the control unit 11 causes the display device 22 to display a screen that overlays the determined target parking position P1 and the derived reversing start position P2 onto the surrounding image.

[0133] Figure 6 This is a schematic diagram of an example of the display screen 30 displayed by the display device 22.

[0134] Display screen 30 is a display screen that overlays images representing the target parking position P1 and the reversing start position P2 onto the surrounding image V.

[0135] When the first predetermined condition is met during the process of the passenger moving the vehicle 1 forward, the control unit 11 causes the display device 22 to display a display screen 30 that superimposes the target parking position P1 and the reversing start position P2 corresponding to the target parking position P1 onto the surrounding image V.

[0136] For example, the target parking position P1 displayed on the display device 22 is represented by a first frame F1. Additionally, the reversing start position P2 displayed on the display device 22 is represented by a second frame F2. The first frame F1 indicates the position of vehicle 1 at the target parking position P1 and the orientation of vehicle 1. The second frame F2 indicates the position of vehicle 1 at the reversing start position P2 and the orientation of vehicle 1. The shapes of the first frame F1 and the second frame F2 are, for example, rectangular shapes that enclose the vehicle 1 when visually confirmed from above.

[0137] Control unit 11 generates a virtual frame VF1 (referencing) that surrounds vehicle 1 when it is estimated to be parked in the determined target parking space S1. Figure 5 The image is used as the first frame F1. Then, the control unit 11 configures the first frame F1 at the position corresponding to the target parking position P1 of the target parking space S1 in the peripheral image V displayed on the display screen 30, so as to indicate the tilt angle of the vehicle 1 when it is parked.

[0138] Additionally, the control unit 11 creates a virtual frame VF2 (see reference) that surrounds the vehicle 1 when it is estimated that the vehicle 1 has stopped at the derived reversing start position P2. Figure 5 The image is used as the second frame F2. Then, the control unit 11 configures the second frame F2 at the position corresponding to the determined reversing start position P2 in the peripheral image V displayed on the display screen 30, so as to indicate the tilt of the vehicle 1 at the derived reversing start position P2.

[0139] like Figure 6 As shown, the first frame F1 representing the target parking position P1 is displayed in display screen 30 (also refer to...). Figure 5 The target parking position P1 is set along a direction with a predetermined angle θ relative to the travel direction D of vehicle 1, and is located at a position on the left LS1 or right RS1 relative to the centerline C of vehicle 1 along the travel direction D. Additionally, the second frame F2 displayed on display screen 30, indicating the reversing start position P2, is located on the right RS1 relative to the centerline C of vehicle 1, opposite to the left LS1 of the detected target parking position P1.

[0140] The reversing start position P2 being located on the same side as the target parking position P1 (i.e., on the opposite side), means that the second frame F2, representing the reversing start position P2, is located on the opposite side from the center line C (towards the center line C, i.e., to the left of the center line C, LS1). Figure 6 In the middle, the position is RS1, which is to the right of the center line C.

[0141] The display screen 30 may also include a line image Lc representing the centerline C of the vehicle 1 along the direction of travel D.

[0142] Alternatively, display screen 30 can also be a screen that displays an icon image representing vehicle 1 at the current position of vehicle 1. Figure 6 The following state is shown: an icon image representing vehicle 1 is placed at the current position of vehicle 1 in the peripheral image V, and the first frame F1, the second frame F2, and the line image Lc representing the center line C are superimposed on the peripheral image V.

[0143] The control unit 11, by displaying a display screen 30 on the display device 22 that overlays the first frame F1 and the second frame F2 onto the surrounding image V, can provide the passenger with the first frame F1 indicating the target parking position P1 and the second frame F2 indicating the reversing start position P2 in a manner that is easily verifiable. That is, the passenger can visually confirm the first frame F1 and the second frame F2 displayed on the display screen 30 while the vehicle 1 is moving forward according to the passenger's driving, thereby easily confirming the target parking position P1 indicated by the first frame F1 and the reversing start position P2 indicated by the second frame F2.

[0144] In addition, the control unit 11 can also cause the display device 22 to display a display screen 30 that also superimposes at least one of the following: an estimation frame F3, a deceleration estimation position frame F4, and an image Ia indicating the recommended steering state, while the vehicle 1 is moving forward according to the driver's driving.

[0145] In detail, when the control unit 11 determines that the second specified condition is met during the process of the vehicle 1 moving forward according to the driver's driving, the display device 22 displays a display screen 30 that also superimposes at least one of the following: an estimation frame F3, a deceleration estimation position frame F4, and an image Ia indicating the recommended steering state.

[0146] The second specified condition refers to: receiving an instruction input from the passenger indicating a request to begin parking, or detecting a change in steering direction towards the reverse start position P2 caused by the passenger's driving. An example of an instruction input from the passenger indicating a request to begin parking could be the passenger pressing a button on the hazard warning light indicator included in the operating device 20.

[0147] The passenger inputs a parking start request by operating the operating device 20. The control unit 11 determines whether it has accepted the instruction input signal indicated by the instruction input signal by detecting whether the operating device 20 has received the parking start request instruction input signal.

[0148] In addition, the control unit 11 can detect whether a change in steering direction has occurred due to the passenger's driving by analyzing the sensor information obtained from the sensor 14 and the passenger's operation received by the operating device 20.

[0149] Then, when the control unit 11 determines that the second specified condition is met during the process of the vehicle 1 moving forward according to the driver's driving, the display device 22 displays a display screen 30 that also superimposes at least one of the estimation frame F3, the deceleration estimation position frame F4, and the image Ia indicating the recommended steering state.

[0150] The estimation box F3 is a frame image showing the estimated position P3 and estimated posture of vehicle 1 when it continues to move forward at the current steering angle based on the driver's direction. The deceleration estimation position box F4 is a frame image showing the recommended position for vehicle 1 to decelerate from the current position to the reverse start position P2. The image Ia showing the recommended steering state is an image showing the recommended steering state corresponding to the recommended steering angle for vehicle 1 to move from the current position to the reverse start position P2. The shapes of the estimation box F3 and the deceleration estimation position box F4 are, for example, the rectangular shape surrounding vehicle 1 when visually confirmed from above.

[0151] Return to Figure 5 Let's continue with the explanation.

[0152] The control unit 11 generates the estimated frame F3 through the following processing. First, the control unit 11 estimates the current position of the vehicle 1 in the same manner as described above. Furthermore, the control unit 11 uses the current vehicle speed, acceleration, steering angle, and accelerator pedal opening of the vehicle 1, as represented by sensor information detected by the sensor 14, to derive an estimated position P3 and an estimated posture that, if the vehicle continues to move forward at the current steering angle, it will travel a predetermined distance forward from the current position. The estimated posture refers to the estimated orientation of the vehicle 1. For this predetermined distance, the distance from the current position to the starting position P2 of the reverse movement can be used. Alternatively, a value obtained by adding or subtracting a predetermined value from this distance can also be used. For example, the control unit 11 can derive the estimated position P3 and the estimated posture using a learning model that takes the sensor information of the vehicle 1 as input and outputs the estimated position P3 and the estimated posture that satisfy the above conditions.

[0153] Then, the control unit 11 generates an image of a virtual frame VF3 representing the vehicle 1 when it is located at the derived estimated position P3 in the derived estimated posture, as the estimated frame F3 (see reference). Figure 6 Then, as Figure 6As shown, the control unit 11 can configure the estimation frame F3 by indicating the tilt of the vehicle 1 when the vehicle 1 is positioned in the peripheral image V displayed on the display screen 30 at the position corresponding to the estimated position P3, which represents the estimated posture of the vehicle 1.

[0154] The control unit 11, by displaying a screen 30 on the display device 22 that also overlays the estimated frame F3 onto the peripheral image V, can provide the passenger with the estimated position P3 and estimated posture of the vehicle 1 when it continues to move forward at the current steering angle according to the passenger's driving, and the deviation between the estimated position P3 and estimated posture and the reverse start position P2 and the orientation of the vehicle 1 when it is in the reverse start position P2. That is, the passenger can visually confirm the estimated frame F3 and the second frame F2 displayed on the display screen 30 while the vehicle 1 is moving forward according to the passenger's driving, thereby easily confirming the deviation of the second frame F2 relative to the second frame F2.

[0155] Furthermore, as described above, the estimation frame F3 is a frame image representing the estimated position P3 and estimated posture of vehicle 1 when it continues to move forward at the current steering angle based on the occupant's driving. Therefore, the estimation frame F3 is superimposed on the position corresponding to the occupant's steering wheel control in the peripheral image V displayed on the display screen 30.

[0156] Therefore, preferably, when the vehicle 1 is moving forward according to the driver's direction, and at least a portion of the estimation frame F3 overlaps with the second frame F2, the control unit 11 also displays text prompting continued forward movement with a fixed steering angle on the display screen 30. Alternatively, when the vehicle 1 is moving forward according to the driver's direction, and at least a portion of the estimation frame F3 overlaps with the second frame F2, the control unit 11 may display the estimation frame F3 and at least one of them in a way that indicates the overlap, such as by emphasizing the overlapping portion of the estimation frame F3 and the second frame F2.

[0157] Furthermore, the control unit 11 preferably causes the display device 22 to display a display screen 30 that also includes an image Ia indicating the recommended steering state. As described above, the image Ia indicating the recommended steering state is an image representing the recommended steering state. Figure 6 As an example, this example shows a case where the image Ia representing the recommended steering state is represented by an image Ib representing the steering wheel and an arrow image Ic representing the recommended direction of rotation of the steering wheel. The control unit 11 displays the arrow image Ic, which is represented by an arrow whose length increases as the required rotation angle of the steering wheel increases.

[0158] As the vehicle 1 moves forward according to the driver's direction, the control unit 11 calculates, in the same manner as described above, a recommended path for the vehicle 1 to move from its current position to the reverse start position P2 without changing direction. Then, the control unit 11 causes the display device 22 to display a display screen 30 that also includes an image Ia showing the recommended steering state for driving along the recommended path.

[0159] The control unit 11, by displaying a screen 30 on the display device 22 that also includes an image Ia indicating a recommended steering state, can provide the passenger with a recommended steering state for moving towards the reverse starting position P2 in a manner that is easily verifiable while the vehicle 1 is moving forward according to the passenger's driving. That is, while the vehicle 1 is moving forward according to the passenger's driving, the passenger can visually confirm the image Ia indicating the recommended steering state displayed on the display screen 30, and by steering according to the steering state indicated by the image Ia, the vehicle 1 can be easily moved to the reverse starting position P2.

[0160] In addition, the control unit 11 generates the deceleration estimation position box F4 through the following processing.

[0161] First, the control unit 11 estimates the current position of the vehicle 1 in the same manner as described above. Then, the control unit 11 determines the area recommended for deceleration to reach the reverse start position P2 from the estimated current position without performing any steering maneuvers on the vehicle 1. The area recommended for deceleration includes the region from the position where the passenger is advised to begin operating the operating device 20 with at least one of the accelerator pedal and brake pedal for deceleration to the position where the operation is recommended to end.

[0162] For example, the control unit 11 calculates a recommended path from the current position to the reverse start position P2 without reversing steering using a known method. Then, using the calculated recommended path, the control unit 11 calculates the required vehicle speed for each position on the recommended path to reach the reverse start position P2, assuming the vehicle is traveling forward from the current position along the recommended path towards the reverse start position P2. The control unit 11 then determines the position closest to the vehicle 1 from the group of positions on the recommended path whose calculated speed is less than the current vehicle 1 speed, as a first position recommended for initiating the deceleration operation. Then, when the deceleration operation begins at the determined position, the control unit 1 determines a second position on the recommended path where the estimated vehicle 1 speed matches the speed calculated for each position on the recommended path. The control unit 11 then defines a region on the recommended path that includes at least the area from the determined first position to the second position as a deceleration recommended region. Control unit 11 generates an image representing a virtual bounding box VF4 along the periphery of the determined deceleration recommendation area as the deceleration estimation position box F4 (see reference). Figure 6 Then, as Figure 6 As shown, the control unit 11 can configure the deceleration estimation position box F4 at the position corresponding to the deceleration recommendation area in the peripheral image V displayed on the display screen 30.

[0163] The control unit 11 causes the display device 22 to display the deceleration estimation position frame F4, thereby providing the passenger with a recommended deceleration position in a verifiable manner. That is, the passenger can easily confirm the recommended deceleration position by checking the deceleration estimation position frame F4 displayed on the display screen 30 while the vehicle 1 is moving according to the passenger's driving.

[0164] Preferably, when the control unit 11 determines that the vehicle 1 has moved forward according to the driver's driving and has reached the reverse start position P2, the display device 22 also displays text indicating that the reverse start position P2 has been reached.

[0165] After vehicle 1 moves forward according to the driver's direction and reaches the reverse start position P2, control unit 11 controls at least the steering wheel to reverse vehicle 1 and moves vehicle 1 from the reverse start position P2 to the target parking position P1 by autonomous driving.

[0166] For example, after vehicle 1 moves forward according to the driver's direction and reaches the reverse start position P2, and the operating device 20 accepts the prescribed operation, the control unit 11 at least controls the steering wheel to reverse vehicle 1, and moves vehicle 1 from the reverse start position P2 to the target parking position P1 by autonomous driving.

[0167] The prescribed operation refers to a predetermined operation for initiating a reverse movement from the reverse start position P2. For example, the prescribed operation means shifting the gear to reverse by the passenger's operation of the gear lever included in the operating device 20.

[0168] In this case, the control unit 11 can determine whether the prescribed operation has been accepted by identifying whether the operating device 20 has received a signal indicating the prescribed operation.

[0169] In addition, in this case, it is preferable that when the control unit 11 determines that the vehicle 1 has moved forward according to the driver's driving and has reached the reverse start position P2, the display device 22 displays text indicating that the reverse start position P2 has been reached and text prompting the specified operation to be performed.

[0170] Then, when the operating device 20 accepts the prescribed operation, the control unit 11 at least controls the steering to make the vehicle 1 reverse, and moves the vehicle 1 from the reversing start position P2 to the target parking position P1 by autonomous driving.

[0171] For example, the control unit 11 derives a recommended path for moving vehicle 1 from the reversing starting position P2 to the target parking position P1 via autonomous driving using a known method. Then, the control unit 11 compares the current position of vehicle 1 in the actual space RS with the recommended path, at least controlling the steering to make the current position of vehicle 1 on the recommended path, and controlling the movement control device 12 to drive along the recommended path. During autonomous driving, vehicle 1 is automatically controlled by the vehicle control device 10 without the need for manual driving operations by the passenger.

[0172] Additionally, for example, the control unit 11 estimates the current position of vehicle 1 in the actual space RS in the same manner as described above. Furthermore, the control unit 11 uses at least one of the sensor information acquired by sensor 14 and the surrounding image V acquired by camera 16 to determine obstacles around the path from the reversing start position P2 to the target parking position P1. Then, the control unit 11 controls the steering by controlling the movement control device 12 to autonomously drive from the reversing start position P2 to the target parking position P1 while avoiding the identified obstacles, and moves vehicle 1 from the reversing start position P2 towards the target parking position P1 through autonomous driving.

[0173] Furthermore, as described above, the reversing start position P2 is a position suitable for vehicle 1 to reverse towards the predetermined target parking position P1 and park at the reversing start position P2. The suitable position for parking at the reversing start position P2 is a position in the actual space RS where the vehicle can reverse autonomously and park to the target parking space S1 in a single stroke without requiring the vehicle 1 to change direction.

[0174] Therefore, when the driving environment around vehicle 1 when it reaches the reversing start position P2 is roughly the same as the driving environment when the target parking position P1 is detected and the reversing start position P2 is derived, vehicle 1 can move from the reversing start position P2 to the target parking position P1 by autonomous driving in a one-way trip, and thus park at the target parking position P1.

[0175] When the current position of vehicle 1 is approximately the same as the target parking position P1, the control unit 11 determines that vehicle 1 has reached the target parking position P1 and completes parking.

[0176] Next, an example of the information processing flow performed by the control unit 11 of the vehicle control device 10 will be described.

[0177] Figure 7 This is a flowchart illustrating an example of the information processing flow performed by the control unit 11.

[0178] When vehicle 1 begins to move forward according to the driver's direction (step S100), control unit 11 determines whether a first predetermined condition is met (step S102). As described above, the first predetermined condition indicates that vehicle 1's speed is below a predetermined speed, and that it has detected entering a parking area based on map data 18B or has received an instruction input from the driver. In step S102, a negative judgment is repeatedly performed (step S102: "No") until a positive judgment is made (step S102: "Yes"). When a positive judgment is made (step S102: "Yes"), the process proceeds to step S104.

[0179] In step S104, the control unit 11 detects the target parking position P1 of the vehicle 1 based on the surrounding images acquired by the camera 16 (step S104).

[0180] Next, in step S106, the control unit 11 causes the display device 22 to display the surrounding image V of the vehicle 1 (step S106). Then, the control unit 11 outputs the reversing start position P2 corresponding to the target parking position P1 detected in step S102, and the orientation of the vehicle 1 at the reversing start position P2 (step S108).

[0181] Then, the control unit 11 overlays the first frame F1, which represents the target parking position P1 detected in step S102, and the second frame F2, which represents the reversing start position P2 derived in step S108 and the orientation of the vehicle 1 at the reversing start position P2, onto the peripheral image V displayed on the display device 22 (step S110).

[0182] Next, the control unit 11 determines whether the second predetermined condition is met (step S112). As described above, the second predetermined condition refers to: receiving an instruction input from the passenger indicating a parking start request, or detecting a change in steering direction towards the reverse start position P2 caused by the passenger's driving. When a negative determination is made in step S112 (step S112: "No"), the process returns to step S102. When a positive determination is made in step S112 (step S112: "Yes"), the process proceeds to step S114.

[0183] In step S114, the control unit 11 causes the display device 22 to display a display screen 30 that also overlays at least one of the estimation frame F3, the deceleration estimation position frame F4, and the image Ia indicating the recommended steering state (step S114).

[0184] Next, the control unit 11 determines whether the estimated box F3 and the second box F2 overlap by at least a portion (step S116). When a negative judgment is made in step S116 (step S116: "No"), the process proceeds to step S120, which will be described later. When a positive judgment is made in step S116 (step S116: "Yes"), the process proceeds to step S118.

[0185] In step S118, the control unit 11 causes the display screen 30 to also display text prompting continued forward movement with the steering angle fixed (step S118). Then, it proceeds to step S120.

[0186] In step S120, the control unit 11 determines whether the vehicle 1 has reached the reversing start position P2 (step S120). If a negative judgment is made in step S120 (step S120: "No"), the process proceeds to step S116. If a positive judgment is made in step S120 (step S120: "Yes"), the process proceeds to step S122.

[0187] In step S122, the control unit 11 causes the display device 22 to display text indicating that the backward start position P2 has been reached, and text prompting the performance of a predetermined operation indicating a start instruction to begin backward movement from the backward start position P2 (step S122).

[0188] Then, the control unit 11 determines whether the operating device 20 has accepted the prescribed operation (step S124). The control unit 11 repeats the negative judgment (step S124: "No") until a positive judgment is made in step S124 (step S124: "Yes"). When the control unit 11 makes a positive judgment in step S124 (step S124: "Yes"), it proceeds to step S126.

[0189] The control unit 11 controls at least the steering wheel to reverse the vehicle 1, and then moves the vehicle 1 from the reversing start position P2 to the target parking position P1 through autonomous driving (step S126). Then, when the current position of the vehicle 1 is approximately consistent with the target parking position P1, the control unit 1 determines that the vehicle 1 has reached the target parking position P1 and completes parking, thus ending the routine.

[0190] As explained above, the vehicle control device 10 of this embodiment is mounted on a vehicle 1, which includes an operating device 20 for receiving passenger operations, a camera 16 for acquiring surrounding images V, a display device 22 for the passenger to visually confirm, and a motion control device 12 for controlling at least the steering wheel. While the vehicle 1 is traveling forward in the direction D according to the passenger's driving, the vehicle control device 10 detects the target parking position P1 of the vehicle 1 based on the surrounding images V acquired by the camera 16, and displays a screen 30 on the display device 22 that overlays the target parking position P1 and the corresponding reversing start position P2 onto the surrounding images V. Then, after the vehicle 1 travels forward according to the passenger's driving and reaches the reversing start position P2, the vehicle control device 10 controls at least the steering wheel to reverse the vehicle 1, and autonomously moves the vehicle 1 from the reversing start position P2 towards the target parking position P1. The detected target parking position P1 is set along a direction with a specified angle θ relative to the direction of travel D, and is located either to the left of the centerline C of vehicle 1 along the direction of travel D, LS1 or to the right of the centerline RS1. The reverse start position P2 displayed on display screen 30 is located at the other position, either to the left of the centerline C, LS2 or to the right of the centerline RS1.

[0191] In the prior art, sometimes the vehicle 1 stops at a reversing starting position where it is difficult to autonomously move towards the target parking position, depending on the driver's actions. In other words, in the prior art, there are situations where appropriate parking assistance is difficult to provide.

[0192] On the other hand, during the process of the vehicle 1 moving forward according to the driver's direction, the vehicle control device 10 of this embodiment displays a screen 30 on the display device 22 that overlays the target parking position P1 and the reversing start position P2 to which the vehicle can autonomously move towards the target parking position P1 onto the surrounding image V. Therefore, by visually confirming the display screen 30 while driving the vehicle 1 forward, the driver can easily confirm the reversing start position P2 to which the vehicle can autonomously move towards the target parking position P1 and perform driving operations towards the reversing start position P2. Therefore, when the vehicle 1 moves forward according to the driver's direction, the vehicle control device 10 of this embodiment can easily guide the vehicle 1 driven by the driver to a suitable reversing start position P2 to which the vehicle can autonomously move towards the target parking position P1.

[0193] Therefore, the vehicle control device 10 of this embodiment can provide appropriate parking assistance.

[0194] (Second Implementation)

[0195] In this embodiment, the control is specifically described when vehicle 1, which is stopped at the reversing stop position in order to park at the target parking position P1, cannot park at the target parking position P1 in a single trip.

[0196] Figure 1 This is a block diagram illustrating an example of the overall structure of vehicle 1B according to this embodiment.

[0197] Vehicle 1B is identical to vehicle 1 in the above-described embodiment, except that it has vehicle control device 10B instead of vehicle control device 10. Therefore, the configuration of sensor 14 and camera 16 in vehicle 1B is also the same as in vehicle 1 (see [reference]). Figure 2 Furthermore, the exterior of vehicle 1B and the structure near the driver's seat 24A are the same as those of vehicle 1 (see reference). Figure 3 , Figure 4 ).

[0198] Vehicle control unit 10B is an electronic control unit that uniformly controls all parts of vehicle 1B. Similar to vehicle control unit 10, vehicle control unit 10C uses sensor information and surrounding images received from sensor 14 and camera 16 respectively to control motion control unit 12 to optimize the driving state of vehicle 1B. Furthermore, vehicle control unit 10B controls motion control unit 12 to enable vehicle 1B to drive autonomously.

[0199] The vehicle control device 10B includes a control unit 11B. Part or all of the control unit 11B can be a software structure implemented through the cooperation of a processor and various programs stored in a memory. Alternatively, part or all of the control unit 11B can also be a hardware structure implemented by dedicated circuitry or the like.

[0200] Control unit 11B provides unified control over all parts of vehicle 1B.

[0201] In this embodiment, the control unit 11B performs information processing for parking assistance of the vehicle 1B. In this embodiment, the following manner is envisioned and described: after the vehicle 1B is driven forward according to the passenger's driving until at least a portion of the vehicle body of the vehicle 1B has passed at least a portion of the target parking position P1, the vehicle 1B is stopped according to the passenger's driving, the vehicle 1B is reversed from the reversing stop position which is the stop position, and the vehicle 1B is moved toward the target parking position P1 by autonomous driving.

[0202] Furthermore, in this embodiment and subsequent embodiments, the backward starting position P2 in the above embodiment will be referred to as the turning reference position P2 for explanation.

[0203] Figure 8A and Figure 8B This is an explanatory diagram of an example of parking assistance in this embodiment.

[0204] Control unit 11B during the process of vehicle 1B moving forward according to the driver's direction (see reference) Figure 8A (In the direction of arrow D in the image), the control unit 11B detects the available parking space for vehicle 1B based on the surrounding images acquired by camera 16. Then, the control unit 11B determines a suitable space within the determined available parking space that is appropriate for the current position of vehicle 1 as the target parking space S1. In addition, the control unit 11B detects the position within the determined target parking space S1 as the target parking position P1. The method for detecting the available parking space, the method for determining the target parking space S1, and the method for detecting the target parking position P1 are the same as in the above embodiment.

[0205] When the control unit 11B detects a target parking position P1 while the vehicle 1B is moving forward according to the driver's direction, it causes the display device 22 to display a display screen 30 that overlays the target parking position P1 and the corresponding U-turn reference position P2 onto the surrounding image V. This display screen 30 is an example of a first display screen.

[0206] For example, the control unit 11B may also cause the display device 22 to display the display screen 30 as the first display screen when the first predetermined condition is met during the process of the vehicle 1B moving forward according to the driver's driving. The first predetermined condition is the same as in the above embodiment. That is, the first predetermined condition means that the vehicle speed of 1 is below a predetermined speed, and the vehicle has detected entering a parking area based on map data 18B or has received an instruction input from the passenger indicating that they wish to park.

[0207] When the control unit 11B determines that the first predetermined condition is met, it derives a U-turn reference position P2 corresponding to the detected target parking position P1. Furthermore, the control unit 11B can also further derive the orientation of the vehicle 1B at the U-turn reference position P2. The method for deriving the U-turn reference position P2 and the orientation of the vehicle 1B at that U-turn reference position P2 is the same as the method for deriving the reversing start position P2 and the orientation of the vehicle 1B at that reversing start position P2 described in the above embodiment.

[0208] As described in the above embodiment, the U-turn reference position (reverse start position) P2 is the position suitable for vehicle 1B to reverse towards the determined target parking position P1 and park at the target parking position P1. The position suitable for parking at the target parking position P1 is the position in the actual space RS where the vehicle can park in a one-way trip by autonomous driving.

[0209] The ability to park in a single stroke means that the vehicle can be parked in the target parking position P1 by simply reversing until reaching the target parking space S1 without changing the direction of the vehicle 1.

[0210] Then, the control unit 11B causes the display device 22 to display a screen 30 that overlays the determined target parking position P1 and the derived U-turn reference position P2 onto the surrounding image. For example, the control unit 11B will... Figure 3 The display screen 30A shown is displayed on the display device 22 as the first display screen. Display screen 30A is an example of display screen 30. Figure 3 The display screen 30A shown is the same as in the above embodiment. Furthermore, in this embodiment, the estimation box F3 and the deceleration estimation position box F4 may not be displayed.

[0211] The control unit 11B, by displaying a display screen 30B on the display device 22 that overlays the first frame F1 and the second frame F2 onto the surrounding image V, can provide the passenger with the first frame F1 indicating the target parking position P1 and the second frame F2 indicating the U-turn reference position P2 in a manner that is easily verifiable. That is, the passenger can visually confirm the first frame F1 and the second frame F2 displayed on the display screen 30 while the vehicle 1B is moving forward according to the passenger's driving, thereby easily confirming the target parking position P1 indicated by the first frame F1 and the reverse start position P2 indicated by the second frame F2.

[0212] Return to Figure 8A and Figure 8B Let's continue with the explanation.

[0213] After the vehicle 1B has moved forward according to the driver's direction, the control unit 11B determines whether it has stopped at the reversing stop position TP. The stopping of the vehicle 1B at the reversing stop position TP is performed by an operation received by the operating device 20. That is, the vehicle 1B stops at the reversing stop position TP based on the braking operation performed by the driver.

[0214] The reversing stop position TP is the position of vehicle 1B in the actual space RS when it stops after the control unit 11B has derived the target parking position P1 and the U-turn reference position P2. In other words, the reversing stop position TP is the stopping position when vehicle 1B stops according to the driver's driving after at least a part of the vehicle body has passed at least a part of the area of ​​the target parking position P1. Alternatively, the reversing stop position TP can also be the position of vehicle 1B in the actual space RS when it stops after the control unit 11B has derived the target parking position P1 and the U-turn reference position P2 and displayed the display screen 30A, which is the first display screen mentioned above.

[0215] Alternatively, the reversing stop position TP can also be the position of vehicle 1B in the actual space RS after the control unit 11B derives the target parking position P1 and the turning reference position P2 and displays the display screen 30A, which is the first display screen mentioned above, stops and the gear of vehicle 1B is shifted to reverse (R).

[0216] It is not limited to the method where the reversing stop position TP and the turning reference position P2 are the same.

[0217] For example, after the control unit 11B outputs the target parking position P1 and the U-turn reference position P2, it determines whether the vehicle 1B has stopped at the reversing stop position TP by distinguishing whether the vehicle 1B decelerates and stops according to the passenger's braking operation.

[0218] Then, when vehicle 1B has stopped at the reversing stop position TP and the operating device 20 has received the prescribed operation, the control unit 11B determines that it has been instructed to perform rapid rear parking, which means parking at the target parking position P1 by autonomous driving. As explained in the above embodiment, the prescribed operation is a pre-determined operation indicating the start of reversing from the U-turn reference position P2. For example, the prescribed operation means shifting the gear to reverse by the passenger operating the gear lever included in the operating device 20.

[0219] When vehicle 1B has stopped at the reversing stop position TP, control unit 11B determines whether it is possible to perform one-way parking from the reversing stop position TP to the target parking position P1 by simply reversing. Specifically, when vehicle 1B has stopped at the reversing stop position TP and the gear has been shifted to reverse by the passenger's operation of the gear lever, control unit 11B determines whether it is possible to park to the target parking position P1 by simply reversing.

[0220] The control unit 11B derives a virtual path using known methods for autonomously moving the vehicle 1B from the reversing stop position TP to the target parking position P1 by reversing only. Furthermore, for example, the control unit 11B determines whether parking at the target parking position P1 can be achieved by reversing only in a single trip by identifying whether the derived virtual path represents a path for parking at the target parking position P1 by reversing only in a single trip.

[0221] Consider the following scenario: Control unit 11B determines that one-way parking is possible, where the vehicle moves from the reversing stop position TP to the target parking position P1 by reversing only. That is, in this case, if... Figure 8A As shown, vehicle 1B stops at the first reversing stop position TP1, which is a reversing stop position TP that can be parked in a single stroke toward the target parking position P1 by simply reversing.

[0222] When vehicle 1B has stopped at the first reversing stop position TP1 and is capable of performing a one-way parking maneuver by reversing to the target parking position P1, the control unit 11B at least controls the steering to make vehicle 1B reverse and autonomously drive to the target parking position P1.

[0223] For example, the control unit 11B generates a reverse path R1 using a known method, which allows the vehicle 1B to move autonomously from the first reversing stop position TP1 to the target parking position P1 in a single stroke and park. Then, the control unit 11B compares the current position of the vehicle 1B in the actual space RS with the reverse path R1, controls the steering to make the current position of the vehicle 1B a position on the reverse path R1, and controls the movement control device 12 to travel along the reverse path R1. During autonomous driving, the vehicle 1B is automatically controlled by the vehicle control device 10 without the need for manual driving operations by the passenger.

[0224] On the other hand, consider the following situation: the control unit 11B determines that it is impossible to perform a one-way parking maneuver by simply reversing from the reversing stop position TP to the target parking position P1.

[0225] That is, in this case, such as Figure 8BAs shown, vehicle 1B stops at the second reversing stop position TP2, which is a reversing stop position TP that cannot be parked in a single stroke toward the target parking position P1 by reversing only.

[0226] If vehicle 1B has stopped at the second reversing stop position TP2 and cannot perform a one-way parking maneuver by simply reversing to the target parking position P1, the control unit 11B shall at least control the steering wheel to reverse vehicle 1B, and then autonomously drive vehicle 1B to reverse to the third reversing stop position TP3.

[0227] The third reversing stop position TP3 is a position different from the target parking position P1. In detail, the third reversing stop position TP3 is a position that can be reached from the second reversing stop position TP2 in a one-way trip by autonomous driving in reverse mode, and it is the position where vehicle 1B can reverse to park in the target parking space S1 after reversing and driving forward at the third reversing stop position TP3.

[0228] Furthermore, the third reversing stop position TP3 is preferably a camera 16D (refer to) located at the rear of vehicle 1B and capturing images of the rear of vehicle 1B when vehicle 1B is reversing steering and moving forward at this third reversing stop position TP3. Figure 2 The obtained photographic image will capture the position of the opening endpoint FP of the target parking space S1. The opening endpoint FP refers to the end of the vehicle 1B in the target parking space S1 on the entry side when parking.

[0229] By setting the position at which the vehicle will be hit by the opening endpoint FP when moving forward from the third reversing stop position TP3 as the third reversing stop position TP3, the parking accuracy when parking at the target parking position P1 by autonomous driving after this forward movement can be improved.

[0230] If vehicle 1B has stopped at the second reversing stop position TP2 and cannot perform a one-way parking maneuver by reversing towards the target parking position P1, the control unit 11B derives a position on the actual space RS that satisfies the above conditions as the third reversing stop position TP3. Furthermore, the control unit 11B can also further derive the orientation of vehicle 1B at the third reversing stop position TP3.

[0231] For example, the control unit 11B uses a learning model that takes the target parking position P1 and the second reversing stop position TP2, which is the current position of the vehicle 1B, as inputs and the third reversing stop position TP3 and the orientation of the vehicle 1B at the third reversing stop position TP3 as outputs, to derive the third reversing stop position TP3 and the orientation of the vehicle 1B.

[0232] Then, the control unit 11B generates a reversing path R2 from the second reversing stop position TP2 to the third reversing stop position TP3. The control unit 11B generates the one-way reversing path R2 from the second reversing stop position TP2 to the third reversing stop position TP3 based on the surrounding image V and by using known methods such as a learning model.

[0233] If vehicle 1B has stopped at the second reversing stop position TP2 and cannot perform a one-way parking maneuver by reversing to the target parking position P1, the control unit 11B causes the display device 22 to display at least one of the following: information indicating that the vehicle is reversing to the third reversing stop position TP3 by autonomous driving, and information indicating that the vehicle cannot park directly from the current position to the target parking position P1.

[0234] Figure 9A This is a schematic diagram of an example of display screen 30B. Display screen 30B is an example of display screen 30 displayed on display device 22.

[0235] When vehicle 1B has stopped at the second reversing stop position TP2 and cannot perform a one-way parking maneuver by reversing to the target parking position P1, control unit 11B causes display device 22 to display display screen 30B.

[0236] Display screen 30B contains information M1. Information M1 is a display of at least one of the following: information indicating that the vehicle is reversing towards the third reversing stop position TP3 via autonomous driving, and information indicating that it is impossible to park directly from the current position to the target parking position P1. The information in information M1 indicating that the vehicle is reversing towards the third reversing stop position TP3 via autonomous driving may also simply indicate that the vehicle is reversing towards the third reversing stop position TP3. Alternatively, information M may also include information indicating that the vehicle should be manually moved forward after reversing to the third reversing stop position TP3.

[0237] exist Figure 9A In this example, information M1 is shown as text representing the following information: information indicating reversing to the third reversing stop position TP3; information indicating that it is impossible to park directly from the current position to the target parking position P1; and information indicating that after reversing to the third reversing stop position TP3, manual operation is required to move forward. Information M1 can also be a still image, animated image, or icon representing at least one of these information, and is not limited to text.

[0238] Additionally, the display screen 30B may also include surrounding images V captured by the camera 16. Figure 9A The display screen 30B shows a method in which images V1 and V2 are used as peripheral images V.

[0239] Image V1 is a peripheral image V of vehicle 1B in its direction of travel. Image V2 is a top-down view of vehicle 1B. A top-down view is an image viewed from above, as if looking down at vehicle 1B from above.

[0240] That is, the display device 22 may also include multiple display areas (a first display area, a second display area, etc.). For example, the control unit 11B may display image V1 in one display area and image V2 in another display area. In addition, the vehicle 1B may also be equipped with multiple display devices 22. Moreover, one of the multiple display devices 22 may be used as the first display area, and another display device may be used as the second display area.

[0241] exist Figure 9A As an example, the following situation is shown: Image V1 and Image V2 are displayed in each of the two display areas respectively, and information M1 is superimposed on these images V1 and Image V2.

[0242] Additionally, the control unit 11B can overlay the third reversing stop position TP3 onto the peripheral image V. Figure 9B The image shown is display screen 30B, which overlays the third reversal stop position TP3 onto the image V2.

[0243] For example, the third reversing stop position TP3 displayed by the display device 22 is represented by a frame FT3. Frame FT3 represents the third reversing stop position TP3 and the position and orientation of vehicle 1B at the third reversing stop position TP3. The shape of frame FT3 is, for example, a rectangle that surrounds vehicle 1B when visually confirmed from above.

[0244] Control unit 11B generates a virtual frame VFP3 (refer to) that surrounds vehicle 1B when it reaches the third reversing stop position TP3. Figure 8B The control unit 11B then configures the frame FT3 at the position corresponding to the third reversing stop position TP3 in the peripheral image V displayed on the display screen 30B, to indicate the tilt angle of the vehicle 1B when it is estimated to have reached that position (see reference). Figure 9A ).

[0245] Display screen 30B can also be a screen with an icon representing vehicle 1B positioned at the current location of vehicle 1B. Figure 9A The following states are shown: an icon image representing vehicle 1B is configured at the current position of vehicle 1B in image V2, and frame FT3 is overlaid on image V2 (surrounding image V).

[0246] By displaying a display screen 30B on which the frame FT3 is superimposed on the surrounding image V, the control unit 11B can provide the passenger with the frame FT3 indicating the third reversing stop position TP3 in a manner that can be easily confirmed.

[0247] In addition, by displaying a display screen 30B on which information M1 is superimposed on the surrounding image V, the control unit 11B can provide the passenger with information indicating that the vehicle is reversing towards the third reversing stop position TP3 by autonomous driving, information indicating that the vehicle cannot park directly from the current position to the target parking position P1, and information indicating that the vehicle is reversing towards the third reversing stop position TP3 contained in information M1 by autonomous driving.

[0248] Then, the control unit 11B causes the display device 22 to display information prompting the reduction of braking operation performed on the braking operation unit included in the operation device 20.

[0249] Figure 9B This is a schematic diagram of an example of display screen 30C. Display screen 30C is an example of display screen 30 displayed on display device 22. In addition, display screen 30C is an example of a third display screen.

[0250] If vehicle 1B has stopped at the second reversing stop position TP2 and cannot perform a one-way parking maneuver by reversing only towards the target parking position P1, the control unit 11B generates a reversing path R2 up to the third reversing stop position TP3 (see reference). Figure 8B ), in the display Figure 9A After displaying screen 30B as shown, the display device 22 displays... Figure 9B The display shown is 30C.

[0251] Display screen 30C contains information M2. Information M2 is information that prompts the reduction of braking operation on the brake pedal (brake operating unit).

[0252] exist Figure 9B In the example shown, information M2 is text that prompts the user to reduce braking force on the brake pedal. Information M2 can also be a still image, animated image, or icon that prompts the user to reduce braking force on the brake pedal, and is not limited to text.

[0253] For example, the control unit 11B causes the display device 22 to display a display screen 30C that overlays information M2 onto the image V1. In addition, similar to the display screen 30B, the control unit 11B may also overlay a frame FT3 indicating the third reversing stop position TP3 and an icon image indicating vehicle 1B onto the image V2.

[0254] By displaying a display screen 30C on which the frame FT3 is superimposed on the surrounding image V, the control unit 11B can provide the passenger with the frame FT3 indicating the third reversing stop position TP3 in a manner that can be easily confirmed.

[0255] In addition, the control unit 11B can provide the passenger with information to prompt them to reduce braking operation by displaying a display screen 30C on the display device 22 that overlays the information M2 onto the surrounding image V.

[0256] Return to Figure 8B Let's continue with the explanation.

[0257] Then, the control unit 11B controls the steering based on the reversing path R2 to make the vehicle 1B reverse, and drives the vehicle 1B to the third reversing stop position TP3 by autonomous driving.

[0258] For example, the control unit 11B compares the reversing path R2 used to move vehicle 1B from the second reversing stop position TP2 to the third reversing stop position TP3 with the current position of vehicle 1B in the actual space RS, and at least controls the steering to make the current position of vehicle 1B the position on the reversing path R2, and controls the movement control device 12 to drive along the reversing path R2. In autonomous driving, vehicle 1B is automatically controlled by the vehicle control device 10 without the need for manual driving operations by the passenger.

[0259] Then, when vehicle 1B reaches the third reversing stop position TP3, control unit 11B causes display device 22 to display information prompting the brake pedal to be depressed.

[0260] Figure 9C This is a schematic diagram of an example of display screen 30D. Display screen 30D is an example of display screen 30.

[0261] Display screen 30D contains information M3. Information M3 is information that prompts the brake pedal (brake operating unit) to be depressed.

[0262] exist Figure 9C In the example shown, information M3 is text that prompts the user to press the brake pedal. Information M3 can also be a still image, animated image, or icon that prompts the user to press the brake pedal, and is not limited to text.

[0263] For example, the control unit 11B causes the display device 22 to display a display screen 30D that overlays information M2 onto the image V1. In addition, similar to the display screen 30B, the control unit 11B may also overlay a frame FT3 indicating the third reversing stop position TP3 and an icon image indicating vehicle 1B onto the image V2.

[0264] By displaying a display screen 30D on which the frame FT3 is superimposed on the surrounding image V, the control unit 11B can provide the passenger with the frame FT3 indicating the third reversing stop position TP3 in a manner that can be easily confirmed.

[0265] In addition, the control unit 11B can provide the passenger with information to prompt them to press the brake pedal by displaying a display screen 30D on the display device 22 that overlays the information M3 onto the surrounding image V.

[0266] Return to Figure 8B Let's continue with the explanation.

[0267] Then, when vehicle 1B reaches the third reversing stop position TP3, control unit 11B causes display device 22 to display a second display screen containing information prompting vehicle 1B to move forward according to the driver's instructions.

[0268] Figure 9D This is a schematic diagram of an example of display screen 30E. Display screen 30E is an example of a second display screen and display screen 30.

[0269] Display screen 30E contains information M4. Information M4 is information that prompts vehicle 1B to move forward according to the driver's direction. In addition, information M4 may also contain information indicating a prescribed steering operation. This prescribed steering operation includes steering at the third reversing stop position TP3 to reverse after moving forward, enabling the vehicle to park in the target parking space S1, as well as shifting gears to reverse for reversing after moving forward.

[0270] exist Figure 9D As an example, the information M4 is shown as text prompting forward movement and the prescribed steering operation. Information M4 can also be a still image, animated image, or icon, etc., prompting forward movement and the prescribed steering operation, and is not limited to text.

[0271] For example, the control unit 11B causes the display device 22 to display a display screen 30E that overlays information M4 onto the image V1.

[0272] In addition, the control unit 11B can further display, overlaid on the surrounding image V, a reference position AP indicating the forward position (also refer to...). Figure 8B The control unit 11B can also overlay the first frame F1, which represents the target parking position P1, onto the image V2.

[0273] The forward reference position AP is the position on the actual space RS that serves as the reference point for forward movement when vehicle 1B has reached the third reversing stop position TP3 and is moving forward according to the driver's direction. Preferably, the forward reference position AP is the position after vehicle 1B has performed a reversing maneuver at the third reversing stop position TP3 and has been moving forward from the third reversing stop position TP3, and is determined by a camera 16D (see reference) located at the rear of vehicle 1B and capturing images of the rear of vehicle 1B. Figure 2 The obtained photographic image will capture the position of the opening end point FP of the target parking space S1. The control unit 11B can determine the forward reference position AP that meets the condition based on the surrounding image V, the third reversing stop position TP3, and the target parking position P1 using a known method.

[0274] Then, the control unit 11B can overlay the frame FT4, which represents the determined forward reference position AP, onto the corresponding position in the image V2.

[0275] The control unit 11B can provide passengers with information to prompt the vehicle 1B, which has reached the third reversing stop position TP3, to move forward and to perform steering operations on the vehicle 1B by displaying a display screen 30E on the display device 22 that overlays the information M4 onto the surrounding image V.

[0276] In addition, by overlaying the frame FT4 representing the forward reference position AP and the icon image representing the vehicle 1B onto the surrounding image V, the control unit 11B can provide the occupants with the positional relationship between the vehicle 1B and the forward reference position AP in a verifiable manner.

[0277] Return to Figure 8B Let's continue with the explanation.

[0278] After the vehicle 1B moves forward according to the driver's direction (see arrow R3), the control unit 11B at least controls the steering to reverse the vehicle 1B and autonomously drives the vehicle 1B to park at the target parking position P1.

[0279] In detail, when the vehicle 1B moves forward according to the driver's direction and reaches the forward reference position AP, the control unit 11B causes the display device 22 to display a fourth display screen.

[0280] Figure 9E This is a schematic diagram of an example of display screen 30F. Display screen 30F is an example of the fourth display screen and display screen 30.

[0281] Display screen 30F contains information M5. Information M5 is information that prompts the operating device 20 to perform a reverse instruction operation. The reverse instruction operation is, for example, information that prompts the shifting of gear to reverse gear by operating the gear lever.

[0282] exist Figure 9E As an example, the text M5 is shown as prompting a back gesture. The text M5 can be a still image, animated image, or icon that prompts a back gesture, and is not limited to text.

[0283] For example, the control unit 11B causes the display device 22 to display a display screen 30F that overlays information M5 onto the image V1.

[0284] In addition, the control unit 11B can further overlay on the surrounding image V at least one of the following: a frame FT4 indicating the forward reference position AP, an icon image indicating the vehicle 1B, and a first frame F1 indicating the target parking position P1.

[0285] The control unit 11B can provide the passenger with information to perform an operation to reverse the vehicle 1B that has reached the forward reference position AP by displaying a display screen 30F on the display device 22 that overlays the information M5 onto the surrounding image V.

[0286] In addition, the fourth display screen may also include information prompting the operator to perform a reverse instruction operation on the operating device 20 and information prompting the operator to reduce the braking operation on the operating device 20.

[0287] Figure 9F This is a schematic diagram of an example of display screen 30G. Display screen 30G is an example of the fourth display screen and display screen 30.

[0288] The display screen 30G contains information M6. Information M6 is information that prompts the reduction of braking operation performed on the operating device 20.

[0289] exist Figure 9F As an example, the information M6 is shown as text prompting the reduction of braking operation on the operating device 20. The information M6 can also be a still image, animated image, or icon prompting the reduction of braking operation, and is not limited to text.

[0290] For example, the control unit 11B causes the display device 22 to display a display screen 30G that overlays information M6 onto the image V1.

[0291] Additionally, vehicle 1B can also cause display device 22 to display display screen 30G showing information M5 and information M6 on peripheral image V. Furthermore, control unit 11B can further overlay on peripheral image V at least one of the following: frame FT4 indicating forward reference position AP, icon image indicating vehicle 1B, and first frame F1 indicating target parking position P1.

[0292] The control unit 11B can provide information to the occupants of the vehicle 1B that has reached the forward reference position AP by displaying a display screen 30G on the display device 22 that overlays the information M6 onto the surrounding image V, prompting them to perform a reverse operation.

[0293] Return to Figure 8B Let's continue with the explanation.

[0294] When the control unit 11B receives an instruction to shift gears to reverse via the operation of the gear lever based on the operation instruction of the passenger to the operating device 20, and receives a braking operation based on the operation of the brake pedal (brake operating unit), it at least controls the steering to reverse the vehicle 1B, and drives the vehicle 1B to park at the target parking position P1 by autonomous driving.

[0295] For example, the control unit 11B generates a reverse path R4 from the forward reference position AP to the target parking position P1. The control unit 11B generates the reverse path R4, which moves backward from the forward reference position AP to the target parking position P1 in a single trip, based on the surrounding image V and using known methods such as a learning model. The reverse path R4 is generated after the vehicle 1B reaches the forward reference position AP and before it begins to reverse towards the target parking position P1 through autonomous driving.

[0296] Then, the control unit 11B compares the current position of vehicle 1B in the actual space RS with the reversing path R4, at least controlling the steering to make the current position of vehicle 1B on the reversing path R4, and controlling the movement control device 12 to drive along the reversing path R4, thereby parking at the target parking position P1. During autonomous driving, vehicle 1B automatically controls its movement through the vehicle control device 10 without relying on manual driving operations by the passenger.

[0297] When vehicle 1B is parked at the target parking position P1, the control unit 11B displays the status of vehicle 1B being parked within the first frame F1 of the target parking position P1 on the display screen 30 displayed on the display device 22.

[0298] Figure 9G This is a schematic diagram of an example of display screen 30H. Display screen 30H is an example of display screen 30.

[0299] When vehicle 1B is parked at the target parking position P1, the control unit 11B causes the display device 22 to display a display screen 30H that shows the icon image of vehicle 1B in the first frame F1 of the target parking position P1 on the peripheral image V and the first frame F1 superimposed on the peripheral image V.

[0300] Next, an example of the information processing flow performed by the control unit 11B in this embodiment will be described.

[0301] Figure 10 This is a flowchart illustrating an example of the information processing flow performed by the control unit 11B.

[0302] When vehicle 1B starts moving forward according to the driver's driving (step S200), control unit 11B detects the target parking position P1 of vehicle 1B based on the surrounding image V acquired by camera 16 (step S202).

[0303] Next, the control unit 11B determines whether the first predetermined condition is met (step S204). As described above, the first predetermined condition indicates that the vehicle speed of 1B is below a predetermined speed, and it has detected entering the parking area based on map data 18B or has received instruction input from the passenger. When a negative determination is made in step S204 (step S204: "No"), the process returns to step S202. When a positive determination is made in step S204 (step S204: "Yes"), the process proceeds to step S206.

[0304] In step S206, the control unit 11B outputs the turning reference position P2 corresponding to the target parking position P1 detected in step S202, and the orientation of the vehicle 1B at the turning reference position P2 (step S206).

[0305] Then, the control unit 11B causes the display device 22 to display a first display screen (display screen 30A) that superimposes the surrounding image V displayed on the display device 22 with a first frame F1 representing the target parking position P1 detected in step S202, and a second frame F2 representing the U-turn reference position P2 derived in step S206 and the orientation of the vehicle 1B at the U-turn reference position P2. (step S208) Figure 3 The display shown is screen 30A.

[0306] Then, after the vehicle 1B moves forward according to the driver's direction, the control unit 11B determines whether it has stopped at the reversing stop position TP (step S212). If a negative judgment is made in step S212 (step S212: "No"), the process returns to step S202. Alternatively, the control unit 11B may return to step S202 if a negative judgment is made after the judgment process in step S212 has been repeated for a predetermined time. If a positive judgment is made in step S212 (step S212: "Yes"), the process proceeds to step S214.

[0307] In step S214, the operating device 20 of the control unit 11B accepts a prescribed operation (step S214). This prescribed operation is an operation indicating the start of rear quick parking. For example, the control unit 11B accepts the prescribed operation by detecting a situation where the gear is shifted to reverse by the passenger's operation of the gear lever included in the operating device 20.

[0308] Next, the control unit 11B determines whether it is possible to perform a one-way parking maneuver by reversing from the current stopping position TP of the vehicle 1B to the target parking position P1 (step S216).

[0309] If the control unit 11B determines that parking can be achieved in a single stroke by simply reversing (step S216: "Yes"), it proceeds to step S218. In step S218, the control unit 11B generates a reversing path R1 for the vehicle 1B to move autonomously from its current stopping position, i.e., the first reversing stopping position TP1, to the target parking position P1 in a single stroke and park by simply reversing (step S218). Then, the control unit 11B compares the current position of the vehicle 1B in the actual space RS with the reversing path R1, controls the steering to make the current position of the vehicle 1B a position on the reversing path R1, and controls the movement control device 12 to travel along the reversing path R1 (step S220). Through the processing in step S220, the vehicle 1B is autonomously reversed and parked in a single stroke towards the target parking position P1. Then, the routine ends.

[0310] On the other hand, if the control unit 11B determines that it is impossible to park in a single stroke by simply reversing (step S216: "No"), it proceeds to step S222.

[0311] In step S222, the control unit 11B derives the third reversing stop position TP3 and generates a reversing path R2 from the current position of vehicle 1B, i.e., the second reversing stop position TP2, to the third reversing stop position TP3 (step S222).

[0312] Then, the control unit 11B causes the display device 22 to display a screen that overlays information M1, which includes at least one of the following: information indicating that the vehicle is reversing towards the third reversing stop position TP3 by autonomous driving, and information indicating that it is impossible to park directly from the current position to the target parking position P1, onto the surrounding image V (step S223). Through the processing in step S223, for example, the display device 22 displays... Figure 9A The display shown is screen 30B.

[0313] Next, the control unit 11B causes the display device 22 to display a display screen 30C, i.e., a third display screen, superimposed on the peripheral image V, which is information M2 that will cause the braking operation on the brake pedal (brake operation unit) to be reduced (step S224). Through the processing in step S224, for example, the display is shown on the display device 22. Figure 9B The display shown is 30C.

[0314] Then, when it is detected that the braking operation on the brake pedal is reduced according to the passenger's operation, the control unit 11B controls the steering based on the reverse path R2 generated in step S222 to reverse the vehicle 1B, and reverses the vehicle 1B to the third reversing stop position TP3 by autonomous driving (step S226).

[0315] The control unit 11B repeatedly performs the negative judgment (step S228: "No") until it determines that the vehicle 1B has reached the third reversing stop position TP3. When the control unit 11B determines that the third reversing stop position TP3 has been reached (step S228: "Yes"), it proceeds to step S230.

[0316] In step S230, the control unit 11B causes the display device 22 to display a display screen 30 superimposed on the surrounding image V, which shows information M3 that will prompt the brake pedal to be depressed (step S230). Through the processing in step S230, for example, the information is displayed on the display device 22... Figure 9C The displayed screen is 30D.

[0317] Then, the control unit 11B causes the display device 22 to display a second display screen containing information M4 that prompts the vehicle 1B to move forward according to the driver's direction (step S232). Through the processing in step S232, for example, the display device 22 shows... Figure 9D The shown screen 30E is used as the second display screen.

[0318] Vehicle 1B moves forward according to the driver's direction (refer to arrow R3) (step S234). Control unit 11B determines whether vehicle 1B has moved forward to the forward reference position AP (step S236). Control unit 11B repeatedly performs negative judgments (step S236: "No") until a positive judgment is made in step S236 (step S236: "Yes"). When a positive judgment is made in step S236 (step S236: "Yes"), control unit 11B proceeds to step S238.

[0319] In step S238, the control unit 11B causes the display device 22 to display a fourth display screen containing information M5 that prompts the operation device 20 to perform a back gesture (step S238). Through the processing in step S238, for example, the display device 22 displays... Figure 9EThe screen shown is 30F, which is used as the fourth display screen.

[0320] Furthermore, as described above, in step S238, the control unit 11B may also cause the display device 22 to display a fourth display screen containing at least one of information M5 and information M6 that prompts the reduction of braking operation on the operating device 20. For example, the control unit 11B may also display a fourth display screen containing at least one of information M5 and information M6 that prompts the reduction of braking operation on the operating device 20. Figure 9E The shown display screen 30F is displayed after the display device 22 as the fourth display screen, and the display device 22 also displays the fourth display screen. Figure 9F The display screen 30G shown is shown. In addition, the control unit 11B can also display the display screen 30, which contains both information M5 and information M6, as a fourth display screen on the display device 22.

[0321] Next, the control unit 11B generates a reverse path R4 toward the target parking position P1 (step S240). Then, the control unit 11B repeatedly performs a negative judgment (step S242: "No") until it is determined that the reverse instruction operation for the operating device 20 has been accepted and the braking operation has been reduced. When a positive judgment is made (step S242: "Yes"), the process proceeds to step S244.

[0322] In step S244, the control unit 11B compares the current position of vehicle 1B in the actual space RS with the reversing path R4, controls the steering to make the current position of vehicle 1B the position on the reversing path R4, and controls the movement control device 12 to drive along the reversing path R4, so that vehicle 1B parks at the target parking position P1 (step S244). Then, the routine ends.

[0323] As described above, the vehicle control device 10B of this embodiment is mounted on a vehicle 1B, which includes an operating device 20 for receiving passenger operations, a camera 16 for acquiring surrounding images V, a display device 22 for passenger visual confirmation, and a motion control device 12 for controlling at least the steering wheel. While the vehicle 1B is moving forward according to the operations received by the operating device 20, the vehicle control device 10B causes the display device 22 to display a first display screen (display screen 30A) that overlays the target parking position P1 detected based on the surrounding images V acquired by the camera 16 and the corresponding U-turn reference position P2 onto the surrounding images V. When the vehicle 1B has stopped at the first reversing stop position TP1 and is capable of one-way parking by reversing only towards the target parking position P1, the vehicle control device 10B controls at least the steering wheel to reverse the vehicle 1B and autonomously drives the vehicle 1B to park towards the target parking position P1. If vehicle 1B has stopped at the second reversing stop position TP2 and cannot park at the target parking position P1 by reversing only, vehicle control device 10B at least controls the steering wheel to reverse vehicle 1B, and autonomously reverses vehicle 1B to a third reversing stop position TP3, which is different from the target parking position P1. When vehicle 1B reaches the third reversing stop position TP3, vehicle control device 10B causes display device 22 to display a second display screen (display screen 30B) containing information M1 that prompts vehicle 1B to move forward according to the passenger's driving. After vehicle 1B moves forward according to the passenger's driving, vehicle control device 10B at least controls the steering wheel to reverse vehicle 1B, and autonomously reverses vehicle 1B to park at the target parking position P1.

[0324] In the prior art, sometimes the vehicle stops at a reversing stop position TP, where it is difficult to autonomously move towards the target parking position P1, depending on the driver's actions; and sometimes it is difficult to autonomously move from the reversing stop position TP to the target parking position P1. In other words, in the prior art, there are situations where it is difficult to provide appropriate parking assistance.

[0325] On the other hand, if vehicle 1B has stopped at the second reversing stop position TP2 and cannot park towards the target parking position P1 in a single trip by simply reversing, the vehicle control device 10B of this embodiment at least controls the steering wheel to reverse vehicle 1B, and then autonomously reverses vehicle 1B towards the third reversing stop position TP3, which is different from the target parking position P1. When vehicle 1B reaches the third reversing stop position TP3, the vehicle control device 10B causes the display device 22 to display a second display screen (display screen 30B) containing information M1 that prompts vehicle 1B to move forward according to the passenger's driving. After vehicle 1B moves forward according to the passenger's driving, the control unit 11B at least controls the steering wheel to reverse vehicle 1B, and then autonomously reverses vehicle 1B towards the target parking position P1.

[0326] Therefore, the vehicle control device 10B of this embodiment can provide appropriate parking assistance.

[0327] (Third Implementation)

[0328] In this embodiment, the following method will be described: During the forward movement of vehicle 1, the target parking position P1 corresponding to the driving location of vehicle 1 and the U-turn reference position P2 corresponding to the target parking position P1 are displayed in sequence.

[0329] Figure 1 This is a block diagram illustrating an example of the overall structure of the vehicle 1C according to this embodiment.

[0330] Vehicle 1C is identical to vehicle 1 in the above-described embodiment, except that it has vehicle control device 10C instead of vehicle control device 10. Therefore, the configuration of sensor 14 and camera 16 in vehicle 1C is also the same as in vehicle 1 (see [reference]). Figure 2 Furthermore, the exterior of vehicle 1C and the structure near the driver's seat 24A are the same as those of vehicle 1 (see reference). Figure 3 , Figure 4 ).

[0331] The vehicle control unit 10C is an electronic control unit that uniformly controls all parts of the vehicle 1C. Similar to the vehicle control unit 10, the vehicle control unit 10C uses sensor information and surrounding images received from the sensor 14 and camera 16 respectively to control the motion control unit 12 to optimize the driving state of the vehicle 1C. Furthermore, the vehicle control unit 10C controls the motion control unit 12 to enable the vehicle 1C to drive autonomously.

[0332] The vehicle control device 10C includes a control unit 11C. Part or all of the control unit 11C may be a software structure implemented through the cooperation of a processor and various programs stored in a memory. Alternatively, part or all of the control unit 11C may be a hardware structure implemented by dedicated circuitry or the like.

[0333] The control unit 11C provides unified control over all parts of vehicle 1C.

[0334] In this embodiment, the control unit 11C performs information processing for parking assistance of the vehicle 1C. In this embodiment, similar to the embodiments described above, the following manner is envisioned and described: after the vehicle 1C is driven forward by the passenger until at least a portion of the vehicle body has passed at least a portion of the target parking position P1, the vehicle 1C is stopped by the passenger, the vehicle 1C reverses from the reversing stop position TP (which is the stopping position), and the vehicle 1C moves towards the target parking position P1 by autonomous driving.

[0335] Figure 11A This is an explanatory diagram of an example of parking assistance in this embodiment.

[0336] During the process of the vehicle 1C moving forward according to the operation of the passenger received by the operating device 20, the control unit 11C (refer to...) Figure 11A (In the direction of arrow D in the image), based on the surrounding image V acquired by camera 16, one or more target parking position candidates SP for which vehicle 1C can park are detected.

[0337] Candidate target parking location SP refers to the candidate target parking location P1 at each travel location DP, which is estimated to be where vehicle 1C can park, as it is moving from a time-varying travel location DP according to the occupant's operation, i.e., the occupant's driving. The current travel location DP of vehicle 1C is represented by the current position of vehicle 1C.

[0338] The control unit 11C analyzes the surrounding images V acquired by the camera 16 at each driving location DP using known methods, thereby detecting candidate parking positions SP. Figure 11A As an example, the control unit 11C is shown to detect multiple target parking position candidates SP (target parking position candidate SP1a to target parking position candidate SP2c) at a certain driving location DP.

[0339] The control unit 11C displays the detected multiple target parking position candidates SP as target parking position candidates on the display device 22.

[0340] Figure 11BThis is a schematic diagram of an example of display screen 30I. Display screen 30I is an example of display screen 30 displayed on display device 22.

[0341] The control unit 11C causes the display device 22 to display a display screen 30I that overlays the target parking position candidate objects SP0B, representing each detected target parking position candidate SP, onto the surrounding image V.

[0342] The target parking location candidate object SP0B is a display representing the target parking location candidate SP. The target parking location candidate object SP0B can be, for example, a still image, character, animated image, or icon representing the parking space of the target parking location candidate SP. Figure 11B As an example, the following scenario is shown: The control unit 11C generates an image of a virtual frame representing the parking space surrounding the target parking position candidate SP in the actual space RS as the target parking position candidate object SP0B, and places it in the surrounding image V at the position corresponding to the target parking position candidate SP.

[0343] The color of the target parking position candidate SP0B displayed on the display device 22 is preferably a color that does not correspond to any of the first, second, third, and fourth colors described later.

[0344] Non-corresponding colors are those that a person can perceive as different colors or unrelated colors. Corresponding colors are those that a person can perceive as the same color or related colors.

[0345] The peripheral image V displayed on the display screen 30 can be either image V1, which is a peripheral image V in the direction of travel of the vehicle 1C, or image V2, which is a top-down view of the vehicle 1C. Furthermore, similar to the above embodiment, images V1 and V2 can also be displayed in different display areas on the display screen 30.

[0346] Then, the control unit 11C detects one of the multiple target parking position candidates SP that is suitable for the current driving location DP of the vehicle 1C as the target parking position P1 corresponding to the driving location DP.

[0347] Figure 12A This is an explanatory diagram of an example of parking assistance in this embodiment.

[0348] like Figure 12A As shown, the control unit 11C detects one of the multiple target parking position candidates SP as the target parking position P1 for the current driving location DP of the vehicle 1C.

[0349] Based on the positional relationship between the current position of the vehicle 1C, i.e. the current driving location DP, and the target parking position candidate SP, the control unit 11C detects a target parking position candidate SP as the target parking position P1.

[0350] For example, the control unit 11C excludes target parking position candidate SPs that vehicle 1C has already passed from among multiple target parking position candidate SPs. The control unit 11C determines that vehicle 1C has passed a target parking position candidate SP if the driver's line of sight (e.g., the center of the wheelbase) exceeds the downstream end of vehicle 1C's forward travel direction in the target parking position candidate SP. Furthermore, the control unit 11C excludes target parking position candidate SPs located at a distance exceeding a predetermined distance from the vehicle body 2 in a direction intersecting with vehicle 1C's forward travel direction (arrow D direction). This predetermined distance can be predetermined. This predetermined distance is, for example, 2.5m, but is not limited to this distance.

[0351] If one target parking position candidate SP remains unremoved by these elimination processes, the control unit 11C detects the remaining target parking position candidate SP as the target parking position P1. On the other hand, if multiple target parking position candidate SPs remain unremoved by these elimination processes, the control unit 11C detects the target parking position candidate SP that is closest to the vehicle 1C among the remaining multiple target parking position candidate SPs as the target parking position P1.

[0352] exist Figure 12A As an example, this section illustrates a scenario where the candidate parking location SPa is detected as the target parking location P1. Additionally, it shows... Figure 12A The driving location DP of vehicle 1C will be referred to as the first driving location DP1. Furthermore, the target parking position P1 detected at the first driving location DP1 will be referred to as the first target parking position P11.

[0353] When the control unit 11C detects a first target parking position P11 based on the surrounding image V at the first driving location DP1, it detects a U-turn reference position P2, i.e., a first U-turn reference position P21, corresponding to the detected first target parking position P11. Furthermore, the control unit 11C can also further derive the orientation of the vehicle 1C at the first U-turn reference position P21. The method for deriving the first U-turn reference position P21 and the orientation of the vehicle 1C at the first U-turn reference position P21 is the same as the method for deriving the reversing start position P2 and the orientation of the vehicle 1C at that reversing start position P2 described in the above embodiment.

[0354] Then, the control unit 11C displays the detected first target parking position P11 and the first U-turn reference position P21 corresponding to the first target parking position P11 on the display device 22 in a manner superimposed on the surrounding image V. That is, while the vehicle 1C is moving forward according to the operation received by the operation device 20, the control unit 11C displays the first target parking position P11 detected at the first driving location DP1 based on the surrounding image (first surrounding image) V acquired by the camera 16, and the first U-turn reference position P21 corresponding to the first target parking position P11 on the display device 22 in a manner superimposed on the surrounding image V.

[0355] For example, the control unit 11C uses a first frame to indicate the first target parking position P11 that the display device 22 will display. In addition, the control unit 11B displays the first target parking position P11, which is the first target parking position, and the first U-turn reference position P21, which is the first U-turn reference position, on the display device 22 in a manner that is superimposed on the surrounding image V (first surrounding image).

[0356] Figure 12B This is a schematic diagram of an example of display screen 30J. Display screen 30J is an example of display screen 30 displayed on display device 22.

[0357] The control unit 11C causes the display device 22 to display a display screen 30J that overlays at least one of the first frame F11 representing the first target parking position P11 detected at the first driving location DP1 and the first target parking position object P11OB, and the first U-turn reference position object P21OB of the first U-turn reference position P21 detected at the first driving location DP1 onto the surrounding image V.

[0358] The first frame F11 is a frame representing the first target parking position P11. The control unit 11C generates the first frame F11 of the virtual frame VF1 representing the target parking space S1 that surrounds the first target parking position P11 in the actual space RS, and superimposes the first frame F11 onto the position corresponding to the first target parking position P11 in the peripheral image V displayed by the display device 22.

[0359] The first target parking location object P11OB represents the display of the first target parking location P11. The first target parking location object P11OB can be, for example, a still image, character, animated image, or icon representing the target parking space S1 of the first target parking location P11. Figure 12BAs an example, the following method is shown: the first target parking position object P11OB is a frame surrounding the target parking space S1 of the first target parking position P11, and a mark indicating the parking lot (a mark with the alphanumeric "P" surrounded by a circle). The control unit 11C overlays the first target parking position object P11OB, representing the first target parking position P11 in the actual space RS, onto the position corresponding to the first target parking position P11 in the peripheral image V displayed by the display device 22.

[0360] The first U-turn reference position object P21OB represents the display of the first U-turn reference position P21. For example, the first U-turn reference position object P21OB can be used to represent the virtual frame VF2 (see reference). Figure 12A The image frame VF2 represents the outer frame of vehicle 1C when it is assumed to have stopped at the first U-turn reference position P21. The first U-turn reference position object P21OB simply represents the display of the first U-turn reference position P21, and can be a still image such as a frame, characters, animated images, or icons.

[0361] exist Figure 12B As an example, the following method is shown: the first U-turn reference position object P21OB is a frame representing the first U-turn reference position P21. The control unit 11C causes the first U-turn reference position object P21OB, representing the first U-turn reference position P21 in the actual space RS, to be superimposed on the peripheral image V displayed by the display device 22 at the position corresponding to the first U-turn reference position P21.

[0362] The first color of the first target parking position object P11OB and the first frame F11 corresponds to the second color of the first U-turn reference position object P21OB. That is, the control unit 11C displays the pairing of the target parking position P1 and the U-turn reference position P2 at the first driving location DP1 using the corresponding color, that is, the pairing of the first target parking position P11 and the first U-turn reference position P21.

[0363] As described above, the corresponding color refers to a color that a person can perceive as the same color or a related color. Therefore, by displaying the pairing of the first target parking position P11 and the first U-turn reference position P21 detected at the first driving location DP1 with the corresponding color on the display device 22, the control unit 11C can provide the occupants of the vehicle 1C with the information that the first target parking position P11 and the first U-turn reference position P21 are in a corresponding relationship at the first driving location DP1 in an easily recognizable manner.

[0364] Furthermore, the peripheral image V displayed on the display screen 30J can be either image V1, which is a peripheral image V in the direction of travel of vehicle 1C, or image V2, which is a top-down view of vehicle 1C. Also, similar to the above embodiment, images V1 and V2 can be displayed in different display areas on the display screen 30J. Additionally, the control unit 11C can also overlay an icon image representing vehicle 1C onto the peripheral image V.

[0365] Figure 13 This is an explanatory diagram of the parking assistance in this embodiment.

[0366] When the control unit 11C displays the first target parking position P11 detected at the first driving location DP1 and the first U-turn reference position P21 corresponding to the first target parking position P11 on the display device 22 in a manner superimposed on the surrounding image V during the forward movement of the vehicle 1C, it determines whether the operation device 20 has accepted the operation of selecting the first target parking position P11.

[0367] The operation of the operating device 20 to select the first target parking position P11 is a steering operation performed by the passenger on the steering operation parts including the steering wheel (driving wheel) of the operating device 20 in the direction toward the first turning reference position P21, when the first target parking position P11 and the first turning reference position P21 are displayed on the display device 22.

[0368] For example, if the control unit 11C adjusts the steering angle of the steering device by the passenger's operation of the steering operation unit, and the steering angle detected by the steering angle sensor that detects the steering angle of the steering device is a specific angle (e.g., 30 degrees or more) deviating towards the first U-turn reference position P21, it can determine that the operation of selecting the first target parking position P11 has been accepted.

[0369] When vehicle 1C moves forward after the operation of selecting the first target parking position P11 has been accepted, for example, as Figure 13 As shown, the vehicle 1C is tilted in the direction of forward travel (refer to arrow D) toward the first U-turn reference position P21 (arrow D').

[0370] When the operating device 20 accepts the operation of selecting the first target parking position P11, the control unit 11C at least fixes the first U-turn reference position P21 on the display device 22.

[0371] The first U-turn reference position P21 is fixedly displayed on the display device 22, which means that even if the vehicle 1C moves forward and reaches another driving location DP downstream of the first driving location DP1 in the direction of forward travel, the first U-turn reference position P21 is continuously superimposed on the surrounding image V.

[0372] In addition, when the operating device 20 accepts the operation of selecting the first target parking position P11, the control unit 11C can also fix the first U-turn reference position P21 and the first target parking position P11 corresponding to the first U-turn reference position P21 on the display device 22.

[0373] Furthermore, preferably, when the operating device 20 accepts the operation of selecting the first target parking position P11, even if the vehicle 1C travels forward and arrives at another driving position DP, i.e., the second driving position DP2, which is downstream of the first driving position DP1 in the direction of travel, the control unit 11C does not cause the display device 22 to display at least one of the target parking position P1 detected at the second driving position DP2, i.e., the second target parking position, and the U-turn reference position P2 corresponding to the second target parking position, i.e., the second U-turn reference position.

[0374] Therefore, when the operating device 20 accepts the operation of selecting the target parking position P1, even if the vehicle 1C travels forward and reaches another driving position DP downstream of the first driving position DP1, the control unit 11C will, for example, Figure 12B The displayed screen 30J is continuously displayed on the display device 22.

[0375] Return to Figure 13 Let's continue with the explanation.

[0376] When the operating device 20 accepts the operation of selecting the first target parking position P11 and the vehicle 1C has stopped at the first reversing stop position TP11 corresponding to the first target parking position P11, the control unit 11C at least controls the steering to make the vehicle 1C reverse, and drives the vehicle 1C to park from the first reversing stop position TP11 to the first target parking position P11 through autonomous driving.

[0377] In detail, when the operating device 20 accepts the operation of selecting the first target parking position P11, the vehicle 1C moves forward according to the driver's direction. That is, the movement from at least the first driving position DP1 to the stop at the reversing stop position TP is carried out through the operation accepted by the operating device 20.

[0378] Then, after the vehicle 1C has moved forward according to the driver's direction, the control unit 11C determines whether it has stopped at the reversing stop position TP. The vehicle 1C stops at the reversing stop position TP by an operation received by the operating device 20. That is, the vehicle 1C stops at the reversing stop position TP according to the braking operation performed by the driver.

[0379] As described in the above embodiments, the reversing stop position TP refers to the position of vehicle 1C in the actual space RS when it stops after the control unit 11C has derived the target parking position P1 and the U-turn reference position P2. In this embodiment, the reversing stop position TP is the position of vehicle 1C in the actual space RS when the control unit 11C detects the target parking position P1 and the U-turn reference position P2, accepts the operation of selecting the target parking position P1, and the vehicle 1C moves forward and stops according to the driver's driving. Alternatively, similar to the above embodiments, the reversing stop position TP can also be the position of vehicle 1C in the actual space RS when the control unit 11C detects the target parking position P1 and the U-turn reference position P2, accepts the operation of selecting the target parking position P1, and the vehicle 1C moves forward and stops according to the driver's driving, and the gear of vehicle 1C is shifted to reverse (R).

[0380] In this embodiment, the position where the vehicle 1C moves forward and stops according to the driver's direction after accepting the operation of selecting the first target parking position P11, i.e., the reversing stop position TP, will be referred to as the first reversing stop position TP11. The first reversing stop position TP11 is an example of the reversing stop position TP. The first reversing stop position TP11 may or may not be the same as the first U-turn reference position P21.

[0381] When the operating device 20 accepts the operation of selecting the first target parking position P11 and the vehicle 1C has stopped at the first reversing stop position TP11 corresponding to the first target parking position P11, the control unit 11C at least controls the steering to make the vehicle 1C reverse, and drives the vehicle 1C to park from the first reversing stop position TP11 to the first target parking position P11 through autonomous driving.

[0382] When vehicle 1C stops at the first reversing stop position TP11 corresponding to the first target parking position P11, and the forward / reverse operation unit receives the reverse instruction operation and the braking operation unit (brake pedal) receives the braking operation to reduce the braking, the control unit 11C at least controls the steering to make vehicle 1C reverse, and drives vehicle 1C to park from the first reversing stop position TP11 to the first target parking position P11 through autonomous driving.

[0383] The forward / reverse control unit handles reverse instruction operations, which refers to shifting the gear to reverse by the passenger operating the gear shift lever (forward / reverse control unit) used for changing the gears of the transmission.

[0384] For example, the control unit 11C generates a reverse path R1 for moving and parking the vehicle 1C from the first reversing stop position TP11 to the first target parking position P11 using a known method. Then, when the forward / reverse operation unit receives a reverse instruction operation and the braking operation unit (brake pedal) receives a braking operation to reduce braking, the control unit 11C controls the steering based on the reverse path R1 to reverse the vehicle 1C and park the vehicle 1C to the first target parking position P11 through autonomous driving. In detail, the control unit 11C compares the current position of the vehicle 1C in the actual space RS with the reverse path R1, controls the steering to make the current position of the vehicle 1C the position on the reverse path R1, and controls the movement control device 12 to drive along the reverse path R1. During autonomous driving, the vehicle 1C is automatically controlled by the vehicle control device 10C without the need for manual driving operations by the passenger.

[0385] On the other hand, consider the following scenario: the operating device 20 does not accept the operation of selecting the first target parking location P11, and the vehicle 1C moves further from the first driving location DP1 according to the operation of the passenger accepted by the operating device 20, and then arrives at the second driving location DP2.

[0386] The second travel point DP2 is another travel point DP downstream of the first travel point DP1 of the vehicle 1C traveling in the direction of travel. Furthermore, the second travel point DP2 is a different travel point DP from the first travel point DP1.

[0387] Figure 14A This is an explanatory diagram of an example of parking assistance in this embodiment.

[0388] like Figure 14A As shown, imagine the following scenario: vehicle 1C passes through the first travel point DP1, and then travels forward to reach the second travel point DP2.

[0389] The control unit 11C performs the same processing as when the vehicle 1C arrives at the first driving location DP1.

[0390] In detail, similar to the process of driving at the first driving location DP1, the control unit 11C detects one or more target parking position candidates SP (e.g., target parking position candidate SP1a to target parking position candidate SP1c) based on the surrounding image V acquired by the camera 16. Then, the control unit 11C displays the detected multiple target parking position candidates SP as target parking position candidates on the display device 22 (illustration omitted), as described above. Then, similar to the case of the first driving location DP, the control unit 11C detects one target parking position candidate SP that is suitable for the current driving location DP of the vehicle 1C, i.e., the second driving location DP2, as the target parking position P1 corresponding to the second driving location DP2, i.e., the second target parking position P12.

[0391] exist Figure 14A As an example, the scenario shown illustrates the detection of the target parking location candidate SPb as the second target parking location P12.

[0392] When the control unit 11C detects a second target parking position P12 based on the surrounding image V at the second driving location DP2, it detects a U-turn reference position P2, i.e., a second U-turn reference position P22, corresponding to the detected second target parking position P12. Furthermore, the control unit 11C can also further derive the orientation of the vehicle 1C at the second U-turn reference position P22. The method for deriving the second U-turn reference position P22 and the orientation of the vehicle 1C at the second U-turn reference position P22 is the same as the method for deriving the reversing start position P2 and the orientation of the vehicle 1C at that reversing start position P2 described in the above embodiment.

[0393] Then, the control unit 11C displays the detected second target parking position P12 and the corresponding second U-turn reference position P22 on the display device 22 in a manner superimposed on the surrounding image V. That is, while the vehicle 1C is moving forward according to the operation received by the operating device 20, the control unit 11C displays the second target parking position P12 detected at the second driving location DP2 based on the surrounding image (second surrounding image) V acquired by the camera 16, and the corresponding second U-turn reference position P22 on the display device 22 in a manner superimposed on the surrounding image V.

[0394] For example, the control unit 11B uses a second frame to indicate the second target parking position P12 that is displayed on the display device 22. In addition, the control unit 11B displays the second target parking position P12, which is the object of the second target parking position, and the second U-turn reference position P22, which is the object of the second U-turn reference position, on the display device 22 in a manner that is superimposed on the surrounding image V (second surrounding image).

[0395] Figure 14B This is a schematic diagram of an example of a display screen 30K. The display screen 30K is an example of a display screen 30 displayed on the display device 22.

[0396] The control unit 11C causes the display device 22 to display a display screen 30K that overlays at least one of the second frame F12, representing the second target parking position P12 detected at the second driving location DP2, and the second U-turn reference position object P22OB, representing the second U-turn reference position P22 detected at the second driving location DP2, onto the peripheral image V (second peripheral image).

[0397] The second frame F12 is a frame representing the second target parking position P12. The control unit 11C generates a second frame F12 representing a virtual frame VF2 of the target parking space S1 that surrounds the second target parking position P12 in the actual space RS, and displays the second frame F12 superimposed on the peripheral image V displayed by the display device 22 at the position corresponding to the second target parking position P12.

[0398] The second target parking location object P12OB represents the display of the second target parking location P12. The second target parking location object P12OB can be, for example, a still image, character, animated image, or icon representing the target parking space S1 of the second target parking location P12. Figure 14B As an example, the following method is shown: the second target parking position object P12OB is a frame surrounding the target parking space S1 of the second target parking position P12, and a mark indicating the parking lot (a mark with the alphanumeric "P" surrounded by a circle). The control unit 11C overlays the second target parking position object P12OB, which represents the second target parking position P12 in the actual space RS, onto the position corresponding to the second target parking position P12 in the peripheral image V displayed by the display device 22.

[0399] The second U-turn reference position object P22OB represents the display of the second U-turn reference position P22. For example, the second U-turn reference position object P22OB can be used to represent the virtual frame VF2 (see reference). Figure 14A The image frame VF2 represents the outer frame of vehicle 1C when it is assumed to have stopped at the second U-turn reference position P22. The second U-turn reference position object P22OB simply represents the display of the second U-turn reference position P22, and can be a still image such as a frame, characters, animated images, or icons.

[0400] exist Figure 14BAs an example, the following arrangement is shown: the second target parking position object P12OB is a marker indicating the parking lot (a marker with the letter "P" surrounded by a circle) and a frame indicating the second target parking position P12. The control unit 11C causes the second U-turn reference position object P22OB, which represents the second U-turn reference position P22 in the actual space RS, to be superimposed on the peripheral image V displayed by the display device 22 at the position corresponding to the second U-turn reference position P22.

[0401] The third color of the second target parking position object P12OB and the second target parking position P12 correspond to the fourth color of the second U-turn reference position object P22OB. That is, the control unit 11C displays the pairing of the target parking position P1 and the U-turn reference position P2 at the second driving location DP2 using the corresponding colors, that is, the pairing of the second target parking position P12 and the second U-turn reference position P22.

[0402] As described above, the corresponding color refers to a color that a person can perceive as the same color or a related color. Therefore, by displaying the pairing of the second target parking position P12 and the second U-turn reference position P22 detected at the second driving location DP2 with the corresponding color on the display device 22, the control unit 11C can provide the occupants of the vehicle 1C with the information that the second target parking position P12 and the second U-turn reference position P22 are in a corresponding relationship at the second driving location DP2 in an easily recognizable manner.

[0403] Thus, even when vehicle 1C reaches the second driving location DP2, just as when it reaches the first driving location DP1, the control unit 11C will overlay the target parking position P1 detected based on the acquired surrounding image V at the driving location DP, which is either the second driving location DP2 or the first driving location DP1, and the U-turn reference position P2 corresponding to the target parking position P1 onto the surrounding image V. Furthermore, even at the second driving location DP2, just as at the first driving location DP1, the control unit 11C will display the pairing of the second target parking position P12 and the second U-turn reference position P22 detected at the second driving location DP2 in a corresponding color on the display device 22.

[0404] However, it is preferable that the first and second colors are not corresponding to the third and fourth colors.

[0405] That is, preferably, the colors (first color and second color) displayed on the display device 22 of the first target parking position object P11OB, the first frame F11, and the first U-turn reference position P21 at the first driving location DP1 are not corresponding to the colors (third color and fourth color) displayed on the display device 22 of the second target parking position object P12OB, the second frame F12, and the second target parking position P12 at the second driving location DP2.

[0406] The control unit 11C displays these first and second colors, along with the third and fourth colors, on the display device 22 in non-corresponding colors, thereby enabling the pairing of target parking positions P1 and U-turn reference positions P2 corresponding to different driving locations DP to be provided to the passenger in an identifiable manner by setting each pair to a different color.

[0407] Then, when the vehicle 1C reaches the second travel location DP2, the control unit 11C performs the same subsequent processing as when it reaches the first travel location DP1.

[0408] Figure 15 This is an explanatory diagram of the parking assistance in this embodiment.

[0409] In detail, when the control unit 11C displays the second target parking position P12 detected at the second driving location DP2 and the second U-turn reference position P22 corresponding to the second target parking position P12 on the display device 22 in a manner superimposed on the surrounding image V during the forward movement of the vehicle 1C, it determines whether the operation device 20 has accepted the operation of selecting the second target parking position P12.

[0410] Similar to the operation of selecting the second target parking position P12, the operation of the operating device 20 to select the second target parking position P12 is a steering operation performed by the passenger on the steering operation parts including the steering wheel (driving wheel) of the operating device 20 in the direction toward the second turning reference position P22, when the second target parking position P12 and the second turning reference position P22 are displayed on the display device 22.

[0411] For example, if the control unit 11C determines that the operation to select the second target parking position P12 has been accepted when the steering angle of the steering device is adjusted by the passenger's operation of the steering operation unit and the steering angle detected by the steering angle sensor that detects the steering angle of the steering device is a specific angle (e.g., 30 degrees or more) deviating towards the direction of the second U-turn reference position P22.

[0412] When vehicle 1C moves forward while the operation to select the second target parking position P12 has been accepted, for example, as Figure 15As shown, the vehicle 1C is tilted in the direction of forward travel (refer to arrow D) toward the second turn reference position P22 (arrow D').

[0413] When the operating device 20 accepts the operation of selecting the second target parking position P12, the control unit 11C at least fixes the second target parking position P12 on the display device 22.

[0414] The second U-turn reference position P22 is fixedly displayed on the display device 22, meaning that even if the vehicle 1C travels forward and reaches another travel location DP downstream of the second travel location DP2 in the direction of forward travel, the second U-turn reference position P22 is continuously superimposed on the surrounding image V.

[0415] In addition, when the operating device 20 accepts the operation of selecting the second U-turn reference position P22, the control unit 11C can also fix the second U-turn reference position P22 and the second target parking position P12 corresponding to the second U-turn reference position P22 on the display device 22.

[0416] Furthermore, preferably, when the operating device 20 accepts the operation of selecting the second U-turn reference position P22, even if the vehicle 1C travels forward and arrives at another travel location DP downstream of the direction of travel that is closer to the second travel location DP2, the control unit 11C will not display at least one of the target parking position P1 detected at the travel location DP and the U-turn reference position P2 corresponding to the target parking position P1 on the display device 22.

[0417] Therefore, when the operating device 20 accepts the operation of selecting the second target parking position P12, even if the vehicle 1C travels forward and reaches another driving position DP downstream of the second driving position DP2, the control unit 11C will, for example, Figure 14B The displayed screen 30J is continuously displayed on the display device 22.

[0418] Return to Figure 15 To continue the explanation. When the operating device 20 accepts the operation of selecting the second target parking position P12, and the vehicle 1C has stopped at the reversing stop position TP corresponding to the second target parking position P12, the control unit 11C at least controls the steering to make the vehicle 1C reverse, and drives the vehicle 1C to park from the reversing stop position TP to the second target parking position P12 through autonomous driving.

[0419] In this embodiment, the position where the vehicle 1C moves forward and stops according to the passenger's driving after accepting the operation of selecting the second target parking position P12, i.e., the reversing stop position TP, will be described as the second reversing stop position TP22. The second reversing stop position TP22 is an example of the reversing stop position TP. The second reversing stop position TP22 may or may not be the same as the second U-turn reference position P22.

[0420] When the operating device 20 accepts the operation of selecting the second target parking position P12, the vehicle 1C moves forward according to the driver's direction. That is, the movement from at least the second driving position DP2 to the stop at the second reversing stop position TP22 is carried out through the operation accepted by the operating device 20.

[0421] Then, after the vehicle 1C has moved forward according to the driver's direction, the control unit 11C determines whether it has stopped at the second reversing stop position TP22. Similar to the reversing stop position TP21, the vehicle 1C stops at the second reversing stop position TP22 through an operation received by the operating device 20. That is, the vehicle 1C stops at the second reversing stop position TP22 based on the braking operation performed by the driver.

[0422] When the operating device 20 accepts the operation of selecting the second target parking position P12, and the vehicle 1C has stopped at the second reversing stop position TP22 corresponding to the second target parking position P12, the control unit 11C at least controls the steering to make the vehicle 1C reverse, and drives the vehicle 1C to park from the second reversing stop position TP22 to the second target parking position P12 by autonomous driving.

[0423] In detail, when vehicle 1C stops at the second reversing stop position TP22 corresponding to the second target parking position P12, and the forward / reverse operation unit receives the reverse instruction operation, and the braking operation unit (brake pedal) receives the braking operation to reduce the braking, the control unit 11C at least controls the steering to make vehicle 1C reverse, and drives vehicle 1C to park from the second reversing stop position TP22 to the second target parking position P12 through autonomous driving.

[0424] Parking control via autonomous driving from the second U-turn reference position P22 to the second target parking position P12 is the same as parking control via autonomous driving from the first U-turn reference position P21 to the first target parking position P11.

[0425] Furthermore, consider the following scenario: the operating device 20 does not accept the operation to select the second U-turn reference position P22, and the vehicle 1C continues to move forward from the second driving location DP2 according to the passenger's operation accepted by the operating device 20, and then arrives at another new driving location DP. In this case, the control unit 11C sets the new driving location DP as the new first driving location DP1 and performs the same processing as described above.

[0426] Next, an example of the information processing flow performed by the control unit 11C in this embodiment will be described.

[0427] Figure 16 This is a flowchart illustrating an example of the information processing flow performed by the control unit 11C.

[0428] When vehicle 1C begins to move forward according to the driver's direction (step S300), control unit 11C detects one or more target parking position candidates SP for which vehicle 1C can park based on the surrounding image V acquired by camera 16 (step S302). Control unit 11C performs the processing of step S302, for example, if the steering angle of the steering device is below a predetermined angle. The predetermined angle is any angle smaller than a specific angle (e.g., 30 degrees) used for selecting the target parking position P1. For example, the predetermined angle is ±10 degrees, but it is not limited to this value.

[0429] Next, the control unit 11C causes the display device 22 to display a display screen 30I (step S304) showing the target parking position candidate object SP0B of each of the multiple target parking position candidate SPs detected in step S302 superimposed on the surrounding image V.

[0430] Through the processing in step S304, for example, a display is shown on the display device 22. Figure 11B The display screen 30I shown above. As described above, the color of the target parking position candidate object SP0B displayed on the display device 22 is a color that does not correspond to the target parking position P1 and the turning reference position P2 corresponding to the target parking position P1, that is, the first color, the second color, the third color and the fourth color mentioned above.

[0431] Therefore, the control unit 11C can provide the passenger with information about the target parking location candidate SP in a way that is easily identifiable.

[0432] Next, the control unit 11C detects one of the multiple target parking position candidates SP detected in step S302 that is suitable for the current driving location DP of the vehicle 1C as the target parking position P1 corresponding to the current driving location DP (step S306).

[0433] Then, the control unit 11C detects the U-turn reference position P2 corresponding to the target parking position P1 detected in step S306 (step S308). In addition, the control unit 11C can further derive the orientation of the vehicle 1C at the U-turn reference position P2.

[0434] Then, the control unit 11C displays the target parking position P1 detected in step S308 and the corresponding U-turn reference position P2 on the display device 22 in a manner superimposed on the surrounding image V (step S310). That is, while the vehicle 1C is moving forward according to the operation received by the operation device 20, the control unit 11C displays the target parking position P1 detected by the surrounding image V acquired by the camera 16 and the corresponding U-turn reference position P2 at the current driving location DP of the vehicle 1C on the display device 22 in a manner superimposed on the surrounding image V.

[0435] Through the processing in step S310, the driving location DP of vehicle 1C is displayed on the display device 22 accordingly. Figure 12B or Figure 14B The display screen shown is 30 (display screen 30J, display screen 30K).

[0436] That is, the control unit 11C, based on the constantly changing driving location DP of the vehicle 1C as it moves forward under the driver's control, displays the pairings of the target parking position P1 detected at each driving location DP based on the surrounding image V acquired by the camera 16 and the corresponding U-turn reference position P2 at each target parking position P1 on the display device 22 in a color corresponding to each pairing. Furthermore, the control unit 11C displays different pairs on the display device 22 in colors that do not correspond to each other. For example, the control unit 11C displays the currently detected pairing of target parking position P1 and U-turn reference position P2 on the display device 22 in a color that does not correspond to the pairings detected previously (e.g., the previous and the time before that). Additionally, if the previously detected pairing and the currently detected pairing exist at the same location in the actual space RS, the control unit 11C displays the currently detected pair on the display device 22 in a color corresponding to the previously detected pairing.

[0437] Next, the control unit 11C determines whether the operation of selecting the target parking position P1 displayed on the display device 22 in step S310 has been accepted (step S312). For example, the control unit 11C determines step S312 by identifying whether the steering angle detected by the steering angle sensor is a specific angle (e.g., 30 degrees or more) deviating from the direction of the U-turn reference position P2 when the target parking position P1 and the U-turn reference position P2 are displayed on the display device 22.

[0438] When a negative judgment is made in step S312 (step S312: "No"), proceed to step S314. In step S314, the control unit 11C determines whether the next candidate detection condition is met (step S314).

[0439] The control unit 11C determines step S314 by identifying whether the current position of the vehicle 1C (current driving location DP) has passed the target parking position P1 detected in the previous step S306.

[0440] The control unit 11C determines whether the vehicle 1C has passed the target parking position P1 detected in the previous step S306 by determining whether the driver's line of sight (e.g., the center of the wheelbase) of the vehicle 1C exceeds the downstream part of the vehicle 1C's forward driving direction in the target parking position P1.

[0441] If an affirmative judgment is made in step S314 (step S314: "Yes"), return to step S302 above. If a negative judgment is made in step S314 (step S314: "No"), return to step S312.

[0442] On the other hand, when the control unit 11C determines in step S312 that it has accepted the operation of selecting the target parking position P1 displayed on the display device 22 in step S310 (step S312: "Yes"), it proceeds to step S316. In step S316, the control unit 11C at least fixes the U-turn reference position P2 corresponding to the target parking position P1 displayed on the display device 22 in step S310 to be displayed on the display device 22 (step S316).

[0443] When the operating device 20 accepts the operation of selecting the target parking position P1, the vehicle 1C moves forward according to the driver's direction. That is, the movement from the moment the affirmative judgment is made in step S312 until it stops at the reversing stop position TP is performed by the operation accepted by the operating device 20. Through the processing in step S316, even if the vehicle 1C has moved forward afterward, the U-turn reference position P2 displayed on the display device 22 in step S310 is continuously superimposed on the surrounding image V. Therefore, by operating the operating device 20 with reference to the U-turn reference position P2 displayed on the display device 22, the driver can easily perform the operation of moving the vehicle 1C toward the U-turn reference position P2.

[0444] Next, the control unit 11C determines whether the operating device 20 has stopped at the reversing stop position TP (step S318). For example, after the control unit 11C makes an affirmative determination in step S312 and the vehicle 1C moves forward according to the passenger's driving, it determines whether the vehicle 1C has stopped at the reversing stop position TP by distinguishing whether the vehicle 1C has stopped according to the passenger's braking operation. Alternatively, as described above, after the control unit 11C makes an affirmative determination in step S312 and the vehicle 1C moves forward according to the passenger's driving, it can also determine whether the vehicle 1B has stopped at the reversing stop position TP by distinguishing whether the vehicle 1C has stopped according to the passenger's braking operation and whether the gear of the vehicle 1B has been shifted to reverse (R).

[0445] The control unit 11C repeatedly performs a negative judgment (step S318: "No") until a positive judgment is made in step S318 (step S318: "Yes"). When the control unit 11C makes a positive judgment in step S318 (step S318: "Yes"), it proceeds to step S320.

[0446] In step S320, the control unit 11C generates a reversing path R1 for moving the vehicle 1C from the reversing stop position TP determined to be stopped in step S318 to the target parking position P1 detected in step S306 and parking by autonomous driving (step S320).

[0447] Then, when the forward / reverse operation unit receives a reverse instruction and the braking operation unit (brake pedal) receives a braking operation to reduce braking, the control unit 11C controls at least the steering wheel to reverse the vehicle 1C from the reversing stop position TP to the target parking position P1, and autonomously drives the vehicle 1C to park at the target parking position P1 (step S322). That is, the control unit 11C controls the movement control device 12 to autonomously drive along the reverse path R1 generated in step S320. Furthermore, when the vehicle 1C begins to reverse through the processing of step S322, the control unit 11C can deselect the display of the turning reference position P2, which was fixedly displayed on the peripheral image V in step S316. Then, this routine ends.

[0448] As described above, the vehicle control device 10C of this embodiment is mounted on a vehicle 1C, which includes an operating device 20 for receiving passenger operations, a camera 16 for acquiring surrounding images V, a display device 22 for the passenger to visually confirm, and a motion control device 12 for controlling steering. During the forward movement of the vehicle 1C according to the operations received by the operating device 20, the vehicle control device 10C displays a first target parking position P11 detected at a first driving location DP1 based on the first surrounding image (surrounding image V) acquired by the camera 16, and a first U-turn reference position P21 corresponding to the first target parking position P11, superimposed on the first surrounding image (surrounding image V) on the display device 22. The vehicle control device 10C then displays a second target parking position P12 detected at a second driving location DP2 based on the second surrounding image (surrounding image V) acquired by the camera 16, and a second U-turn reference position P22 corresponding to the second target parking position P12, superimposed on the second surrounding image (surrounding image V) on the display device 22. When the operating device 20 accepts the operation of selecting the second target parking position P12 and the vehicle 1C has stopped at the reversing stop position TP (second reversing stop position TP2), the vehicle control device 10C at least controls the steering to make the vehicle 1C reverse, and drives the vehicle 1C from the reversing stop position TP (second reversing stop position TP2) to the second target parking position P12 through autonomous driving.

[0449] In the prior art, there is insufficient assistance for autonomous driving-based parking corresponding to the detected target parking position, making it difficult to provide appropriate parking assistance.

[0450] On the other hand, during the forward movement of the vehicle 1C according to the operation received by the operation device 20, the vehicle control device 10C displays the first target parking position P11 detected based on the first surrounding image (surrounding image V) at the first driving location DP1 and the first U-turn reference position P21 corresponding to the first target parking position P11 on the display device 22 in a manner superimposed on the first surrounding image (surrounding image V). In addition, the vehicle control device 10C displays the second target parking position P12 detected based on the second surrounding image (surrounding image V) at the second driving location DP2 after subsequent forward movement, and the second U-turn reference position P22 corresponding to the second target parking position P12 on the display device 22 in a manner superimposed on the second surrounding image (surrounding image V). Furthermore, when the operating device 20 accepts the operation of selecting the second target parking position P12 and the vehicle 1C has stopped at the second reversing stop position TP2, the vehicle control device 10C at least controls the steering to make the vehicle 1C reverse, and drives the vehicle 1C to park from the second reversing stop position TP2 to the second target parking position P12 through autonomous driving.

[0451] Thus, the vehicle control device 10C of this embodiment can sequentially display the target parking position P1 detected at each driving location DP and the corresponding U-turn reference position P2 of the vehicle 1C moving according to the driver's driving on the display device 22 in a manner superimposed on the surrounding image V. Moreover, when the vehicle 1C stops at the reversing stop position TP corresponding to the target parking position P1 after the driver selects the currently displayed target parking position P1 by operating the operation device 20, the vehicle control device 10C autonomously drives the vehicle 1C to park from the reversing stop position TP to the target parking position P1.

[0452] Therefore, the vehicle control device 10C of this embodiment can provide appropriate assistance for autonomous parking corresponding to the detected target parking position P1.

[0453] Therefore, the vehicle control device 10C of this embodiment can provide appropriate parking assistance.

[0454] Furthermore, in this embodiment, the following method is described as an example: the vehicle control device 10C detects one target parking position candidate SP as the target parking position P1 from among a plurality of target parking position candidate SPs detected at each travel location DP. However, the vehicle control device 10C may also detect a predetermined number of target parking position candidate SPs from the plurality of target parking position candidate SPs detected at the travel location DP as the target parking position P1. The predetermined number is, for example, 2 or 3, but is not limited to this value.

[0455] In this case, the vehicle control device 10C also pairs the target parking position P1 detected at the current driving location DP with the U-turn reference position P2 corresponding to the target parking position P1, and displays each pair in a color corresponding to the pair on the display device 22. Alternatively, it is preferable that the colors of the multiple pairs are displayed on the display device 22 in different colors.

[0456] (Fourth Implementation)

[0457] In this embodiment, the following method will be described: the selection of the target parking position P1 is accepted by parking assistance processing to the target parking position P1 and by the passenger's steering operation on the steering wheel (driving wheel, steering control unit) 20A.

[0458] Figure 1 This is a block diagram illustrating an example of the overall structure of the vehicle 1D according to this embodiment.

[0459] Vehicle 1D is identical to Vehicle 1 in the above-described embodiment, except that it has Vehicle Control Device 10D instead of Vehicle Control Device 10. Therefore, the configuration of Sensor 14 and Camera 16 in Vehicle 1D is also the same as in Vehicle 1 (see [reference]). Figure 2 Furthermore, the exterior of vehicle 1D and the structure near the driver's seat 24A are the same as those of vehicle 1 (see reference). Figure 3 , Figure 4 ).

[0460] The vehicle control unit 10D is an electronic control unit that uniformly controls all parts of the vehicle 1D. Similar to the vehicle control unit 10, the vehicle control unit 10D uses sensor information and surrounding images received from the sensor 14 and camera 16 respectively to control the motion control unit 12 to optimize the driving state of the vehicle 1D. Furthermore, the vehicle control unit 10D controls the motion control unit 12 to enable the vehicle 1D to drive autonomously.

[0461] The vehicle control device 10D includes a control unit 11D. Part or all of the control unit 11D can be a software structure implemented through the cooperation of a processor and various programs stored in a memory. Alternatively, part or all of the control unit 11D can also be a hardware structure implemented by dedicated circuitry or the like.

[0462] The control unit 11D provides unified control over all parts of vehicle 1D.

[0463] In this embodiment, the control unit 11D performs information processing for parking assistance of the vehicle 1D. In this embodiment, similar to the embodiments described above, the following approach is envisioned and described: after the vehicle 1D is driven forward until at least a portion of the vehicle body crosses at least a portion of the target parking position P1, the vehicle 1D is stopped according to the passenger's driving, and the vehicle 1D reverses from the reversing stop position TP, which is the stopping position, and moves towards the target parking position P1 by autonomous driving.

[0464] First, an overview of the parking assistance provided by the control unit 11D in this embodiment will be explained.

[0465] Figure 17A and Figure 17B This is an explanatory diagram of an example of parking assistance in this embodiment.

[0466] Figure 17A This is an explanatory diagram illustrating an example of a target parking position P1 detected by the control unit 11D.

[0467] like Figure 17A As shown, during the process of the vehicle 1D moving forward according to the operation of the passenger received by the operating device 20, the control unit 11D (refer to...) Figure 17A When the operating device 20 receives a parking assist start instruction (path R0), it detects the target parking position P1. Then, when the vehicle 1D stops and the prescribed operation is received, the control unit 11C controls the steering to reverse the vehicle 1D (refer to the reversing path R1), and autonomously drives the vehicle 1D to park in the target parking space S1 of the target parking position P1.

[0468] Figure 17B This is an explanatory diagram illustrating an example where the control unit 11D detects two target parking positions P1.

[0469] like Figure 17B As shown, during the process of the vehicle 1D moving forward according to the operation of the passenger received by the operating device 20, the control unit 11D (refer to...) Figure 17B When the operating device 20 receives a parking assist start instruction (path R0), it detects multiple target parking positions P1 (first target parking position P10, second target parking position P20). Then, when the vehicle 1D stops and the prescribed operation is received, the control unit 11D controls at least the steering wheel to reverse the vehicle 1D (refer to the reversing path R1), and autonomously drives the vehicle 1D toward the target parking space S1 of one of the two target parking positions P1 selected by the passenger's steering operation on the steering wheel (driving wheel, steering operation unit) 20A.

[0470] Next, the processing procedures for the control unit 11D when it detects one target parking position P1 and two target parking positions P1 will be explained in detail.

[0471] First, the parking assistance processing when the control unit 11D detects a target parking position P1 will be explained in detail. That is, the parking assistance processing utilizing... Figure 17A The parking assistance modes described above will be explained in detail.

[0472] Figure 18A This is an explanatory diagram of an example of parking assistance in this embodiment.

[0473] During the process of the vehicle 1D moving forward according to the operation device 20 received by the passenger, the control unit 11D (refer to...) Figure 18A The control unit 11D directs the display device 22 to display the peripheral image V acquired by the camera 16 (in the direction of arrow D). Furthermore, during autonomous driving, the control unit 11D also directs the display device 22 to display the peripheral image V acquired by the camera 16. This is also the case in the above embodiment.

[0474] Furthermore, in the technology disclosed in this specification, the real-time peripheral image V is displayed on the display device 22 in principle. That is, in this embodiment and the embodiments described above, when the display device 22 displays the peripheral image V, the display device 22 displays the real-time peripheral image V.

[0475] When the vehicle 1D is moving forward according to the operation of the passenger received by the operation device 20, and the operation device 20 receives a parking assistance start instruction, the control unit 11D detects the target parking position P1 where the vehicle 1D can park based on the surrounding image V acquired by the camera 16.

[0476] Parking assist start indication is an indication that parking assistance has begun for vehicles autonomously driving into any parking space within the actual space RS that is capable of parking.

[0477] The passenger initiates the parking assistance instruction by performing an operation on the operating device 20 that indicates a parking assistance start instruction. The operation indicating the parking assistance start instruction is a predetermined operation for the operating device 20. In this embodiment, for example, the operating device 20 accepts the parking assistance start instruction by the passenger operating the display area (button image) displayed on the display screen 30 for accepting the parking assistance start instruction. However, the operation indicating the parking assistance start instruction can also be an operation performed on a specific button for accepting the parking assistance start instruction, included in the operating button 20B (see Figure 8), which is an example of the operating device 20, and is not limited to an operation on the button image.

[0478] Figure 18B This is a schematic diagram of an example of display screen 30L. Display screen 30L is an example of display screen 30 displayed on the operating device 20.

[0479] Display screen 30L is a display screen 30 located at a position corresponding to the current position of vehicle 1C in the surrounding image V acquired by camera 16, displaying an image of vehicle 1C. Display screen 30L includes button image B. Button image B is a display area for responding to parking assist start instructions.

[0480] When the passenger instructs the button image B on the display device 22 while the vehicle 1D is moving forward according to the passenger's operation, the operating device 20 receives the parking assist start instruction. When the operating device 20 receives the parking assist start instruction, the control unit 11D detects the target parking position P1 where the vehicle 1D can park based on the surrounding image V acquired by the camera 16.

[0481] Furthermore, the control unit 11D may begin detecting the target parking position P1 before the operating device 20 receives the parking assist start instruction. For example, the control unit 11D may continuously detect the target parking position P1 during the movement of the vehicle 1D. In this embodiment, the method in which the control unit 11D detects the target parking position P1 when it receives the parking assist start instruction will be described as an example.

[0482] Figure 19A This is an explanatory diagram of an example of parking assistance in this embodiment.

[0483] When the operating device 20 receives a parking assistance start instruction, similarly to the embodiment described above, the control unit 11D detects one or more target parking position candidates SP for which the vehicle 1D can park based on the surrounding image V acquired by the camera 16. The definition of the target parking position candidate SP is the same as described above.

[0484] Similar to the above embodiment, the control unit 11D analyzes the surrounding image V acquired by the camera 16 using a known method, thereby detecting multiple target parking position candidates SP. Figure 19A As an example, the control unit 11D is shown to detect the status of multiple target parking position candidates SP (target parking position candidate SPa ~ target parking position candidate SPc).

[0485] Then, the control unit 11D detects the target parking position P1 as the appropriate target parking position candidate SP among the multiple target parking position candidate SPs detected, which is suitable for the current position of the vehicle 1D, i.e., the current driving location DP.

[0486] The control unit 11D detects a target parking position candidate SP as the target parking position P1 in the same manner as in the above embodiment. That is, the control unit 11D detects a target parking position candidate SP as the target parking position P1 based on the positional relationship between the current position of the vehicle 1D, i.e., the current driving location DP, and the target parking position candidate SP.

[0487] exist Figure 19A As an example, the scenario shown is where the control unit 11D detects the target parking position candidate SPa as the target parking position P1.

[0488] When the control unit 11D detects a target parking position P1, it detects a U-turn reference position P2 corresponding to the detected target parking position P1. Furthermore, the control unit 11D can also further derive the orientation of the vehicle 1D at the U-turn reference position P2. The method for deriving the U-turn reference position P2 and the orientation of the vehicle 1D at the U-turn reference position P2 is the same as the method for deriving the reversing start position P2 and the orientation of the vehicle 1D at the reversing start position P2 described in the above embodiment.

[0489] Then, the control unit 11D displays the detected target parking position P1 on the display device 22 in a superimposed manner on the surrounding image V. Additionally, the control unit 11D can also display both the target parking position P1 and the corresponding U-turn reference position P2 on the display device 22 in a superimposed manner on the surrounding image V. Furthermore, the control unit 11D can also display multiple target parking position candidates SP used to select the target parking position P1 on the display device 22 in a superimposed manner on the surrounding image V.

[0490] Figure 19B This is a schematic diagram of an example of display screen 30M. Display screen 30M is an example of display screen 30 displayed on display device 22.

[0491] The control unit 11D causes the display device 22 to display a display screen 30M that overlays a first frame F1 representing the detected target parking position P1, a second frame F2 representing the U-turn reference position P2 corresponding to the target parking position P1, and a frame FS representing the target parking position candidate SP used to select the target parking position P1 onto the surrounding image V.

[0492] As described in the above embodiment, the first frame F1 is an image representing the target parking position P1. The second frame F2 is an image representing the U-turn reference position P2. The frame FS is an image representing the target parking position candidate SP. The control unit 11D can display the target parking position P1, the target parking position candidate SP, and the U-turn reference position P2 respectively, superimposed on the surrounding image V. Their display method is not limited to the images representing the frames. For example, they can also be displayed as still images, characters, animated images, icons, etc., outside the frames.

[0493] Similarly to the above-described embodiment, for example, the control unit 11D generates a first frame F1 of a virtual frame VF1 representing a target parking space S1 that surrounds a target parking position P1 in the actual space RS, a virtual frame VF2 representing a turning reference position P2, and a virtual frame representing a candidate target parking position SP, and displays them superimposed on the peripheral image V displayed by the display device 22 at their respective corresponding positions.

[0494] When the control unit 11C displays the target parking position candidate SP overlaid on the surrounding image V, it is preferable to display the target parking position candidate SP on the display device 22 in a display method that is less likely to cause the passenger to look at it compared to the target parking position P1 and the U-turn reference position P2.

[0495] In addition, as described above, the control unit 11D can either display the target parking position P1 superimposed on the surrounding image V, or it can display at least one of the U-turn reference position P2 and the target parking position candidate SP superimposed on the surrounding image V.

[0496] In this embodiment, the peripheral image V displayed on the display screen 30 can be either image V1, which is a peripheral image V in the direction of travel of vehicle 1D, or image V2, which is a top-down view of vehicle 1D. Furthermore, similar to the embodiment described above, in this embodiment, images V1 and V2 can be displayed in different display areas on the display screen 30. Additionally, the control unit 11D can also overlay an icon image representing vehicle 1D onto the peripheral image V.

[0497] Figure 20 This is an explanatory diagram of the parking assistance in this embodiment.

[0498] When the control unit 11D receives a parking assist start instruction while the vehicle 1D is moving forward, and displays the detected target parking position P1 and the turn reference position P2 corresponding to the target parking position P1 on the display device 22 in a manner superimposed on the surrounding image V, it determines whether to accept the operation of selecting the target parking position P1.

[0499] The operation of the operating device 20 to select the target parking position P1 is a steering operation performed by the passenger on the steering wheel (steering control unit, driving wheel) 20A of the operating device 20 in the direction of the turning reference position P2.

[0500] For example, if the control unit 11D adjusts the steering angle of the steering device by the passenger's operation of the steering operation unit, and the steering angle detected by the steering angle sensor that detects the steering angle of the steering device is a specific angle (e.g., 30 degrees or more) deviating towards the turning reference position P2, it can determine that the operation of selecting the turning reference position P2 has been accepted.

[0501] When the vehicle is moving forward in the state of accepting the operation to select the U-turn reference position P2, for example, as Figure 20 As shown, the vehicle 1D is tilted in the direction of forward travel (refer to arrow D) toward the turning reference position P2 (arrow D').

[0502] When the operating device 20 accepts the operation of selecting the U-turn reference position P2, the control unit 11D determines that the target parking position P1 corresponding to the currently displayed U-turn reference position P2 has been selected.

[0503] Then, when the vehicle 1D stops after it has moved further forward according to the driver's direction, the control unit 11D sets the stop position to the reversing stop position TP.

[0504] The vehicle 1D can be stopped after it has moved forward, either by a stop control achieved by the control unit 11D based on the surrounding image V, or by strengthening the braking operation at the brake pedal (brake operation unit) of the operating device 20.

[0505] In this embodiment, the case in which the vehicle 1D stops after moving forward is achieved by strengthening the braking operation at the brake pedal (brake operation unit) of the operating device 20 is used as an example for explanation.

[0506] Strengthening the braking operation at the brake pedal (brake operating part) of the operating device 20 refers to increasing the amount and force with which the passenger presses the brake pedal. In other words, strengthening the braking operation at the brake pedal (brake operating part) of the operating device 20 refers to the braking operation performed by the passenger to bring the vehicle 1D to a stop.

[0507] As described in the above embodiments, the reversing stop position TP refers to the position of vehicle 1D in the actual space RS when vehicle 1D stops after the target parking position P1 and the U-turn reference position P2 are derived by the control unit 11D. The reversing stop position TP may or may not be the same as the U-turn reference position P2.

[0508] When vehicle 1D stops at the reversing stop position TP, and the forward / reverse operation unit (gear lever) of the operation device 20 receives a reverse instruction operation, and the braking operation unit (brake pedal) of the operation device 20 receives a braking operation to reduce the braking, the control unit 11D at least controls the steering to make vehicle 1D reverse and park by autonomous driving.

[0509] Similar to the above-described embodiments, the forward / reverse operation unit accepts reverse instruction operation, which means that the gear is shifted to reverse by the operation of the shift lever (forward / reverse operation unit) used by the passenger for shifting the gear of the transmission.

[0510] Preferably, the control unit 11D causes the display device 22 to display information prompting the vehicle 1D to stop and changing the gear lever to reverse (R) after the vehicle 1D stops at the reversing stop position TP.

[0511] Figure 21A This is a schematic diagram of an example of display screen 30N. Display screen 30N is an example of display screen 30.

[0512] For example, after the target parking position P1 is selected, as the vehicle 1C moves forward according to the operation of the operating device 20 by the passenger, the control unit 11D causes the display device 22 to display information M10.

[0513] Information M10 is a message that prompts the vehicle 1D to stop after forward travel has been selected at the target parking position P1, and then shifts the gear lever to reverse after stopping. Figure 21A As an example, the following method is shown: Information M10 is a character that prompts the vehicle 1D to stop after forward travel following the selection of the target parking position P1, and to shift the gear lever to reverse after stopping. However, information M10 can also be a still image, animated image, icon, etc.

[0514] Furthermore, preferably, when the vehicle 1D stops at the reversing stop position TP according to the driver's driving and the gear is changed to reverse according to the driver's operation of the gear lever, the control unit 11D causes the display device 22 to display information prompting the braking operation to be reduced.

[0515] Reduction of braking operation refers to the reduction in the amount and force applied to the brake pedal by the passenger. In other words, reduction of braking operation refers to the actions taken by the passenger to change the vehicle 1D from a stopped state to a moving state by applying the brake pedal.

[0516] Figure 21B This is a schematic diagram of an example of display screen 30O. Display screen 30O is an example of display screen 30.

[0517] For example, when vehicle 1D stops at the reversing stop position TP based on the driver's actions, and the gear is shifted to reverse by the driver's operation of the gear lever, control unit 11D causes display device 22 to display information M11. Information M11 is information prompting a reduction in braking operation. Figure 21B In the example shown, information M11 is a character that prompts the braking operation. Information M11 simply needs to be information that prompts the braking operation, and its display method is not limited; it can be a still image, an animated image, an icon, etc.

[0518] Return to Figure 20 To continue the explanation, when the braking operation unit (brake pedal) receives a braking operation to reduce braking, the control unit 11D at least controls the steering to reverse the vehicle 1D, and then autonomously drives the vehicle 1D to park at the target parking position P1. During this autonomous driving, the control unit 11D can also at least control the steering and acceleration / deceleration to reverse the vehicle 1D, and then autonomously drive the vehicle 1D to park at the target parking position P1.

[0519] Similar to the embodiment described above, the control unit 11D compares the current position of the vehicle 1D in the actual space RS with the reversing path R1, at least controls the steering to make the current position of the vehicle 1D a position on the reversing path R1, and controls the movement control device 12 to drive along the reversing path R1. During autonomous driving, the vehicle 1D is automatically controlled by the vehicle control device 10D without the need for manual driving operations by the passenger.

[0520] Preferably, when vehicle 1D is parked at the target parking position P1, the control unit 11D causes the display device 22 to display information indicating that parking is complete.

[0521] Figure 21C This is a schematic diagram of an example of display screen 30P. Display screen 30P is an example of display screen 30. Display screen 30P contains information M12. Information M12 indicates that parking is complete. Figure 21C In the example shown, information M12 indicates that parking is complete. Information M12 simply indicates that parking is complete; its display method is not limited and can be a still image, animated image, icon, etc.

[0522] Next, a detailed explanation of the parking assistance processing when the control unit 11D detects two target parking positions P1 will be provided. That is, the processing utilizing... Figure 17B The parking assistance modes described above will be explained in detail.

[0523] Furthermore, in this mode, the control unit 11D can detect two or more target parking positions P1, and is not limited to two. In this embodiment, the method in which the control unit 11D detects two target parking positions P1 will be described as an example.

[0524] Figure 18A This is an explanatory diagram of an example of parking assistance in this embodiment.

[0525] Similar to the above mode, in this mode, the control unit 11D also moves forward in the vehicle 1D according to the passenger's operation received by the operating device 20 (see...). Figure 18A (In the direction of arrow D in the image), the display device 22 displays the surrounding image V acquired by the camera 16. When the control unit 11D receives a parking assist start instruction from the operating device 20 while the vehicle 1D is moving forward according to the passenger's operation received by the operating device 20, the operating device 20 detects the target parking position P1 where the vehicle 1D can park based on the surrounding image V acquired by the camera 16.

[0526] Similarly to the above model, for example, the passenger operates... Figure 18B The button image B on the display screen 30L is used to input the parking assistance start instruction. When the passenger operates the button image B on the display device 22, the operating device 20 accepts the parking assistance start instruction.

[0527] When the operating device 20 receives a parking assist start instruction, the control unit 11D detects a target parking position P1 where the vehicle 1D can park based on the surrounding image V acquired by the camera 16. In this mode, the case where two target parking positions P1 are detected will be described. Similarly, in the above mode, the control unit 11D detects multiple candidate target parking positions SP where the vehicle 1D can park based on the surrounding image V acquired by the camera 16.

[0528] Then, the control unit 11D detects two target parking position candidate SPs from the multiple detected target parking position candidate SPs that are suitable for the current position of the vehicle 1D, i.e., the current driving location DP, as two target parking positions P1. In detail, based on the positional relationship between the current position of the vehicle 1D, i.e., the current driving location DP, and the target parking position candidate SPs, the control unit 11D detects two target parking position candidate SPs that can autonomously reverse and park from the predicted position where the vehicle will continue to move and stop as target parking positions P1.

[0529] Figure 22 This is an explanatory diagram of an example of parking assistance in this embodiment.

[0530] exist Figure 22 As an example, the scenario is shown after the control unit 11D detects the first target parking position P10 and the second target parking position P20 as the target parking position P1.

[0531] When the control unit 11D detects two target parking positions P1, it detects the U-turn reference position P2 corresponding to the detected target parking positions P1. Furthermore, the control unit 11D can also further derive the orientation of the vehicle 1D at the U-turn reference position P2. The method for deriving the U-turn reference position P2 and the orientation of the vehicle 1D at the U-turn reference position P2 is the same as the method for deriving the reversing start position P2 and the orientation of the vehicle 1D at the reversing start position P2 described in the above embodiment.

[0532] In addition, Figure 22 In this example, a scenario is shown where a (same) U-turn reference position P2 is detected for two detected target parking positions P1. However, the control unit 11D can also detect different U-turn reference positions P2 for each of the two target parking positions P1. These U-turn reference positions P2 are positions where the vehicle can autonomously reverse and park at each of the two target parking positions P1. In this embodiment, the scenario where a single U-turn reference position P2 is detected for two detected target parking positions P1 will be described.

[0533] Then, the control unit 11D displays the two detected target parking positions P1 on the display device 22 in a manner superimposed on the surrounding image V. Additionally, the control unit 11D can also display both the two target parking positions P1 and the corresponding U-turn reference position P2 on the display device 22 in a manner superimposed on the surrounding image V. Furthermore, the control unit 11D can also display multiple target parking position candidates SP used to select the target parking position P1 on the display device 22 in a manner superimposed on the surrounding image V.

[0534] Figure 23A This is a schematic diagram of an example of display screen 30Q. Display screen 30Q is an example of display screen 30 displayed on display device 22.

[0535] The control unit 11D causes the display device 22 to display a display screen 30Q that overlays a first frame F1a representing the detected first target parking position P10, a second frame F2a representing the second target parking position P20, and a second frame F2 representing the U-turn reference position P2 corresponding to these target parking positions P1 onto the surrounding image V.

[0536] The first frame F1a is an image representing the first target parking position P10. The first frame F1b is an image representing the second target parking position P20. As described in the above embodiment, the second frame F2 is an image representing the U-turn reference position P2. The control unit 11D can display each position in the target parking position P1 and the U-turn reference position P2 by superimposing it onto the surrounding image V. Their display method is not limited to the images in the frames. For example, their display method can also be a still image, characters, animated images, icons, etc. outside the frames.

[0537] In this embodiment, the control unit 11D generates a first frame F1a and a first frame F1b representing the target parking spaces S1 of the first target parking position P10 and the second target parking position P20 in the actual space RS, respectively, and displays the first frames F1a and F1b superimposed on the peripheral image V displayed by the display device 22 at their respective corresponding positions.

[0538] Return to Figure 22 Let's continue with the explanation.

[0539] When the control unit 11D receives a parking assist start instruction while the vehicle 1D is moving forward, and displays the detected target parking position P1 (first target parking position P10, second target parking position P20) and the U-turn reference position P2 on the display device 22 in a manner superimposed on the surrounding image V, it determines whether to accept the operation of selecting the target parking position P1.

[0540] Similarly, in this mode, the control unit 11D determines whether the operation to select the target parking position P1 has been accepted by identifying whether the steering operation performed by the passenger toward the turning reference position P2 corresponding to the target parking position P1 (first target parking position P10, second target parking position P20).

[0541] Similarly, for example, if the control unit 11D adjusts the steering angle of the steering device by the passenger's operation of the steering operation unit, and the steering angle detected by the steering angle sensor that detects the steering angle of the steering device is a specific angle (e.g., 30 degrees) or more deviating towards the U-turn reference position P2, it can determine that a steering operation towards the U-turn reference position P2 has been accepted. Then, if the control unit 11D accepts a steering operation towards the U-turn reference position P2, it can select a target parking position P1 (first target parking position P10, second target parking position P20) corresponding to the U-turn reference position P2.

[0542] When vehicle 1D is moving forward after receiving a steering operation in the direction of the turn-around reference position P2, for example, Figure 22 As shown, the vehicle 1D is tilted in the direction of forward travel (refer to arrow D) toward the turning reference position P2 (arrow D').

[0543] When the operating device 20 accepts the operation of selecting the U-turn reference position P2, the control unit 11D determines that the two target parking positions P1 corresponding to the currently displayed U-turn reference position P2 have been selected.

[0544] Furthermore, when the control unit 11D receives a steering operation performed by the passenger on the steering wheel (steering control unit, steering wheel) 20A of the operating device 20 in the direction of the turn reference position P2, when the vehicle 1D stops after it has moved further forward according to the driver's driving, the stop position is set as the reversing stop position TP.

[0545] The vehicle 1D can be stopped after it has moved forward, either by a stop control achieved by the control unit 11D based on the surrounding image V, or by strengthening the braking operation at the brake pedal (brake operation unit) of the operating device 20.

[0546] In this embodiment, similarly to the above, we will take the case where the vehicle 1D stops after moving forward by strengthening the braking operation at the brake pedal (brake operation unit) of the operating device 20 as an example.

[0547] Furthermore, consider the following scenario: Vehicle 1D stops at the reversing stop position TP, and the forward / reverse control unit (gear lever) of the operating device 20 receives a reverse instruction operation, while the braking control unit (brake pedal) of the operating device 20 receives a braking operation to reduce braking. In this case, the control unit 11D displays the two target parking positions P1 (first target parking position P10, second target parking position P20) superimposed on the surrounding image V on the display device 22, and at least controls the steering wheel to reverse vehicle 1D, and autonomously drives vehicle 1D to park.

[0548] Figure 23A This is a schematic diagram of an example of display screen 30Q. Display screen 30N is an example of display screen 30.

[0549] For example, after receiving a steering operation in the direction of the turn reference position P2, the vehicle 1C moves forward according to the passenger's operation of the operating device 20, and the control unit 11D causes the display device 22 to display information M13.

[0550] Information M13 is the same as information M10. That is, information M13 causes the vehicle 1D to stop after forward travel has been selected at the target parking position P1, and then the gear lever is shifted to reverse after stopping. Figure 23A As an example, the following method is shown: Information M13 is a character that prompts the vehicle 1D to stop after forward driving following the selection of the target parking position P1, and then shifts the gear lever to reverse after stopping. However, information M13 can also be a still image, animated image, icon, etc.

[0551] Furthermore, when the vehicle 1D stops at the reversing stop position TP according to the driver's driving, and the gear is changed to reverse according to the driver's operation of the gear lever, the control unit 11D causes the display device 22 to display information prompting the braking operation to be reduced.

[0552] Figure 23B This is a schematic diagram of an example of display screen 30R. Display screen 30R is an example of display screen 30.

[0553] For example, when vehicle 1D stops at the reversing stop position TP based on the passenger's driving, and the gear is shifted to reverse by the passenger's operation of the gear lever, control unit 11D causes display device 22 to display information M14. Information M14 is the same as information M11. That is, information M14 is information that prompts the reduction of braking operation. Figure 23B In the example shown, information M14 is a character that prompts the braking operation. Information M14 simply needs to be information that prompts the braking operation, and its display method is not limited; it can be a still image, an animated image, an icon, etc.

[0554] Furthermore, in this mode, the control unit 11D accepts the selection of a target parking position P1 based on the steering operation performed by the passenger on the steering wheel (steering control unit, steering wheel) 20A included in the operating device 20.

[0555] In detail, when the vehicle 1D stops and the gear lever 20C (forward and reverse operation unit) of the operating device 20 is switched to reverse gear to receive a reverse instruction operation, the control unit 11D accepts the steering operation performed by the passenger on the steering wheel (steering operation unit, driving wheel) 20A of the operating device 20 to select the first target parking position P10 or the second target parking position P20.

[0556] use Figure 17B Please provide an explanation.

[0557] The steering operation to select the first target parking position P10 is a steering operation performed on the steering wheel 20A from the position where vehicle 1D has stopped (reversing stop position TP), in the direction of vehicle 1D toward the first target parking position P10 (refer to arrow Y1). The steering operation to select the second target parking position P20 is a steering operation performed on the steering wheel 20A from the position where vehicle 1D has stopped, in the direction of vehicle 1D toward the opposite direction to the direction toward the first target parking position P10 (refer to arrow Y2). Figure 17B As an example, the following situation is shown: the direction of arrow Y1 toward the first target parking position P10 is the direction in which the steering wheel 20A is turned to the left, and the direction opposite to arrow Y1, namely arrow Y2, is the direction in which the steering wheel 20A is turned to the right.

[0558] Then, the control unit 11D highlights the selected target parking position P1.

[0559] Figure 23B This is a schematic diagram of an example of display screen 30R. Display screen 30R is an example of display screen 30. Figure 23B This illustrates a scenario where a steering operation has been accepted to select the first target parking position P10.

[0560] When the control unit 11D receives a steering operation to select the first target parking position P10, it will highlight the selected first target parking position P10 among the first target parking position P10 and the second target parking position P20 displayed by the display device 22.

[0561] Figure 23C This is a schematic diagram illustrating an example of display screen 30S. Display screen 30S is an example of display screen 30. Figure 23C The scenario depicts the handling operation that accepts the selection of the second target parking position P20.

[0562] When the control unit 11D receives a steering operation to select the second target parking position P20, it will highlight the selected second target parking position P20 among the first target parking position P10 and the second target parking position P20 displayed by the display device 22.

[0563] Emphasis display refers to displaying information in a way that draws the passenger's attention. Figure 23B As an example, the control unit 11D emphasizes the selected first target parking position P10 by displaying the first frame F1a of the selected first target parking position P10 with a frame that is thicker than the first frame F1b of the second target parking position P20. Figure 23C As an example, the control unit 11D emphasizes the selected second target parking position P20 by displaying the first frame F1b of the selected second target parking position P20 with a frame that is thicker than the first frame F1a of the first target parking position P10.

[0564] Furthermore, the control unit 11D can display the selected target parking position P1 in a more attention-grabbing display method by using more attention-grabbing colors, flashing, or animated image movement, and is not limited to displaying the selected target parking position P1 with a thicker frame.

[0565] Therefore, by having the passenger operate the steering wheel 20A in either the left or right direction, it is easy to select either the first target parking position P10 or the second target parking position P20. Furthermore, the control unit 11D can easily confirm with the passenger which target parking position P10 has been selected.

[0566] Additionally, display screens 30R and 30S contain information M14. Information M14 prompts for braking and indicates that the target parking position P1 can be changed via steering wheel operation 20A. Figure 23B and Figure 23C In the example shown, information M14 is a way of displaying these information characters, but the way this information is displayed is not limited and can also be a still image, animated image, icon, etc.

[0567] When the control unit 11 receives an operation to select the first target parking position P10 and the braking operation unit (brake pedal) receives a braking operation to reduce braking, it generates a reverse path R1 for parking from the reversing stop position TP to the selected first target parking position P10 by autonomous driving, similar to the embodiment described above. Then, the control unit 11D controls at least the steering wheel to reverse the vehicle 1D and drives the vehicle 1D to park at the first target parking position P10 by autonomous driving. During this autonomous driving, the control unit 11D may also control at least the steering wheel and acceleration / deceleration to reverse the vehicle 1D and drive the vehicle 1D to park at the first target parking position P10 by autonomous driving.

[0568] On the other hand, when the control unit 11D receives the operation of selecting the second target parking position P20 and the braking operation unit (brake pedal) receives the braking operation to reduce, it generates a reverse path R1 for parking from the reversing stop position TP to the selected second target parking position P20 by autonomous driving, just like in the above embodiment.

[0569] Then, the control unit 11D controls at least the steering wheel to reverse the vehicle 1D, and autonomously drives the vehicle 1D to park at the second target parking position P20. During this autonomous driving, the control unit 11D can also control at least the steering wheel and acceleration / deceleration to reverse the vehicle 1D, and autonomously drive the vehicle 1D to park at the second target parking position P20. During autonomous driving, the vehicle 1D is automatically controlled by the vehicle control unit 10D without the need for manual driving operations by the passenger.

[0570] Preferably, when vehicle 1D is parked at the first target parking position P10 or the second target parking position P20, the control unit 11D causes the display device 22 to display information indicating that parking is complete. Similarly to the above mode, the control unit 11D, for example, causes... Figure 21C The displayed screen is shown in 30P.

[0571] Furthermore, the control unit 11D can also accept the selection of the first target parking position P10 or the second target parking position P20 even after the brake operation unit (brake pedal) has eased the braking operation, even after the brake operation has been eased. That is, if the passenger performs the steering wheel 20A operation to select the first target parking position P10 or the second target parking position P20, the target parking position P1 that has been selected once can be changed even after the operation of the brake operation unit (brake pedal) has been eased.

[0572] Next, an example of the information processing flow performed by the control unit 11D in this embodiment will be described.

[0573] Figure 24 This is a flowchart illustrating an example of the information processing flow performed by the control unit 11D. Figure 24 The diagram illustrates the parking assistance process when the control unit 11D detects a target parking position P1.

[0574] Vehicle 1D begins to move forward according to the driver's instructions (step S400). During the process of vehicle 1D moving forward according to the driver's instructions received by the operating device 20, the operating device 20 receives a parking assist start instruction (step S402). When the parking assist start instruction is received, the control unit 11D performs the processing of steps S404 to S422.

[0575] The control unit 11D detects candidate target parking positions SP for which the vehicle 1D can park based on the surrounding image V acquired by the camera 16 (step S404). Then, the control unit 11D detects the target parking position SP that is suitable for the current driving location DP of the vehicle 1D from the multiple candidate target parking positions SP detected in step S404 as the target parking position P1 corresponding to the current driving location DP (step S406).

[0576] Then, the control unit 11D detects the U-turn reference position P2 corresponding to the target parking position P1 detected in step S506 (step S408). In addition, the control unit 11D can also further detect the orientation of the vehicle 1D at the U-turn reference position P2.

[0577] Then, the control unit 11D displays the target parking position P1 detected in step S408 and the corresponding U-turn reference position P2 on the display device 22 in a manner superimposed on the surrounding image V (step S410). That is, while the vehicle 1D is moving forward according to the operation received by the operation device 20, the control unit 11D displays the target parking position P1 detected by the surrounding image V acquired by the camera 16 and the corresponding U-turn reference position P2 at the current driving location DP of the vehicle 1D on the display device 22 in a manner superimposed on the surrounding image V.

[0578] Next, the control unit 11D determines whether the operation to select the target parking position P1 displayed on the display device 22 in step 410 has been accepted (step S412). The control unit 11D performs the determination in step S412 by identifying whether the passenger has performed a steering operation on the steering wheel (steering control unit, steering wheel) 20A of the operating device 20 in the direction of the U-turn reference position P2.

[0579] For example, if the control unit 11D determines that the operation of selecting the target parking position P1 has been accepted when the steering angle of the steering device is adjusted by the passenger's operation of the steering operation unit and the steering angle detected by the steering angle sensor of the steering device is a specific angle (e.g., 30 degrees or more) deviating towards the U-turn reference position P2.

[0580] When the operating device 20 accepts the operation of selecting the U-turn reference position P2, the control unit 11D determines that the target parking position P1 corresponding to the currently displayed U-turn reference position P2 has been selected.

[0581] If a negative judgment is made in step S412 (step S412: "No"), return to step S404 above. If a positive judgment is made in step S412 (step S412: "Yes"), proceed to step S414.

[0582] When the vehicle 1D stops, the control unit 11D sets the stopping position to the reversing stop position TP (step S414). When the forward and reverse operation unit (gear lever) of the operating device 20 receives the reverse instruction operation (step S416) and the braking operation unit (brake pedal) of the operating device 20 receives the braking operation to reduce the braking (step S418), the control unit 11D generates a reverse path R1 from the reversing stop position TP to the selected target parking position P1 (step S420).

[0583] Additionally, the control unit 11D displays the target parking position P1 selected in step S412 on the display device 22 in a manner that overlays the surrounding image V (step S422).

[0584] Then, the control unit 11D controls the steering to reverse the vehicle 1D from the reversing stop position TP to the target parking position P1, and autonomously drives the vehicle 1D to park at the target parking position P1 (step S424). Then, the routine ends.

[0585] Next, the parking assistance process when the control unit 11D detects two target parking positions P1 will be explained.

[0586] Figure 24 This is a flowchart illustrating an example of the information processing flow performed by the control unit 11D. Figure 25 The diagram illustrates the parking assistance process when the control unit 11D detects two target parking positions P1.

[0587] Vehicle 1F begins to move forward according to the driver's instructions (step S500). While vehicle 1D is moving forward according to the driver's instructions received by operating device 20, operating device 20 receives a parking assist start instruction (step S502). When the parking assist start instruction is received, control unit 11D performs the processing of steps S504 to S526.

[0588] The control unit 11D detects candidate target parking positions SP for which the vehicle 1F can park based on the surrounding image V acquired by the camera 16 (step S504). Then, the control unit 11D detects two target parking position candidate SPs that are suitable for the current driving location DP of the vehicle 1D from the multiple target parking position candidate SPs detected in step S504 as target parking positions P1 (first target parking position P10, second target parking position P20) corresponding to the current driving location DP (step S506).

[0589] Then, the control unit 11F detects the U-turn reference position P2 corresponding to the two target parking positions P1 detected in step S506 (step S508). In addition, the control unit 11D can further detect the orientation of the vehicle 1F at the U-turn reference position P2.

[0590] Then, the control unit 11D displays the two target parking positions P1 (first target parking position P10, second target parking position P20) detected in step S508 and the turning reference position P2 corresponding to the target parking position P1 on the display device 22 in a manner superimposed on the surrounding image V (step S510).

[0591] Next, the control unit 11D determines whether the operation of selecting the two target parking positions P1 displayed on the display device 22 in step 510 has been accepted (step S512). The control unit 11D performs the determination in step S512 by identifying whether the passenger has performed a steering operation on the steering wheel (steering control unit, steering wheel) 20A of the operating device 20 in the direction of turning reference position P2.

[0592] For example, if the control unit 11D determines that the operation of selecting the target parking position P1 has been accepted when the steering angle of the steering device is adjusted by the passenger's operation of the steering operation unit and the steering angle detected by the steering angle sensor of the steering device is a specific angle (e.g., 30 degrees or more) deviating towards the U-turn reference position P2.

[0593] When the operating device 20 accepts the operation of selecting the U-turn reference position P2, the control unit 11D determines that two target parking positions P1 (first target parking position P10 and second target parking position P2) corresponding to the currently displayed U-turn reference position P2 have been selected.

[0594] If a negative judgment is made in step S512 (step S512: "No"), return to step S504 above. If a positive judgment is made in step S512 (step S512: "Yes"), proceed to step S514.

[0595] When the vehicle 1D stops, the control unit 11D sets the stop position to the reversing stop position TP (step S514). When the forward and reverse operation unit (gear lever) of the operating device 20 receives the reverse instruction operation (step S516), it proceeds to step S518.

[0596] In step S518, the control unit 11D accepts the selection of a target parking position P1 based on the passenger's steering operation on the steering wheel (steering control unit, steering wheel) 20A included in the operating device 20 (step S518). The control unit 11D highlights the target parking position P1 selected in step S518 (first target parking position P10 or second target parking position P20) (step S520).

[0597] Next, when the braking operation unit (brake pedal) of the operating device 20 receives the braking operation to reduce (step S522), the control unit 11D generates a reverse path R1 from the reversing stop position TP to the selected target parking position P1 (first target parking position P10 or second target parking position P20) (step S524).

[0598] Additionally, the control unit 11D displays the target parking position P1 selected in step S518 on the display device 22 in a manner that overlays the surrounding image V (step S526).

[0599] Then, the control unit 11D controls the steering to cause the vehicle 1D to reverse from the reversing stop position TP along the reverse path R1 generated in step S524 towards the target parking position P1, and autonomously drives the vehicle 1D to park at the target parking position P1 (first target parking position P10 or second target parking position P20) selected in step S518 (step S528). Then, the routine ends.

[0600] As described above, the vehicle control device 10D of this embodiment is mounted on a vehicle 1D, which includes: an operating device 20 for receiving operations from passengers, an operating device 20 having at least a forward / reverse operating unit (gear lever 20C) and a braking operating unit (brake pedal), a camera 16 for acquiring surrounding images V, a display device 22 that passengers can visually confirm, and a movement control device 12 for at least controlling steering.

[0601] When the vehicle 1D is moving forward according to the operation received by the operation device 20, and the operation device 20 receives a parking assist start instruction, the vehicle control device 10D detects the target parking position P1 of the vehicle 1D based on the surrounding image V acquired by the camera 16, and displays the target parking position P1 on the display device 22 in a manner superimposed on the surrounding image V. When the vehicle 1D stops, and the forward / reverse operation unit (gear lever 20C) of the operation device 20 receives a reverse instruction operation, and the braking operation unit (brake pedal) of the operation device 20 receives a braking operation to reduce braking, the vehicle control device 10D at least controls the steering to reverse the vehicle 1D, and then autonomously drives the vehicle 1D to park at the target parking position P1.

[0602] In this regard, there are situations where it is difficult to provide appropriate parking assistance in the existing technology.

[0603] On the other hand, in this embodiment, when the vehicle 1D is moving forward according to the operation received by the operation device 20 and the operation device 20 receives a parking assist start instruction, the vehicle control device 10D detects the target parking position P1 of the vehicle 1D based on the surrounding image V acquired by the camera 16, and displays the target parking position P1 on the display device 22 in a manner superimposed on the surrounding image V. When the vehicle 1D stops, and the forward / reverse operation unit (gear lever 20C) of the operation device 20 receives a reverse instruction operation, and the braking operation unit (brake pedal) of the operation device 20 receives a braking operation to reduce braking, the vehicle control device 10D at least controls the steering to reverse the vehicle 1D, and then autonomously drives the vehicle 1D to park at the target parking position P1.

[0604] Thus, when the vehicle control device 10D of this embodiment receives a parking assist start instruction, it overlays the target parking position P1 detected by the surrounding image V acquired by the camera 16 of the vehicle 1D which is moving forward according to the driver's driving onto the surrounding image V. Then, when the vehicle 1D stops, and the forward / reverse operation unit (gear lever 20C) of the operation device 20 receives a reverse instruction operation, and the braking operation unit (brake pedal) of the operation device 20 receives a braking operation to reduce the braking, the vehicle control device 10D at least controls the steering to reverse the vehicle 1D, and drives the vehicle 1D to park at the target parking position P1 by autonomous driving.

[0605] Therefore, the vehicle control device 10D of this embodiment can provide appropriate assistance for autonomous parking corresponding to the detected target parking position P1.

[0606] Therefore, the vehicle control device 10D of this embodiment can provide appropriate parking assistance.

[0607] Furthermore, in this embodiment, the following method is described as an example: when the vehicle control device 10D is moving forward according to the operation device 20's operation, i.e., when the operation device 20 receives a parking assist start instruction while the vehicle 1D is moving forward according to the operation device 20's operation, the vehicle control device 10D detects the target parking position P1 and performs the above-described processing. However, when the operation device 20 receives a parking assist start instruction while the vehicle 1D is moving autonomously, the vehicle control device 10D may also detect the target parking position P1 in the same way as described above, and then perform the above-described parking assist control.

[0608] (Fifth Implementation)

[0609] In this embodiment, the following method will be described: During the forward movement of vehicle 1, the target parking position candidate SP that exists on the right and left sides relative to the direction of travel, and that exists on the side closest to the driving location DP of vehicle 1, is set as target parking position P1.

[0610] Figure 1 This is a block diagram illustrating an example of the overall structure of the vehicle 1E according to this embodiment.

[0611] Vehicle 1E is identical to vehicle 1 in the above-described embodiment, except that it has vehicle control device 10E instead of vehicle control device 10. Therefore, the configuration of sensor 14 and camera 16 in vehicle 1E is also the same as in vehicle 1 (see [reference]). Figure 2 Furthermore, the exterior of vehicle 1E and the structure near the driver's seat 24A are the same as those of vehicle 1 (see reference). Figure 3 , Figure 4 ).

[0612] The vehicle control unit 10E is an electronic control unit that uniformly controls all parts of the vehicle 1E. Similar to the vehicle control unit 10, the vehicle control unit 10E uses sensor information and surrounding images V received from the sensor 14 and camera 16, respectively, to control the motion control unit 12 to optimize the driving state of the vehicle 1E. Furthermore, the vehicle control unit 10E controls the motion control unit 12 to enable the vehicle 1E to drive autonomously.

[0613] The vehicle control device 10E includes a control unit 11E. Part or all of the control unit 11E may be a software structure implemented through the cooperation of a processor and various programs stored in a memory. Alternatively, part or all of the control unit 11E may also be a hardware structure implemented by dedicated circuitry or the like.

[0614] The control unit 11CE provides unified control over all parts of vehicle 1E.

[0615] In this embodiment, the control unit 11E performs information processing for parking assistance of the vehicle 1E. In this embodiment, similar to the embodiments described above, the following approach is envisioned and described: after the vehicle 1E is driven forward by the passenger until at least a portion of the vehicle body has passed at least a portion of the target parking position P1, the vehicle 1E is stopped by the passenger, the vehicle 1E is reversed from the reversing stop position TP (which is the stopping position), and the vehicle 1E moves towards the target parking position P1 by autonomous driving.

[0616] Figure 26A This is an explanatory diagram of an example of parking assistance in this embodiment.

[0617] During the process of the control unit 11E moving forward in the vehicle 1E according to the operation device 20 received by the passenger (see reference) Figure 26A (In the direction of arrow D in the image), based on the surrounding image V acquired by camera 16, one or more target parking position candidates SP for which vehicle 1E can park are detected. Hereinafter, the direction of forward travel of vehicle 1E, i.e., the direction of arrow D, will sometimes be referred to as the driving direction D.

[0618] The definition of the target parking position candidate SP is the same as in the above implementation method. That is, the target parking position candidate SP refers to the candidate target parking position P1 at each travel location DP that is estimated to be able to park for the vehicle 1E, which is traveling at a time-varying travel location DP according to the operation of the passenger, i.e., the driver's operation. The current travel location DP of the vehicle 1E is represented by the current position of the vehicle 1E. The current travel location DP of the vehicle 1E is an example of a defined travel location DP.

[0619] Based on the surrounding images V acquired by the camera 16 at each driving location DP, the control unit 11E analyzes the surrounding images V using a known method, thereby detecting target parking position candidates SP at the right RSR and left RSL relative to the driving direction D of the vehicle 1. Then, the control unit 11E sets the target parking position candidate SP that is located on the side closer to the driving location DP from the target parking position candidate SP that exists at the right RSR and left RSL relative to the driving direction D as the target parking position P1.

[0620] exist Figure 26A As an example, the following situation is shown: At the driving location DP, the control unit 11E detects a first target parking position candidate SP1 on the left RSL facing the driving direction D, and a second target parking position candidate SP2 on the right RSR facing the driving direction D. The first target parking position candidate SP1 and the second target parking position candidate SP2 are examples of target parking position candidates SP.

[0621] Left RSL refers to the space in the actual space RS that is to the left of the virtual driving line DL of vehicle 1E traveling along the driving direction D at driving location DP. The virtual driving line DL is a line passing through the driving location DP of vehicle 1E along the driving direction D. In other words, left RSL refers to the space to the right of the virtual driving line DL of vehicle 1E traveling along the virtual driving line DL at driving location DP, with driving location DP as the origin and the downstream side of the driving direction D as the frontal direction.

[0622] The right RSR refers to the space in the actual space RS that is to the right of the virtual driving line DL of the vehicle 1E traveling along the driving direction D at the driving location DP. In other words, the right RSR refers to the space to the left of the virtual driving line DL of the vehicle 1E traveling along the virtual driving line DL in the driving direction D at the driving location DP, with the driving location DP as the origin and the downstream side of the driving direction D as the front direction.

[0623] In addition, Figure 26A As an example, the following method is shown: the distance Y2 between the first target parking position candidate SP1 and the virtual driving line DL is less than the distance Y1 between the second target parking position candidate SP2 and the virtual driving line DL.

[0624] Distance Y2 is the shortest distance between each of the first target parking position candidate SP1 and the third target parking position candidate SP3 (described later) and the virtual driving line DL. Distance Y1 is the shortest distance between the second target parking position candidate SP2 and the virtual driving line DL.

[0625] At the current driving location DP of vehicle 1E, if a first target parking position candidate SP1 exists on the left RSL facing the driving direction D, and a second target parking position candidate SP2 exists on the right RSR facing the driving direction D, and the first target parking position candidate SP1 is closer to the driving location DP than the second target parking position candidate SP2, then the control unit 11E detects the first target parking position candidate SP1 as the target parking position P1. The control unit 11E analyzes the surrounding image V and sensor information acquired by sensor 14 using known methods, thereby deriving the distances between the current driving location DP of vehicle 1E and each of the target parking position candidates, including the second target parking position candidate SP2 and the first target parking position candidate SP1.

[0626] As mentioned above, in Figure 26A In the example shown, the distance Y2 between the first target parking position candidate SP1 and the virtual driving line DL is less than the distance Y1 between the second target parking position candidate SP2 and the virtual driving line DL. Specifically, for example, if distance Y1 is greater than a specified distance (e.g., 2.5m), distance Y2 is less than that specified distance. Therefore, in Figure 26A In the example shown, the distance between the current driving location DP of vehicle 1E and the first target parking position candidate SP1 is less than the distance between the current driving location DP of vehicle 1E and the second target parking position candidate SP2. In this case, the control unit 11E detects the first target parking position candidate SP1, which is closer to the left RSL of vehicle 1E's current driving location DP, as the target parking position P1.

[0627] Alternatively, the control unit 11E may detect the first target parking position candidate SP1 as the target parking position P1 at the driving location DP, where there is a first target parking position candidate SP1 on the left RSL facing the driving direction D, and a second target parking position candidate SP2 on the right RSR facing the driving direction D, and the first target parking position candidate SP1 and the second target parking position candidate SP2 are within a specified distance based on the current driving location DP of the vehicle 1E, and the first target parking position candidate SP1 is closer to the driving location DP than the second target parking position candidate SP2.

[0628] The first target parking position candidate SP1 and the second target parking position candidate SP2 being within a specified distance based on the current driving location DP of vehicle 1E means that the shortest distance from the driving location DP to each of the first target parking position candidate SP1 and the second target parking position candidate SP2 is less than or equal to the specified distance. Alternatively, the first target parking position candidate SP1 and the second target parking position candidate SP2 being within a specified distance based on the current driving location DP of vehicle 1E can also mean that the shortest distance (distance Y1, distance Y2) from each of the first target parking position candidate SP1 and the second target parking position candidate SP2 to the virtual driving line DL is less than or equal to the specified distance.

[0629] The specified distance can be predetermined. This specified distance is, for example, 2.5m, but it is not limited to this distance.

[0630] Figure 26B This is an explanatory diagram of an example of parking assistance in this embodiment.

[0631] exist Figure 26B As an example, the following situation is shown: at the driving location DP, the control unit 11E detects the first target parking position candidate SP1 and the third target parking position candidate SP3 on the left RSL facing the driving direction D, and detects the second target parking position candidate SP2 on the right RSR facing the driving direction D.

[0632] The third target parking position candidate SP3 is an example of a target parking position candidate SP. The third target parking position candidate SP3 is a target parking position candidate SP located further away from the driving location DP than the first target parking position candidate SP1.

[0633] In addition, Figure 26A As an example, the following arrangement is shown: the distance Y2 between each of the first target parking position candidate SP1 and the third target parking position candidate SP3 and the virtual driving line DL is less than the distance Y1 between the second target parking position candidate SP2 and the virtual driving line DL.

[0634] At the driving location DP, if there is a first target parking position candidate SP1 on the left RSL facing the driving direction D, and a third target parking position candidate SP3 that is farther from the current driving location DP than the first target parking position candidate SP1, and a second target parking position candidate SP2 on the right RSR facing the driving direction D, and the first target parking position candidate SP1 and the third target parking position candidate SP3 are closer to the driving location DP than the second target parking position candidate SP2, then the control unit 11E detects the first target parking position candidate SP1 as the target parking position P1. The control unit 11E analyzes the surrounding image V and sensor information acquired by the sensor 14 using known methods, thereby deriving the distances between the current driving location DP of the vehicle 1E and each of the target parking position candidates, including the second target parking position candidate SP2, the third target parking position candidate SP3, and the first target parking position candidate SP1.

[0635] As mentioned above, in Figure 26B In the example shown, the distance Y2 between each of the first target parking position candidate SP1 and the third target parking position candidate SP3 and the virtual driving line DL is less than the distance Y1 between the second target parking position candidate SP2 and the virtual driving line DL. Therefore, in Figure 26B In the example shown, the distance between the current driving location DP of vehicle 1E and the first target parking position candidate SP1 is less than the distance between the current driving location DP of vehicle 1E and each of the second and third target parking position candidates SP2 and SP3. In this case, the control unit 11E detects the first target parking position candidate SP1, which is located on the left RSL and is closer to the current driving location DP of vehicle 1E, as the target parking position P1.

[0636] In addition, Figure 26B In the example, the control unit 11E can also detect the first target parking position candidate SP1 as the target parking position P1 if the first target parking position candidate SP1, the second target parking position candidate SP2, and the third target parking position candidate SP3 are within a predetermined distance from the current driving location DP of the vehicle 1E, and if the first target parking position candidate SP1 and the third target parking position candidate SP3 are closer to the driving location DP than the second target parking position candidate SP2. The predetermined distance is the same as described above.

[0637] Figure 27A This is an explanatory diagram of an example of parking assistance in this embodiment.

[0638] exist Figure 27AAs an example, the following situation is shown: At the driving location DP, the control unit 11E detects a fourth target parking position candidate SP4 on the left RSL, facing the driving direction D, and a fifth target parking position candidate SP5 on the right RSR, also facing the driving direction D. The fourth target parking position candidate SP4 and the fifth target parking position candidate SP5 are examples of target parking position candidates SP.

[0639] In addition, Figure 27A As an example, the following method is shown: the distance Y1' between the fifth target parking position candidate SP5 and the virtual driving line DL is less than the distance Y2' between the fourth target parking position candidate SP4 and the virtual driving line DL.

[0640] Distance Y2' is the shortest distance between the fourth target parking position candidate SP4 and the virtual driving line DL. Distance Y1' is the shortest distance between each of the fifth and sixth target parking position candidates SP5 and the virtual driving line DL.

[0641] At the current driving location DP of vehicle 1E, if there is a fourth target parking position candidate SP4 on the left RSL facing the driving direction D, and a fifth target parking position candidate SP5 on the right RSR facing the driving direction D, and the fifth target parking position candidate SP5 is closer to the driving location DP than the fourth target parking position candidate SP4, then the control unit 11E detects the fifth target parking position candidate SP5 as the target parking position P1. The control unit 11E analyzes the surrounding image V and sensor information acquired by sensor 14 using known methods, thereby deriving the distances between the current driving location DP of vehicle 1E and each of the fourth and fifth target parking position candidates SP4 and SP5.

[0642] As mentioned above, in Figure 27A In the example shown, the distance Y1' between the fifth target parking position candidate SP5 and the virtual driving line DL is less than the distance Y2' between the fourth target parking position candidate SP4 and the virtual driving line DL. Therefore, in Figure 27A In the example shown, the distance between the current driving location DP of vehicle 1E and the fifth target parking position candidate SP5 is less than the distance between the current driving location DP of vehicle 1E and the fourth target parking position candidate SP4. In this case, the control unit 11E detects the fifth target parking position candidate SP5, which is closer to the current driving location DP of vehicle 1E on the right RSR, as the target parking position P1.

[0643] Additionally, the control unit 11E may also detect the fifth target parking position candidate SP5 as the target parking position P1 at the driving location DP, where there is a fourth target parking position candidate SP4 on the left RSL facing the driving direction D, and a fifth target parking position candidate SP5 on the right RSR facing the driving direction D, and the fourth target parking position candidate SP4 and the fifth target parking position candidate SP5 are within a specified distance based on the current driving location DP of the vehicle 1E, and the fifth target parking position candidate SP5 is closer to the driving location DP than the fifth target parking position candidate SP5.

[0644] The fourth target parking position candidate SP4 and the fifth target parking position candidate SP5 being within a specified distance based on the current driving location DP of vehicle 1E means that the shortest distance from the driving location DP to each of the target parking position candidates in the fourth target parking position candidate SP4 and the fifth target parking position candidate SP5 is less than or equal to the specified distance. Alternatively, the fourth target parking position candidate SP4 and the fifth target parking position candidate SP5 being within a specified distance based on the current driving location DP of vehicle 1E can also mean that the shortest distance (distance Y1', distance Y2') from each of the target parking position candidates in the fourth target parking position candidate SP4 and the fifth target parking position candidate SP5 to the virtual driving line DL is less than or equal to the specified distance.

[0645] The specified distance can be predetermined. Similarly, the specified distance is, for example, 2.5m, but is not limited to this distance.

[0646] Figure 27B This is an explanatory diagram of an example of parking assistance in this embodiment.

[0647] exist Figure 27B As an example, the following situation is shown: at the driving location DP, the control unit 11E detects the fourth target parking position candidate SP4 on the left RSL facing the driving direction D, and detects the fifth target parking position candidate SP5 and the sixth target parking position candidate SP6 on the right RSR facing the driving direction D.

[0648] The sixth target parking position candidate SP6 is an example of a target parking position candidate SP. The sixth target parking position candidate SP6 is a target parking position candidate SP located further away from the driving location DP than the fifth target parking position candidate SP5.

[0649] In addition, Figure 27AAs an example, the following arrangement is shown: the distance Y1' between each of the target parking position candidates SP5 and SP6 and the virtual driving line DL is less than the distance Y2' between the fourth target parking position candidate SP4 and the virtual driving line DL.

[0650] At the driving location DP, if there is a fourth target parking position candidate SP4 on the left RSL facing the driving direction D, and a fifth target parking position candidate SP5 and a sixth target parking position candidate SP6 on the right RSR facing the driving direction D, and if the fifth and sixth target parking position candidates SP5 and SP6 are closer to the driving location DP than the fourth target parking position candidate SP4, then the control unit 11E detects the fifth target parking position candidate SP5 as the target parking position P1. The control unit 11E analyzes the surrounding image V and sensor information acquired by the sensor 14 using known methods, thereby deriving the distances between the vehicle 1E's current driving location DP and each of the fourth, fifth, and sixth target parking position candidates SP4, SP5, and SP6.

[0651] As mentioned above, in Figure 27B In the example shown, the distance Y1' between each of the sixth target parking position candidate SP6 and the fifth target parking position candidate SP5 and the virtual driving line DL is less than the distance Y2' between the fourth target parking position candidate SP4 and the virtual driving line DL. Therefore, in Figure 27B In the example shown, the distance between the current driving location DP of vehicle 1E and the fifth target parking position candidate SP5 is less than the distance between the current driving location DP of vehicle 1E and each of the sixth target parking position candidate SP6 and the fourth target parking position candidate SP4. In this case, the control unit 11E detects the fifth target parking position candidate SP5, which is located on the right RSR and is closer to the current driving location DP of vehicle 1E, as the target parking position P1.

[0652] In addition, Figure 27B In the example, the control unit 11E can also detect the fifth target parking position candidate SP5 as the target parking position P1 if the fourth target parking position candidate SP4, the fifth target parking position candidate SP5, and the sixth target parking position candidate SP6 are within a specified distance from the current driving location DP of the vehicle 1E, and the fifth target parking position candidate SP5 and the sixth target parking position candidate SP6 are closer to the driving location DP than the fourth target parking position candidate SP4. The specified distance is the same as described above.

[0653] When the control unit 11E detects a target parking position P1 based on the surrounding image V at the current driving location DP, it detects a U-turn reference position P2 corresponding to the detected target parking position P1. Furthermore, the control unit 11E can also further derive the orientation of the vehicle 1CE at the U-turn reference position P2. The method for deriving the U-turn reference position P2 and the orientation of the vehicle 1E at the U-turn reference position P2 is the same as the method for deriving the reversing start position P2 and the orientation of the vehicle 1 at the reversing start position P2 described in the above embodiment.

[0654] Then, the control unit 11E displays the detected target parking position P1 and the corresponding U-turn reference position P2 on the display device 22 in a manner superimposed on the surrounding image V. That is, while the vehicle 1E is moving forward according to the operation received by the operation device 20, the control unit 11E displays the target parking position P1 detected by the surrounding image V acquired by the camera 16 and the corresponding U-turn reference position P2 at the current driving location DP on the display device 22 in a manner superimposed on the surrounding image V.

[0655] For example, the control unit 11E uses a first frame to represent the target parking position P1 that the display device 22 will display. Additionally, the control unit 11E uses a second frame to represent the U-turn reference position P2. Then, the control unit 11E displays the first frame representing the target parking position P1 and the second frame representing the U-turn reference position P2 on the display device 22 in a manner superimposed on the surrounding image V. Therefore, when the display device 22 displays, for example... Figure 12B The displayed screen is 30J.

[0656] As described above, display screen 30J is a display screen 30 that overlays the first frame F11 representing the target parking position P1 and the second frame, namely the first U-turn reference position object P21OB, representing the U-turn reference position P2 onto the surrounding image V.

[0657] In addition, the control unit 11E can also perform the following controls: Figure 26A In the following state, that is, when there is a first target parking position candidate SP1 on the left RSL and a second target parking position candidate SP2 on the right RSR, and both the first target parking position candidate SP1 and the second target parking position candidate SP2 are more than the distance specified above from the driving location DP, the display device 22 will not display the target parking position P1 determined based on these first target parking position candidates SP1 and second target parking position candidates SP2, and the U-turn reference position P2 corresponding to the target parking position P1.

[0658] When the control unit 11E displays the target parking position P1 detected at the current driving location DP and the U-turn reference position P2 corresponding to the target parking position P1 on the display device 22 in a manner superimposed on the surrounding image V during the forward movement of the vehicle 1E, it determines whether the operation device 20 has accepted the operation of selecting the target parking position P1.

[0659] The operation of the operating device 20 to select the target parking position P1 is a steering operation performed by the passenger on the steering control parts of the operating device 20, such as the steering wheel (driving wheel), in the direction of the turning reference position P2, when the target parking position P1 and the turning reference position P2 are displayed on the display device 22.

[0660] For example, if the control unit 11E adjusts the steering angle of the steering device by the passenger's operation of the steering operation unit, and the steering angle detected by the steering angle sensor that detects the steering angle of the steering device is a specific angle (e.g., 30 degrees or more) deviating towards the U-turn reference position P2, it can determine that the operation of selecting the target parking position P1 has been accepted.

[0661] When the operating device 20 receives an operation to select a target parking position P1, the control unit 11E can, in the same manner as in the above embodiment, at least fix the U-turn reference position P2 on the display device 22. Furthermore, when the operating device 20 receives an operation to select a target parking position P1, the control unit 11E can also fix the U-turn reference position P2 and the corresponding target parking position P1 on the display device 22.

[0662] When the operating device 20 accepts the operation of selecting the target parking position P1 and the vehicle 1E has stopped at the reversing stop position TP corresponding to the target parking position P1, the control unit 11E controls at least acceleration, deceleration and steering to make the vehicle 1E reverse, and drives the vehicle 1E to park from the reversing stop position TP to the target parking position P1 through autonomous driving.

[0663] In detail, when the operating device 20 receives an operation to select a target parking position P1, the vehicle 1E moves forward according to the driver's direction. Then, after the vehicle 1E has moved forward according to the driver's direction, the control unit 11E determines whether it has stopped at the reversing stop position TP. The stopping of the vehicle 1E at the reversing stop position TP is performed by the operation received by the operating device 20. That is, the vehicle 1E stops at the reversing stop position TP according to the braking operation performed by the driver.

[0664] As described in the above embodiments, the reversing stop position TP refers to the position of vehicle 1E in the actual space RS when it stops after the control unit 11E has derived the target parking position P1 and the U-turn reference position P2. In this embodiment, the reversing stop position TP is the position of vehicle 1E in the actual space RS when the control unit 11E detects the target parking position P1 and the U-turn reference position P2, accepts the operation of selecting the target parking position P1, and the vehicle 1E moves forward and stops according to the driver's driving. Alternatively, similar to the above embodiments, the reversing stop position TP can also be the position of vehicle 1E in the actual space RS when the control unit 11E detects the target parking position P1 and the U-turn reference position P2, accepts the operation of selecting the target parking position P1, and the vehicle 1E moves forward and stops according to the driver's driving, and the gear of vehicle 1E is shifted to reverse (R).

[0665] Similar to the above embodiments, the reversing stop position TP and the U-turn reference position P2 may or may not be the same. Furthermore, in this embodiment, it is preferable that the reversing stop position TP and the U-turn reference position P2 are within a predetermined distance.

[0666] When the operating device 20 accepts the operation of selecting the target parking position P1, and the vehicle 1E stops at the reversing stop position TP corresponding to the target parking position P1, and the forward / reverse operation unit accepts the reverse instruction operation, and the braking operation unit (brake pedal) accepts the braking operation to reduce the braking, the control unit 11E controls the steering to make the vehicle 1E reverse, and drives the vehicle 1E to park from the reversing stop position TP to the target parking position P1 through autonomous driving.

[0667] The forward / reverse control unit handles reverse instruction operations, which refers to shifting the gear to reverse by the passenger operating the gear shift lever (forward / reverse control unit) used for changing the gears of the transmission.

[0668] For example, the control unit 11E generates a reverse path using a known method for moving and parking the vehicle 1E from the reversing stop position TP to the target parking position P1 via autonomous driving. Then, when the forward / reverse operation unit receives a reverse instruction operation and the braking operation unit (brake pedal) receives a braking operation to reduce braking, the control unit 11E controls the steering based on the reverse path to reverse the vehicle 1E and parks the vehicle 1E towards the target parking position P1 via autonomous driving. In detail, the control unit 11E compares the current position of the vehicle 1E in the actual space RS with the reverse path, controls the steering to make the current position of the vehicle 1E a position on the reverse path, and controls the movement control device 12 to drive along the reverse path. During autonomous driving, the vehicle 1E is automatically controlled by the vehicle control device 10E without the need for manual driving operations by the passenger.

[0669] Next, an example of the information processing flow performed by the control unit 11E in this embodiment will be described.

[0670] Figure 28 This is a flowchart illustrating an example of the information processing flow performed by the control unit 11E.

[0671] The vehicle 1E begins to move forward according to the driver's instructions (step S500). During the process of the vehicle 1E moving forward according to the driver's instructions received by the operating device 20, the operating device 20 receives a parking assist start instruction (step S502). When the parking assist start instruction is received, the control unit 11E performs the processing of steps S504 to S526.

[0672] The control unit 11E detects a target parking position candidate SP for which the vehicle 1E can park based on the surrounding image V acquired by the camera 16 (step S504). Then, the control unit 11E detects one of the multiple target parking position candidate SPs detected in step S504 that corresponds to the current driving location DP of the vehicle 1E as the target parking position P1 corresponding to the current driving location DP (step S506).

[0673] Then, the control unit 11E detects the U-turn reference position P2 corresponding to the target parking position P1 detected in step S506 (step S508). In addition, the control unit 11E may further detect the orientation of the vehicle 1E at the U-turn reference position P2.

[0674] Then, the control unit 11E displays the target parking position P1 detected in step S508 and the corresponding U-turn reference position P2 on the display device 22 in a manner superimposed on the surrounding image V (step S510). That is, while the vehicle 1E is moving forward according to the operation received by the operation device 20, the control unit 11E displays the target parking position P1 detected by the surrounding image V acquired by the camera 16 and the corresponding U-turn reference position P2 at the current driving location DP of the vehicle 1E on the display device 22 in a manner superimposed on the surrounding image V.

[0675] Next, the control unit 11E determines whether the operation to select the target parking position P1 displayed on the display device 22 in step S510 has been accepted (step S512). The control unit 11E performs the determination in step S512 by determining whether the passenger has performed a steering operation on the steering wheel (steering control unit, steering wheel) 20A of the operating device 20 in the direction of the U-turn reference position P2.

[0676] For example, if the control unit 11E adjusts the steering angle of the steering device by the passenger's operation of the steering operation unit, and the steering angle detected by the steering angle sensor that detects the steering angle of the steering device is a specific angle (e.g., 30 degrees or more) deviating towards the U-turn reference position P2, it can determine that the operation of selecting the target parking position P1 that is being displayed has been accepted.

[0677] When the operating device 20 accepts the operation of selecting the U-turn reference position P2, the control unit 11E determines that the target parking position P1 corresponding to the currently displayed U-turn reference position P2 has been selected.

[0678] When a negative judgment is made in step S512 (step S512: "No"), proceed to step S514. In step S514, the control unit 11E determines whether the next candidate detection condition is met (step S514).

[0679] The control unit 11E determines step S514 by identifying whether the current position of the vehicle 1E (current driving location DP) has passed the target parking position P1 detected in the previous step S506.

[0680] The control unit 11E determines whether the vehicle 1E has passed the target parking position P1 detected in the previous step S506 by determining whether the driver's line of sight (e.g., the center of the wheelbase) of the vehicle 1E exceeds the downstream part of the vehicle 1E's forward driving direction in the target parking position P1.

[0681] If a negative judgment is made in step S514 (step S514: "No"), return to step S512. If a positive judgment is made in step S514 (step S514: "Yes"), return to step S504 above.

[0682] On the other hand, when a positive judgment is made in step S512 (step S512: "Yes"), proceed to step S516.

[0683] In step S516, the control unit 11E repeatedly performs negative judgments (step S516: "No") until it determines that vehicle 1E has stopped (step S516: "Yes"). The control unit 11E, for example, analyzes the sensor information acquired by sensor 14 using a known method, thereby determining that vehicle 1E has stopped when its speed is below a predetermined speed (e.g., 0 km / h). When a positive judgment is made in step S516 (step S516: "Yes"), the control unit 11E sets the stopping position to the reversing stop position TP and proceeds to step S518.

[0684] When the forward / backward operation unit (gear lever) of the operating device 20 receives a backward instruction operation (step S518) and the braking operation unit (brake pedal) of the operating device 20 receives a braking operation to reduce braking (step S520), the control unit 11E generates a backward path from the reversing stop position TP to the selected target parking position P1 (step S522).

[0685] Additionally, the control unit 11E displays the target parking position P1 selected in step S512 on the display device 22 in a manner that overlays the surrounding image V (step S524).

[0686] Then, the control unit 11E controls the steering to reverse the vehicle 1E from the reversing stop position TP to the target parking position P1, and autonomously drives the vehicle 1E to park at the target parking position P1 (step S526). Then, this routine ends.

[0687] As explained above, the vehicle control device 10E of this embodiment is mounted on a vehicle 1E, which includes an operating device 20 for receiving passenger operations, a camera 16 for acquiring surrounding images V, a display device 22 for visual confirmation by the passenger, and a motion control device 12 for controlling at least steering. While the vehicle 1C is moving forward according to the operation received by the operating device 20, the vehicle control device 10E displays the target parking position P1 detected by the surrounding images V acquired by the camera 16 at the designated (current) driving location DP, and the turning reference position P2 corresponding to the target parking position P1, superimposed on the surrounding images V, on the display device 22. When the operating device 20 receives an operation to select the target parking position P1, and the vehicle 1E has stopped at the turning stop position TP, the vehicle control device 10E causes the motion control device 12 to control at least acceleration / deceleration and steering, and autonomously drives the vehicle 1E from the turning stop position TP to the target parking position P1.

[0688] Furthermore, in this embodiment, the vehicle control device 10E sets the first target parking position candidate SP1 as the target parking position P1 when there is a first target parking position candidate SP1 on the left RSL facing the direction of travel D, and a second target parking position candidate SP2 on the right RSR facing the direction of travel D, and the first target parking position candidate SP1 is closer to the specified (current) driving location DP than the second target parking position candidate SP2.

[0689] In the prior art, parking assistance based on autonomous driving is not sufficiently provided, and there are situations where it is difficult to provide appropriate parking assistance.

[0690] On the other hand, in this embodiment, the vehicle control device 10E sets the first target parking position candidate SP1 as the target parking position P1 when there is a first target parking position candidate SP1 on the left RSL facing the direction of travel D, and a second target parking position candidate SP2 on the right RSR facing the direction of travel D, and the first target parking position candidate SP1 is closer to the specified (current) driving location DP than the second target parking position candidate SP2.

[0691] Therefore, the vehicle control device 10E of this embodiment can set a suitable target parking position candidate SP corresponding to the driving location DP of the vehicle 1E as the target parking position P1.

[0692] Therefore, the vehicle control device 10E of this embodiment can provide appropriate parking assistance.

[0693] (Sixth Implementation Method)

[0694] In this embodiment, the following method will be described: During the forward movement of vehicle 1, the target parking position P1 corresponding to the driving location DP of vehicle 1 and the U-turn reference position P2 corresponding to the target parking position P1 are continuously displayed on the display device 22. When the specified conditions are met, the continuous display of the U-turn reference position P2 is stopped.

[0695] Figure 1 This is a block diagram illustrating an example of the overall structure of vehicle 1F according to this embodiment.

[0696] Vehicle 1F is identical to Vehicle 1 in the above-described embodiment, except that it has Vehicle Control Device 10F instead of Vehicle Control Device 10. Therefore, the configuration of Sensor 14 and Camera 16 in Vehicle 1F is also the same as in Vehicle 1 (see [reference]). Figure 2 Furthermore, the exterior of vehicle 1F and the structure near the driver's seat 24A are the same as those of vehicle 1 (see reference). Figure 3 , Figure 4 ).

[0697] The vehicle control unit 10F is an electronic control unit that uniformly controls all parts of the vehicle 1F. Similar to the vehicle control unit 10, the vehicle control unit 10F uses sensor information and surrounding images V received from the sensor 14 and camera 16 respectively to control the motion control unit 12 to optimize the driving state of the vehicle 1F. Furthermore, the vehicle control unit 10F controls the motion control unit 12 to enable the vehicle 1F to drive autonomously.

[0698] The vehicle control unit 10F includes a control unit 11F. Part or all of the control unit 11F may be a software structure implemented through the cooperation of a processor and various programs stored in a memory. Alternatively, part or all of the control unit 11F may be a hardware structure implemented by dedicated circuitry or the like.

[0699] The control unit 11F provides unified control over all parts of vehicle 1F.

[0700] In this embodiment, the control unit 11F performs information processing for parking assistance of the vehicle 1F. In this embodiment, similar to the above embodiment, the following manner is envisioned and described: after the vehicle 1F is driven forward according to the passenger's driving until at least a portion of the vehicle body of the vehicle 1F has passed at least a portion of the area of ​​the target parking position P1, the vehicle 1F is stopped according to the passenger's driving, the vehicle 1F is reversed from the reversing stop position TP which is the stopping position, and the vehicle 1F moves toward the target parking position P1 by autonomous driving.

[0701] Figure 29A This is an explanatory diagram of an example of parking assistance in this embodiment.

[0702] During the process of the control unit 11F moving forward in the vehicle 1F according to the operation device 20 received by the passenger (see reference) Figure 29A The direction of arrow D in the image (hereinafter referred to as the driving direction D) is as follows: When the operating device 20 receives the parking assist start instruction, it detects the target parking position P1 where the vehicle 1F can park based on the surrounding image V acquired by the camera 16.

[0703] The definition of the parking assistance start instruction and the method for accepting the parking assistance start instruction are the same as those described above.

[0704] When the operating device 20 receives a parking assistance start instruction, similarly to the above embodiment, the control unit 11F detects one or more target parking position candidates SP for which the vehicle 1F can park, based on the surrounding images V acquired by the camera 16. The definition and detection method of the target parking position candidates SP are the same as in the above embodiment.

[0705] exist Figure 29A As an example, this illustrates the state where the control unit 11F detects multiple target parking position candidates SP (target parking position candidates SP1a to SP1c) at a certain driving location DP. Similar to the embodiment described above, the control unit 11F detects one target parking position candidate SP as the target parking position P1. That is, the control unit 11F detects one target parking position candidate SP as the target parking position P1 based on the positional relationship between the current position of the vehicle 1F (i.e., the current driving location DP) and the target parking position candidate SP.

[0706] exist Figure 29A As an example, the scenario shown is where the control unit 11F detects the target parking position candidate SPa as the target parking position P1.

[0707] When the control unit 11F detects a target parking position P1, it detects a U-turn reference position P2 corresponding to the detected target parking position P1. Furthermore, the control unit 11F can also further derive the orientation of the vehicle 1F at the U-turn reference position P2. The method for deriving the U-turn reference position P2 and the orientation of the vehicle 1F at the U-turn reference position P2 is the same as the deriving method described in the above embodiment.

[0708] Then, the control unit 11F displays the detected target parking position P1 on the display device 22 in a superimposed manner on the surrounding image V. Additionally, the control unit 11F can also display both the target parking position P1 and the corresponding U-turn reference position P2 on the display device 22 in a superimposed manner on the surrounding image V. Furthermore, the control unit 11F can also display multiple target parking position candidates SP used to select the target parking position P1 on the display device 22 in a superimposed manner on the surrounding image V.

[0709] Figure 29B This is a schematic diagram of an example of display screen 30S. Display screen 30S is an example of display screen 30 displayed on display device 22.

[0710] The control unit 11F causes the display device 22 to display a display screen 30S that superimposes a first frame F1 representing the detected target parking position P1 and a second frame F2 representing the turn reference position P2 corresponding to the target parking position P1 onto the surrounding image V.

[0711] As described in the above embodiment, the first frame F1 is an image representing the target parking position P1. As described in the above embodiment, the second frame F2 is an image representing the U-turn reference position P2. The control unit 11F can display each position in the target parking position P1 and the U-turn reference position P2 as superimposed on the surrounding image V; their display method is not limited to the images representing the frames. For example, their display method could also be a still image, characters, animated images, icons, etc., outside the frames.

[0712] Similarly to the above-described embodiment, for example, the control unit 11F generates a virtual frame VF1 representing the target parking space S1 that surrounds the target parking position P1 in the actual space RS (refer to...). Figure 29A The first box F1 and the virtual box VF2 representing the turning reference position P2 (refer to) Figure 29A That is, the second frame F2, and the first frame F1 and the second frame F2 are superimposed and displayed in the peripheral image V displayed by the display device 22 at their respective corresponding positions (refer to...). Figure 29B ).

[0713] When the control unit 11F receives a parking assist start instruction while the vehicle 1F is moving forward, and displays the detected target parking position P1 and the corresponding U-turn reference position P2 superimposed on the surrounding image V on the display device 22, it determines whether to accept the operation of selecting the target parking position P1.

[0714] Similarly to the above embodiment, the operation of the operating device 20 to select the target parking position P1 is a steering operation performed by the passenger on the steering wheel (steering control unit, steering wheel) 20A of the operating device 20 in the direction of the turning reference position P2, while the target parking position P1 and the turning reference position P2 are displayed on the display device 22.

[0715] Similar to the above embodiment, if the control unit 11F adjusts the steering angle of the steering device by the passenger's operation of the steering operation unit, and the steering angle detected by the steering angle sensor that detects the steering angle of the steering device is a specific angle (e.g., 30 degrees or more) deviating towards the turning reference position P2, it can determine that the operation of selecting the turning reference position P2 has been accepted.

[0716] When the operating device 20 accepts the operation of selecting the U-turn reference position P2, the control unit 11F determines that the target parking position P1 corresponding to the currently displayed U-turn reference position P2 has been selected.

[0717] When vehicle 1F moves forward after accepting the operation to select target parking position P1, the direction of forward movement (driving direction D) of vehicle 1F is tilted toward the U-turn reference position P2.

[0718] When the operating device 20 accepts the operation of selecting the target parking position P1, the control unit 11F keeps the turning reference position P2 continuously displayed on the display device 22.

[0719] The U-turn reference position P2 is continuously displayed on the display device 22, meaning that even if the vehicle 1F continues to move forward and reaches another driving location DP downstream of the current driving location DP in the direction of forward movement, the U-turn reference position P2 is continuously superimposed on the surrounding image V.

[0720] In addition, when the operating device 20 accepts the operation of selecting the target parking position P1, the control unit 11F can also make the target parking position P1 and the turning reference position P2 corresponding to the target parking position P1 continuously displayed on the display device 22 in a manner superimposed on the surrounding image V.

[0721] When the operating device 20 accepts the operation of selecting the target parking position P1, the control unit 11F continuously displays the U-turn reference position P2 and the target parking position P1 on the display device 22. Then, when the vehicle 1F has stopped at the reversing stop position TP, the control unit 11F causes the movement control device 12 to control at least acceleration, deceleration and steering, so that the vehicle 1F reverses and autonomously drives the vehicle 1F from the reversing stop position TP to the target parking position P1.

[0722] In detail, when the operating device 20 accepts the operation of selecting the target parking position P1, the vehicle 1F moves forward according to the driver's direction. That is, the movement from the driving position DP when the parking assist start instruction is accepted to the stop at the reversing stop position TP is carried out through the operation accepted by the operating device 20.

[0723] Then, after the vehicle 1F has moved forward according to the driver's direction, the control unit 11F determines whether it has stopped at the reversing stop position TP. Similarly to the embodiment described above, the vehicle 1F stops at the reversing stop position TP through an operation received by the operating device 20. That is, the vehicle 1F stops at the reversing stop position TP based on the braking operation performed by the driver.

[0724] As described in the above embodiment, the reversing stop position TP refers to the position of vehicle 1F in the actual space RS when vehicle 1F stops after the target parking position P1 and the U-turn reference position P2 are derived by the control unit 11F. Similarly to the above embodiment, the reversing stop position TP and the U-turn reference position P2 may or may not be the same.

[0725] When the operating device 20 accepts the operation of selecting the target parking position P1 and the vehicle 1F has stopped at the reversing stop position TP corresponding to the target parking position P1, the control unit 11F causes the motion control device 12 to control at least acceleration, deceleration and steering, so that the vehicle 1F reverses and drives the vehicle 1F from the reversing stop position TP to the target parking position P1 through autonomous driving.

[0726] In detail, when the operating device 20 accepts the operation of selecting the target parking position P1, and the vehicle 1F has stopped at the reversing stop position TP corresponding to the target parking position P1, the control unit 11F continuously displays the target parking position P1 on the display device 22 in a manner superimposed on the surrounding image V. Furthermore, in this case, the control unit 11F continuously displays the target parking position P1 on the display device 22 in a manner superimposed on the surrounding image V, and stops displaying the U-turn reference position P2. Stopping the display of the U-turn reference position P2 means de-displaying the U-turn reference position P2 superimposed on the surrounding image V, i.e., stopping the display. Moreover, simultaneously with this display control, the control unit 11F instructs the movement control device 12 to control at least acceleration / deceleration and steering, so that the vehicle 1F reverses and autonomously drives the vehicle 1F from the reversing stop position TP to the target parking position P1.

[0727] Similar to the above embodiment, when vehicle 1F stops at the reversing stop position TP, and the forward / reverse operation unit (gear lever) of the operating device 20 receives a reverse instruction operation, and the braking operation unit (brake pedal) of the operating device 20 receives a braking operation to reduce braking, the control unit 11F causes the movement control device 12 to control at least acceleration / deceleration and steering, thereby causing vehicle 1F to reverse and to park vehicle 1F through autonomous driving. The control of parking to the target parking position P1 through autonomous driving is the same as in the above embodiment.

[0728] Here, the following situation exists: depending on the passenger's driving, vehicle 1F continues to move forward without stopping, thus passing the reversing stop position TP and the target parking position P1. Specifically, after the passenger selects the target parking position P1, the passenger may sometimes wish to continue driving or park in another parking space instead of parking at the target parking position P1. In this case, vehicle 1F may sometimes continue to move forward, passing the reversing stop position TP and the target parking position P1, depending on the passenger's driving.

[0729] Therefore, in this embodiment, after the operation device 20 accepts the operation of selecting the target parking position P1 and at least makes the U-turn reference position P2 continuously displayed on the display device 22 in a manner superimposed on the surrounding image V, the control unit 11F stops the continuous display of the U-turn reference position P2 on the display device 22 if the stop condition for the continuous display of the U-turn reference position P2 is met.

[0730] Figure 30 This is an explanatory diagram of an example of parking assistance in this embodiment.

[0731] exist Figure 30 The image shows a state where vehicle 1F, which has been moving forward according to the passenger's operation received by the operating device 20, has passed the target parking position P1 without stopping. Additionally, in... Figure 30 The following scenario is shown as an example: Vehicle 1F passes through or near the U-turn reference position P2 and moves further in the driving direction D in a direction away from the target parking position P1.

[0732] After the operating device 20 accepts the operation of selecting the target parking position P1 and at least makes the U-turn reference position P2 continuously displayed on the display device 22 in a manner superimposed on the surrounding image V, the control unit 11F stops the continuous display of the U-turn reference position P2 on the display device 22 if the stop condition for the continuous display of the U-turn reference position P2 is met.

[0733] Furthermore, when the operating device 20 accepts the operation of selecting the target parking position P1 and at least continuously displays the target parking position P1 and the U-turn reference position P2 on the display device 22 in a manner superimposed on the surrounding image V, if the continuous display stop condition is met after the display, the control unit 11F stops the continuous display of the target parking position P1 and the U-turn reference position P2 on the display device 22.

[0734] The continuously displayed stopping condition refers to the vehicle 1F moving more than a specified distance away from the U-turn reference position P2 in the direction away from the designated travel point RDP. In other words, the continuously displayed stopping condition means that, based on the U-turn reference position P2, the vehicle 1F moves more than a specified distance away from the designated travel point RDP.

[0735] The instruction receiving driving location (RDP) is an example of the driving location (DP). The instruction receiving driving location (RDP) is the driving location (designated driving location DP) of vehicle 1F when the control unit 11F receives the parking assist start instruction.

[0736] The control unit 11F analyzes the surrounding image V and sensor information using known methods to determine whether the vehicle 1F has moved more than a specified distance away from the U-turn reference position P2 in the direction away from the designated driving location RDP.

[0737] The prescribed distance can be predetermined. Alternatively, the prescribed distance can be appropriately changed based on the operating instructions given by the passenger or others to the operating device 20.

[0738] In this case, if vehicle 1F moves more than a predetermined distance away from the designated driving location RDP from the U-turn reference position P2, the control unit 11F stops the continuous display of the target parking position P1 and the U-turn reference position P2 on the display device 22.

[0739] Alternatively, the continuously displayed stopping conditions may include, in addition to the condition that vehicle 1F moves more than a specified distance away from the designated driving location RDP from the U-turn reference position P2, the following conditions.

[0740] In detail, the continuously displayed stopping condition means that vehicle 1F moves more than a specified distance away from the U-turn reference position P2 in a direction away from the designated driving location RDP, and vehicle 1F is located on the side further away from the designated driving location RDP than the first straight line L1.

[0741] The first straight line L1 is a straight line that passes through the U-turn reference position P2 and is orthogonal to the direction of travel (driving direction D) of vehicle 1F. In detail, the first straight line L1 is a line that passes through the U-turn reference position P2 and is orthogonal to the direction of travel (driving direction D) of vehicle 1F at the point in time when it has passed the nearest point of passage of the U-turn reference position P2.

[0742] exist Figure 30 In the example shown, the area in the actual space RS where vehicle 1F is further away from the indicated destination RDP than the first straight line L1 is... Figure 30 Region E1 in the middle.

[0743] In this case, if vehicle 1F moves more than a predetermined distance away from the U-turn reference position P2 in a direction away from the designated driving location RDP, and vehicle 1F is located in area E1, which is further away from the designated driving location RDP than the first straight line L1, the control unit 11F stops the continuous display of the target parking position P1 and the U-turn reference position P2 on the display device 22.

[0744] Alternatively, the continuously displayed stopping conditions may include, in addition to the condition that vehicle 1F moves more than a specified distance away from the designated driving location RDP from the U-turn reference position P2, the following conditions.

[0745] In detail, the continuously displayed stopping condition refers to: the vehicle 1F moving more than a specified distance away from the U-turn reference position P2 in a direction away from the designated travel point RDP, and the vehicle 1F being located on at least one of the sides that are farther away from the designated travel point RDP than the first straight line L1 and the side that is farther away from the designated travel point RDP than the second straight line L2.

[0746] The second straight line L2 is a straight line that passes through the U-turn reference position P2 and is parallel to the long side direction of the target parking space S1 (refer to the direction of arrow LG).

[0747] The target parking space S1 is the space including the target parking position P1, which is the space where vehicle 1F can park. The target parking space S1 is the area with the dimensions, shape, and angle of inclination of vehicle 1F when it is parked in the area represented by the target parking space S1 in a recommended position and posture. Specifically, the target parking space S1 is a rectangular area that surrounds the outer side of vehicle 1F at a predetermined interval from the body 2 of vehicle 1F.

[0748] exist Figure 30 In the example shown, the area in the actual space RS where vehicle 1F is further away from the indicated destination RDP than the second straight line L2 is... Figure 30 Region E2. Therefore, if vehicle 1F is located on at least one of the sides that are farther from the designated service point RDP than the first straight line L1, and on the side that is farther from the designated service point RD than the second straight line L2, vehicle 1F is located in... Figure 30 The region is any one of the following: the overlapping region of region E1 and region E2, the region of region E1 excluding the overlapping region, and the region of region E2 excluding the overlapping region.

[0749] In this case, if vehicle 1F moves more than a predetermined distance away from the designated driving location RDP from the U-turn reference position P2, and vehicle 1F is located in at least one of the areas E1 that are farther away from the designated driving location RDP than the first straight line L1 and the area E2 that are farther away from the designated driving location RDP than the second straight line L2, the control unit 11F stops the continuous display of the target parking position P1 and the U-turn reference position P2 on the display device 22.

[0750] Alternatively, the continuously displayed stopping conditions may include, in addition to the condition that vehicle 1F moves more than a specified distance away from the designated driving location RDP from the U-turn reference position P2, the following conditions.

[0751] In detail, the continuously displayed stopping conditions are as follows: the vehicle 1F moves more than a specified distance away from the U-turn reference position P2 in a direction away from the designated acceptance driving location RDP; the vehicle 1F is located on at least one of the sides that are farther away from the designated acceptance driving location RDP than the first straight line L1 and the side that is farther away from the designated acceptance driving location RD than the second straight line L2; and the vehicle 1F is located outside the circular area C with a specified radius centered on the U-turn reference position P2.

[0752] exist Figure 30 In the example shown, when vehicle 1F is located on at least one of the sides farther from the designated service location RDP than the first straight line L1 and farther from the designated service location RD than the second straight line L2, and vehicle 1F is located outside the circular region C, vehicle 1F is located Figure 30 The region outside the circular region C within the entire region includes the overlapping region of region E1 and region E2, the region of region E1 excluding the overlapping region, and the region of region E2 excluding the overlapping region. In other words, the region outside the circular region C within the entire region is the region within the entire region that does not overlap with the circular region C.

[0753] In this case, if vehicle 1F moves more than a predetermined distance away from the U-turn reference position P2 in a direction away from the designated driving location RDP, and vehicle 1F is located on at least one of the sides that are farther away from the designated driving location RDP than the first straight line L1 and the side that is farther away from the designated driving location RDP than the second straight line L2, and vehicle 1F is located outside the circular area C, the control unit 11F stops the continuous display of the target parking position P1 and the U-turn reference position P2 on the display device 22.

[0754] Next, an example of the information processing flow performed by the control unit 11F in this embodiment will be described.

[0755] Figure 31 This is a flowchart illustrating an example of the information processing flow performed by the control unit 11F.

[0756] The vehicle 1F begins to move forward according to the driver's instructions (step S600). During the forward movement of the vehicle 1F according to the driver's instructions received by the operating device 20, the operating device 20 receives a parking assist start instruction (step S602). When the parking assist start instruction is received, the control unit 11F executes the processing steps S604 to S630.

[0757] The control unit 11F detects a target parking position candidate SP for which the vehicle 1F can park based on the surrounding image V acquired by the camera 16 (step S604). Then, the control unit 11F detects one of the multiple target parking position candidate SPs detected in step S604 that corresponds to the current driving location DP of the vehicle 1F as the target parking position P1 corresponding to the current driving location DP (step S606).

[0758] Then, the control unit 11F detects the U-turn reference position P2 corresponding to the target parking position P1 detected in step S606 (step S608). In addition, the control unit 11F can also further detect the orientation of the vehicle 1F at the U-turn reference position P2.

[0759] Then, the control unit 11F displays the target parking position P1 detected in step S608 and the corresponding U-turn reference position P2 on the display device 22 in a manner superimposed on the surrounding image V (step S610). That is, while the vehicle 1F is moving forward according to the operation received by the operation device 20, the control unit 11F displays the target parking position P1 detected by the surrounding image V obtained by the camera 16 and the corresponding U-turn reference position P2 at the current driving location DP of the vehicle 1F on the display device 22 in a manner superimposed on the surrounding image V.

[0760] Next, the control unit 11F determines whether the operation to select the target parking position P1 displayed on the display device 22 in step S610 has been accepted (step S612). The control unit 11E performs the determination in step S612 by determining whether the passenger has performed a steering operation on the steering wheel (steering control unit, steering wheel) 20A of the operating device 20 in the direction of the U-turn reference position P2.

[0761] For example, if the control unit 11F determines that the operation of selecting the target parking position P1 has been accepted when the steering angle of the steering device is adjusted by the passenger's operation of the steering operation unit and the steering angle detected by the steering angle sensor of the steering device is a specific angle (e.g., 30 degrees or more) deviating towards the U-turn reference position P2.

[0762] When the operating device 20 accepts the operation of selecting the U-turn reference position P2, the control unit 11F determines that the target parking position P1 corresponding to the currently displayed U-turn reference position P2 has been selected.

[0763] When a negative judgment is made in step S612 (step S612: "No"), proceed to step S614. In step S614, the control unit 11F determines whether the next candidate detection condition is met (step S614).

[0764] The control unit 11F determines step S614 by identifying whether the current position of the vehicle 1F (current driving location DP) has passed the target parking position P1 detected in the previous step S606.

[0765] The contro...

Claims

1. A vehicle control method, executed by a vehicle control device mounted on a vehicle, the vehicle having an operating device for receiving operations from a passenger, a camera for acquiring surrounding images, a display that the passenger can visually confirm, and at least a motion control device for controlling steering, wherein in the vehicle control method, While the vehicle is traveling in the direction of travel according to the driver's direction of travel, the target parking position of the vehicle is detected based on the surrounding images acquired by the camera, and the display shows a screen that overlays the target parking position and the corresponding reversing start position onto the surrounding images. After the vehicle has moved forward according to the driver's instructions and reached the reversing starting position, at least the steering wheel is controlled to reverse the vehicle, and the vehicle is then moved autonomously from the reversing starting position to the target parking position. The detected target parking position is set along a direction at a predetermined angle relative to the direction of travel, and is located either to the left or right of the vehicle's centerline along the direction of travel. The starting position for reversing, as displayed on the screen, is located on the other side of the left and right sides of the vehicle's centerline.

2. The vehicle control method according to claim 1, wherein, The specified angle is any one of the following: between 10 and 170 degrees, between 20 and 160 degrees, between 30 and 150 degrees, between 40 and 140 degrees, between 50 and 130 degrees, between 60 and 120 degrees, between 70 and 110 degrees, and between 80 and 100 degrees.

3. The vehicle control method according to claim 1, wherein, After the vehicle has moved forward according to the driver's instructions and reached the reversing start position, and the operating device has performed a prescribed operation, at least the steering wheel is controlled to make the vehicle reverse, and the vehicle is moved from the reversing start position to the target parking position through autonomous driving.

4. The vehicle control method according to claim 1, wherein, When the vehicle meets the prescribed conditions while it is moving forward according to the driver's instructions, the display screen will show the display image.

5. The vehicle control method according to claim 1, wherein, The target parking location displayed on the monitor is indicated by a first frame. The back-start position displayed on the display is indicated by the second box.

6. The vehicle control method according to claim 5, wherein, The first box indicates the orientation of the vehicle at the target parking location. The second box indicates the orientation of the vehicle at the starting position of the reversal.

7. The vehicle control method according to claim 5, wherein, As the vehicle moves forward according to the driver's direction, the display shows a screen that also overlays an estimated frame onto the surrounding image. The estimated frame represents the estimated position and estimated posture of the vehicle as it continues to move forward at the current steering angle according to the driver's direction.

8. The vehicle control method according to claim 7, wherein, As the vehicle moves forward according to the driver's direction, when the estimated frame and the second frame overlap by at least a portion, text prompting continued forward movement with a fixed steering angle is also displayed on the screen.

9. The vehicle control method according to claim 1, wherein, As the vehicle moves forward according to the driver's direction, the display shows a screen that also overlays a recommended deceleration position box onto the surrounding image. The recommended deceleration position box indicates a position where deceleration is recommended from the current position to the starting position of the reversal.

10. The vehicle control method according to claim 1, wherein, As the vehicle moves forward according to the driver's direction, the display shows an image representing a recommended steering state corresponding to the recommended steering angle for moving from the current position to the starting position of reversing.

11. A vehicle control device mounted on a vehicle, the vehicle having an operating device for receiving operations from a passenger, a camera for acquiring surrounding images, a display that the passenger can visually confirm, and at least a motion control device for controlling steering, the vehicle control device performing the following processing: While the vehicle is traveling in the direction of travel according to the driver of the passenger, the target parking position of the vehicle is detected based on the surrounding images acquired by the camera, and the display shows a display screen that overlays the target parking position and the reversing start position corresponding to the target parking position onto the surrounding images. After the vehicle moves forward according to the driver's direction and reaches the reversing start position, at least the steering is controlled to make the vehicle reverse, and the vehicle is moved from the reversing start position to the target parking position by autonomous driving. The detected target parking position is set along a direction with a predetermined angle relative to the direction of travel, and is located either to the left or right of the centerline of the vehicle along the direction of travel. as well as The starting position for reversing, as displayed on the screen, is located on the other side of the left and right sides of the vehicle's centerline.

12. The vehicle control device according to claim 11, wherein, The specified angle is any one of the following: between 10 and 170 degrees, between 20 and 160 degrees, between 30 and 150 degrees, between 40 and 140 degrees, between 50 and 130 degrees, between 60 and 120 degrees, between 70 and 110 degrees, and between 80 and 100 degrees.

13. The vehicle control device according to claim 11, wherein, After the vehicle has moved forward according to the driver's instructions and reached the reversing start position, and the operating device has performed a prescribed operation, at least the steering wheel is controlled to make the vehicle reverse, and the vehicle is moved from the reversing start position to the target parking position through autonomous driving.

14. The vehicle control device according to claim 11, wherein, When the vehicle meets the prescribed conditions while it is moving forward according to the driver's instructions, the display screen will show the display image.

15. The vehicle control device according to claim 11, wherein, The target parking location displayed on the monitor is indicated by a first frame. The back-start position displayed on the display is indicated by the second box.

16. The vehicle control device according to claim 15, wherein, The first box indicates the orientation of the vehicle at the target parking location. The second box indicates the orientation of the vehicle at the starting position of the reversal.

17. The vehicle control device according to claim 15, wherein, As the vehicle moves forward according to the driver's direction, the display shows a screen that also overlays an estimated frame onto the surrounding image. The estimated frame represents the estimated position and estimated posture of the vehicle as it continues to move forward at the current steering angle according to the driver's direction.

18. The vehicle control device according to claim 17, wherein, As the vehicle moves forward according to the driver's direction, when the estimated frame and the second frame overlap by at least a portion, text prompting continued forward movement with a fixed steering angle is also displayed on the screen.

19. The vehicle control device according to claim 11, wherein, As the vehicle moves forward according to the driver's direction, the display shows a screen that also overlays a recommended deceleration position box onto the surrounding image. The recommended deceleration position box indicates a position where deceleration is recommended from the current position to the starting position of the reversal.

20. The vehicle control device according to claim 11, wherein, As the vehicle moves forward according to the driver's direction, the display shows an image representing a recommended steering state corresponding to the recommended steering angle for moving from the current position to the starting position of reversing.

Images