Vehicle control method and vehicle control device

Abstract

To provide more suitable parking support.SOLUTION: A vehicle control device set to be installed on a vehicle includes: an operation device for receiving an operation by an occupant; a camera for acquiring a peripheral image; a display unit which can be visually recognized by the occupant; a movement control device for controlling at least steering. The vehicle control device is set to control at least the steering on the basis of at least a peripheral image and make the vehicle execute an autonomous travel to a parking target position along a teacher route acquired by a teacher travel to the parking target position from a first position. When the vehicle is in a prescribed range from a second position on the teacher route during manual travel of the vehicle by the occupant's operation, the vehicle control device makes the display unit display a first display showing that the vehicle is near a place where automatic parking is allowed.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to a vehicle control method and a vehicle control device. [Background technology]

[0002] Generally, parking spaces in homes are often narrow and can be difficult to park in, so there is a high demand for automated parking and exit.

[0003] Vehicle control devices that realize this type of autonomous driving are known. For example, a technology has been disclosed in which a driver drives a vehicle from a predetermined position outside a parking lot to a target parking position, the route traveled during this process is stored in advance as a training route, and the vehicle autonomously drives along the training route in subsequent parking situations (see Patent Document 1). [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent No. 7120036 Summary of the Invention [Problem to be solved by the invention]

[0005] However, with conventional technology, when starting autonomous driving, the vehicle must first be stopped at the starting position for autonomous driving, and then autonomous driving begins, which can be inconvenient and leaves room for further improvement.

[0006] The present disclosure has been made in consideration of the above-mentioned problems, and aims to provide a vehicle control method and a vehicle control device that can improve the convenience of passengers when performing parking assistance or automatic parking of a vehicle compared to conventional methods. [Means for solving the problem]

[0007] The vehicle control method according to the present disclosure is a vehicle control method in which a vehicle control device mounted on a vehicle equipped with an operating device that accepts operations from a passenger, a camera that acquires surrounding images, a display that is visible to the passenger, and a mobile control device that controls at least steering controls at least steering based on at least the surrounding image, causing the vehicle to perform autonomous driving to the parking target position along a teacher route obtained by teacher driving from a first position to the parking target position, and when the vehicle is manually driven by the passenger, if the vehicle is within a predetermined range from a second position on the teacher route, a first display is displayed on the display indicating that the vehicle is near a location where automatic parking is possible, and next, when the vehicle is at a third position on the teacher route that is a predetermined distance from the first position or has been driven for a predetermined time from the second position as a starting point and is closer to the parking target position than the second position, a second display is displayed on the display indicating that autonomous driving to the parking target position is possible, and next, when the operating device receives a predetermined operation, the vehicle performs autonomous driving by controlling at least steering based on at least the surrounding image from a fourth position that is closer to the parking target position than the third position on the teacher route to the parking target position. [Effects of the Invention]

[0008] According to the vehicle control method of the present disclosure, convenience for the driver or user can be improved compared to the conventional method. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 is a block diagram showing an example of the overall configuration of a vehicle according to an embodiment. [Figure 2] FIG. 2 is an explanatory diagram of an example of a teacher route according to the embodiment. [Figure 3] FIG. 3 is a diagram illustrating an example of a data configuration of teacher route data according to the embodiment. [Figure 4] FIG. 4 is a schematic diagram of an example of map data according to the embodiment. [Figure 5] FIG. 5 is a schematic diagram of an example of map data according to the embodiment. [Figure 6] FIG. 6 is a schematic diagram showing an example of a display screen according to the embodiment. [Figure 7] FIG. 7 is a schematic diagram showing an example of a display screen according to the embodiment. [Figure 8] FIG. 8 is a schematic diagram showing an example of a display screen according to the embodiment. [Figure 9] FIG. 9 is a schematic diagram showing an example of a display screen according to the embodiment. [Figure 10] FIG. 10 is an explanatory diagram of an example of a process for estimating the current position of the vehicle, which is executed by the control unit in the autonomous driving mode according to the embodiment. [Figure 11] FIG. 11 is a flowchart showing an example of the flow of information processing executed by the control unit according to the embodiment when switching from manual driving to autonomous driving mode. [Figure 12] FIG. 12 is a block diagram illustrating an example of the hardware configuration of the vehicle control device according to the embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, embodiments of a vehicle control method and a vehicle control device according to the present disclosure will be described with reference to the drawings.

[0011] (Overall vehicle configuration) FIG. 1 is a block diagram showing an example of the overall configuration of a vehicle 1 according to an embodiment.

[0012] The vehicle 1 includes a vehicle control device 10, a movement control device 12, a sensor 14, a camera 16, a storage device 18, an operation device 20, a display 22, a speaker 24, and a position detection device 26.

[0013] The vehicle control device 10 is connected to the movement control device 12, the sensor 14, the camera 16, the storage device 18, the operation device 20, the display 22, the speaker 24, and the position detection device 26 so as to be able to exchange data or signals. In other words, the vehicle control device 10 is configured to be communicatively connected to at least the movement control device 12, the camera 16, the operation device 20, and the display 22.

[0014] The mobility control device 12 controls at least the steering of the vehicle 1. The mobility control device 12 is a means for realizing the driving, braking, and turning movements required for the vehicle 1 to travel. For example, the mobility control device 12 is configured to include a drive motor, a power transmission mechanism, a brake device, a steering device, etc., and an electronic vehicle control device that controls them. The mobility control device 12 travels the vehicle 1, for example, by generating power using the drive motor and transmitting the power to the wheels via the power transmission mechanism. The power transmission mechanism is, for example, a propeller shaft, a differential gear, a drive shaft, etc.

[0015] The sensors 14 are various sensors mounted on the vehicle 1 and detect the traveling state of the vehicle 1 and the state around the vehicle 1. The sensors 14 include, for example, an accelerator opening sensor that detects an accelerator opening, a steering angle sensor that detects a steering angle of a steering device, an acceleration sensor that detects acceleration acting in the longitudinal direction of the vehicle 1, a torque sensor that detects torque acting on a power transmission mechanism between the wheels of the vehicle 1 and the drive motor, a vehicle speed sensor that detects the vehicle speed of the vehicle 1, a wheel speed sensor, etc. The sensors 14 also include LiDAR (Light detection and ranging), radar, ultrasonic sensors, etc. The sensors 14 output sensor information obtained by detection to the vehicle control device 10.

[0016] The camera 16 is a surroundings sensor mounted on the vehicle 1 and monitors the environment surrounding the vehicle 1. In this embodiment, the camera 16 captures images of the surroundings of the vehicle 1 and sequentially outputs the captured images to the vehicle control device 10. In this embodiment, the camera 16 outputs a surroundings video consisting of a plurality of captured images in time series obtained by capturing images of the surroundings of the vehicle 1 in time series to the vehicle control device 10. The camera 16 acquires the surroundings video. Hereinafter, the surroundings video may be simply referred to as video. In addition, in this embodiment, the camera 16 is also used to detect objects present around the vehicle 1 and estimate the location of the vehicle 1 from the positional relationship between the vehicle 1 and the objects present around the vehicle 1.

[0017] The positions, number of cameras 16, and shooting directions are adjusted in advance so that the cameras 16 can capture images of the surroundings of the vehicle 1. For example, the vehicle 1 is provided with four cameras 16 arranged so as to capture images in four directions, namely, the front, rear, left, and right directions of the vehicle 1. Note that the number of cameras 16 provided on the vehicle 1 is not limited to four.

[0018] The storage device 18 stores various types of data. In this embodiment, the storage device 18 stores data such as teacher route data D1 and map data D2. The teacher route data D1 and the map data D2 will be described in detail later. The storage device 18 is, for example, an auxiliary storage device such as an HDD (Hard Disk Drive), an SSD (Solid State Drive), or a flash memory.

[0019] The operation device 20 acquires input operations by the passenger. The operation device 20 is, for example, an input device such as a steering wheel, an accelerator pedal, a brake pedal, a turn signal pedal, a push switch, a keyboard, or a touch panel.

[0020] The display 22 is a display that displays various images. The display 22 is installed in a position where it can be seen by a user who is a passenger of the vehicle 1. Examples of the display include a liquid crystal display (LCD), an organic electroluminescence (EL) display, and a projector. The display 22 may be a touch panel display that is an integral configuration of the display 22 with a keyboard and a touch panel. The display 22 is an example of an HMI.

[0021] The speaker 24 emits sound toward the passengers. The speaker 24 is a device configured to be able to output sound. The speaker 24 is able to output predetermined notifications as sound. The speaker 24 is provided in the vehicle 1 so that the driver of the vehicle 1 can hear the output sound. In this embodiment, sound is not limited to language and includes notification sounds and warning sounds.

[0022] The position detection device 26 detects at least the latitude and longitude. For example, the position detection device 26 is a device that detects latitude and longitude information indicating the latitude and longitude of the position of the vehicle 1 based on a GPS signal received by a GPS antenna. The GPS antenna is an antenna that can receive GPS signals.

[0023] The vehicle control device 10 is an electronic control unit that controls each part of the vehicle 1 in an integrated manner.

[0024] The vehicle control device 10 uses sensor information and images received from the sensors 14 and the cameras 16 to control the movement control device 12 so as to optimize the driving conditions of the vehicle 1. The vehicle control device 10 also controls the movement control device 12 to cause the vehicle 1 to drive autonomously.

[0025] The vehicle control device 10 includes a control unit 11. Part or all of the control unit 11 may be a software configuration realized by cooperation between a processor and various programs stored in memory. Alternatively, part or all of the control unit 11 may be a hardware configuration realized by a dedicated circuit or the like.

[0026] The control unit 11 controls each part of the vehicle 1 in an integrated manner.

[0027] In this embodiment, the control unit 11 is configured to be able to switch the driving mode between the teacher driving mode and the autonomous driving mode based on the user's input operation on the operation device 20. Note that the driving modes that can be executed by the vehicle 1 may include various driving modes other than the teacher driving mode and the autonomous driving mode.

[0028] The supervised driving mode is a mode for registering a supervised route for autonomous driving of the vehicle 1. The supervised route is a route obtained by supervised driving from a first position to a target parking position. In the supervised driving mode, the vehicle 1 is controlled to travel by the driving operation of the user. That is, in the supervised driving mode, the control unit 11 controls the movement control device 12 so that the vehicle travels in accordance with the driving operation of the user.

[0029] The autonomous driving mode is a mode in which the vehicle 1 travels autonomously. In this embodiment, the autonomous driving mode refers to a mode in which the vehicle 1 travels autonomously along a teacher route. In the autonomous driving mode, the control unit 11 controls at least steering and controls the mobile control device 12 so that the vehicle travels along the teacher route. In the autonomous driving mode, the vehicle 1 is automatically controlled to travel by the vehicle control device 10 without any driving operation by the user.

[0030] FIG. 2 is an explanatory diagram of an example of a teacher route R1 according to the embodiment.

[0031] In the supervised driving mode, the user drives the vehicle to supervise driving from a first position P1 to a target parking position P2. The target parking position P2 is, for example, a parking lot, but is not limited to this. The first position P1 may be any position of the user in real space.

[0032] The travel route R traveled by the teacher travel is treated as a teacher route R1, and teacher route data D1 of the teacher route R1 is stored in the storage device 18. During teacher travel, the user may perform a driving operation to travel from the parking target position P2 toward the first position P1, or may perform a driving operation to travel from the first position P1 toward the parking target position P2.

[0033] When the vehicle 1 travels from the parking target position P2 toward the first position P1 in the teacher driving mode, the control unit 11 can create teacher route data D1 for the teacher route R1, which is the opposite of the traveling direction during teacher driving of the travel route R. When the vehicle 1 travels from the first position P1 toward the parking target position P2 during teacher driving, the control unit 11 can create teacher route data D1 for the teacher route R1, which is the same as the traveling direction during teacher driving of the travel route R. The creation of the teacher route data D1 will be described in detail later.

[0034] In the autonomous driving mode, the control unit 11 controls at least the steering based on at least the surrounding image along the driving route R obtained by the supervised driving from the first position P1 to the parking target position P2, and causes the vehicle 1 to perform autonomous driving to the parking target position P2. Note that in the autonomous driving mode, the control unit 11 performs steering control and front and rear acceleration / deceleration control of the vehicle 1, but at least a part of the front and rear acceleration / deceleration control may be performed by the driver's operation.

[0035] Next, the control by the control unit 11 in the teacher driving mode and the autonomous driving mode will be described in detail.

[0036] (Instructor driving mode) First, the control of the control unit 11 in the teacher running mode will be described in detail.

[0037] When the control unit 11 receives a signal indicating an instruction to start the teacher-driven driving mode by a user operating the operation device 20, the control unit 11 switches the driving mode to the teacher-driven driving mode. Then, when the control unit 11 is in the teacher-driven driving mode, the control unit 11 executes the following processing.

[0038] The control unit 11 acquires sensor information indicating the traveling state of the vehicle 1 from the sensor 14. Then, the control unit 11 estimates the current position of the vehicle 1 based on the temporal changes in the sensor values ​​represented by the sensor information. For example, the control unit 11 calculates the amount of movement of the vehicle 1 from a reference position, such as the traveling start position when the teacher traveling mode was started, based on the temporal changes in the vehicle speed and yaw rate represented by the sensor values, and estimates the current position of the vehicle 1 based on the amount of movement.

[0039] Note that the accuracy of estimating the current position based on the amount of movement may be low. Therefore, the control unit 11 may correct the estimated current position based on an image of the surroundings of the vehicle 1 acquired by the camera 16, and use the corrected result as the current position.

[0040] The control unit 11 sequentially stores the current position of the vehicle 1 estimated sequentially as the vehicle 1 travels in the storage device 18. In detail, the control unit 11 defines the travel route R during teacher travel represented by a group of current positions sequentially estimated from the time when an instruction to start the teacher travel mode is received until the time when an instruction to end the teacher mode is received as a teacher route R1, and stores teacher route data D1 representing the teacher route R1 in the storage device 18.

[0041] FIG. 3 is a diagram showing an example of the data configuration of teacher route data D1 according to the embodiment. The teacher route data D1 is composed of a group of travel information for each position, which is the current position sequentially estimated during teacher travel. Information related to the teacher route R1 includes latitude and longitude information of the teacher route R1. The travel information includes an index, a travel position, a bearing, a travel direction, and reference travel information.

[0042] INDEX is identification information for the driving information. The driving position is the estimated position of vehicle 1. The position of vehicle 1 is, for example, a position detected by a position detection device, and is indicated using latitude and longitude. The heading indicates the orientation of vehicle 1 at that position. The driving direction indicates the direction in which vehicle 1 is driving at that position, and is expressed, for example, as forward or reverse. The reference driving information is information that indicates the driving condition, etc. at that position. The reference driving information is, for example, information such as steering angle, vehicle speed, etc. detected at each position during teacher driving.

[0043] Furthermore, when the vehicle 1 is traveling on a road map, the control unit 11 creates map data D2 for estimating the current position of the vehicle 1 from the image captured by the camera 16. The method for estimating the current position of the vehicle 1 from the image may be the SLAM (Simultaneous Localization and Mapping) method or the like. The map data D2 includes latitude and longitude information detected by the position detection device 26.

[0044] Fig. 4 is a schematic diagram of an example of map data D2 according to the embodiment. Fig. 4 shows a bird's-eye view of the real-space position of a feature point Q in the actual scene with latitude and longitude information stored in the map data D2. The map data D2 is map data in which a plurality of feature points Q around the vehicle 1 when traveling along the training route R1 are registered.

[0045] The feature point Q is a characteristic point obtained from latitude and longitude information detected by the position detection device 26 during teacher driving. The feature point Q is also a characteristic point obtained by image analysis of the image captured by the camera 16 during teacher driving. For example, the feature point Q is a portion of an object (e.g., a tree, wall, or pillar) that can serve as a landmark in the actual scene, from which a characteristic image pattern is obtained by analyzing the captured image. Such a portion is, for example, the edge of the object. The map data D2 includes a plurality of feature points Q, and each feature point Q is assigned an identification number and registered so that it can be identified.

[0046] The feature point Q is represented by feature point data including latitude and longitude, a three-dimensional position, and a feature amount.

[0047] The latitude and longitude of the feature point Q are expressed, for example, in a geographic coordinate system (V, W). The three-dimensional position of the feature point Q is the three-dimensional position of the feature point Q in real space, and is expressed, for example, in a three-dimensional Cartesian coordinate system (X, Y, Z).

[0048] The feature amount of the feature point Q is a characteristic amount expressed by image analysis of the captured image of the feature point Q. The feature amount of the feature point Q is, for example, the brightness or density on the captured image, a SIFT (Scale Invariant Feature Transform) feature amount, a SURF (Speeded Up Robust Features) feature amount, etc.

[0049] In the map data D2, one feature point Q is registered for each identical latitude and longitude and three-dimensional position. Note that, for the same three-dimensional position, the map data D2 may register multiple feature points Q for each photographing position and photographing direction of the camera 16 at that three-dimensional position. Furthermore, the feature point data of the feature point Q registered in the map data D2 may further include image data of an object having the feature point Q.

[0050] During teacher driving, the control unit 11 identifies the coordinates of the feature point Q in the actual scene, for example, based on stereo photogrammetry. Specifically, the control unit 11 reads out multiple captured images taken at different times and associates the same feature point Q that appears in the multiple captured images. Then, the control unit 11 estimates the virtual position of the vehicle 1 when the multiple captured images were taken, for example, and identifies the virtual coordinates of the feature point Q in the actual scene based on the principle of triangulation.

[0051] Then, the control unit 11 performs bundle adjustment using, for example, the virtual position of the vehicle 1 and the virtual coordinates of the feature point Q in the actual scene as reference information, and calculates the formal position of the vehicle 1 and the formal coordinates of the feature point Q in the actual scene so as to minimize the re-projection error when each feature point Q in the actual scene is projected onto all captured images.The control unit 11 then stores in the storage device 18 map data D2 in which the feature point Q is registered, the feature point data being represented by feature point data including the formal coordinates of the feature point Q in the actual scene as latitude, longitude, and three-dimensional position.

[0052] The three-dimensional position of the feature point Q registered in the map data D2 may be a position measured in advance using a LiDAR (Light Detection and Ranging) or a stereo camera instead of using the SLAM method. However, from the viewpoint of suppressing a decrease in the accuracy of position estimation, it is preferable to use the SLAM method.

[0053] The control unit 11 may generate one piece of map data D2 that includes the entire teacher route R1, as shown in FIG. 4, or may create multiple pieces of partial map data that are at least partially non-overlapping along the teacher route R1 as the map data D2.

[0054] Fig. 5 is a schematic diagram of an example of map data D2 according to an embodiment. Fig. 4B shows a bird's-eye view of the position in real space of feature point Q in a real scene stored in map data D2. As shown in Fig. 5, map data D2 may be made up of multiple partial map data D2P that are at least partially non-overlapping along a training route R1. Each partial map data D2P may have registered therein feature point data for feature point Q in the same manner as described above.

[0055] By configuring the map data D2 from a plurality of partial map data D2P, the data volume of the map data D2 can be reduced compared to when one piece of map data D2 including the entire teacher route R1 is created.

[0056] As described above, the control unit 11 executes the above process in the teacher driving mode. Therefore, in the teacher driving mode, the control unit 11 generates teacher route data D1 of the teacher route R1 obtained by teacher driving from the first position P1 to the parking target position P2, and map data D2 in which the latitude and longitude of each of a plurality of feature points Q around the vehicle 1 while traveling along the teacher route R1, the three-dimensional position of each of the feature points Q, and the feature amounts of the feature points Q are registered, and stores these in the storage device 18.

[0057] (Switching from manual to autonomous driving mode) Next, the control of the control unit 11 when switching from manual driving to automatic driving mode will be described in detail.

[0058] Based on the travel longitude information of the teacher route R1 and the latitude and longitude detected by the position detection device 26, the control unit 11 determines whether the vehicle 1 is within a specified range from the second position on the teacher route R1 while the vehicle 1 is being manually driven by the passenger.

[0059] In addition, when the vehicle 1 is being manually driven by the passenger, and the vehicle 1 is within a predetermined range from the second position on the teacher route R1, the control unit 11 causes the display 22 to display a first display indicating that the vehicle is near a location where automatic parking is possible.

[0060] The first display will now be described with reference to Fig. 6. Fig. 6 is a schematic diagram showing an example of a display screen according to the embodiment. The display screen of the first display displayed by the display device 22 is, for example, a screen in which the second position P3, the teacher route R1, and the message M1 are superimposed on the image 32.

[0061] 6 shows an example in which the second position P3 is displayed by placing an icon representing the vehicle 1 within a predetermined range of the second position P3 on the image 32. The second position P3 may be, for example, near the first position P1, coincident with the first position P1, or between the first position P1 and the parking target position P2.

[0062] FIG. 6 also shows an example in which a line image representing the teacher route R1 is superimposed on the image 32. Furthermore, FIG. 6 shows an example in which the message M1 includes "You are near a location where automatic parking is possible," which is an example of a sentence indicating that you are near a location where automatic parking is possible. The first display includes a display in which at least a portion of the teacher route R1 is superimposed on the surrounding image acquired by the camera 16. This allows the passenger to understand that they are traveling along the teacher route R1 through manual operation.

[0063] The first display may also include a display in which the parking target position P2 is superimposed on the surrounding image captured by the camera 16. In this case, the parking target position P2 is displayed, for example, as a frame. Furthermore, the display screen of the first display may further include a screen in which an icon CP representing the current position and current attitude of the vehicle 1 is superimposed on the image 33.

[0064] Furthermore, when the vehicle 1 is within a predetermined range from the second position P3 on the instructor route R1 while the vehicle 1 is being manually driven by the passenger, the control unit 11 may cause the display 22 to display a first display indicating that the vehicle 1 is near a location where automatic parking is possible, and may cause the speaker 24 to emit a sound indicating that the vehicle 1 is near a location where automatic parking is possible. The sound is emitted in conjunction with the first display displayed by the display 22.

[0065] Next, when the vehicle 1 is at a third position on the teacher route R1 that is a predetermined distance away from the first position P1 or that has traveled a predetermined time from the second position P3 as a starting point and is closer to the parking target position P2 than the second position P3, the control unit 11 causes the display 22 to display a second display indicating that autonomous driving to the parking target position P2 is possible. The second display includes a display in which at least a part of the teacher route R1 is superimposed on the surrounding image acquired by the camera 16.

[0066] The second display will now be described with reference to Fig. 7. Fig. 7 is a schematic diagram showing an example of a display screen according to the embodiment. The display screen of the second display displayed by the display device 22 is, for example, a screen in which the second position P3, the third position P4, the teacher route R1, and the message M2 are superimposed on the image 34.

[0067] 7 shows an example in which the second position P3 and the third position P4 are displayed by placing an icon representing the vehicle 1 at a third position P4 on the teacher route R1 on the image 32, the third position P4 being a predetermined distance away from the first position P1 or a predetermined time after traveling from the second position P3 as a starting point and closer to the parking target position P2 than the second position P3. Also shown in FIG. 7 is an example in which an arrow image representing the teacher route R1 is superimposed on the image 34.

[0068] 7 shows an example in which the message M2 includes "Automatic parking is possible," which is an example of a sentence indicating that autonomous driving is possible to the parking target position P2, and "Press the button to start parking," which is an example of a sentence prompting switching to autonomous driving mode. The second display includes a display in which at least a portion of the teacher route R1 is superimposed on the surrounding image acquired by the camera 16. This allows the passenger to understand that they are manually driving along the teacher route R1.

[0069] The second display may also include a display indicating the parking target position P2. In this case, the parking target position P2 is displayed, for example, in a schematic manner. Furthermore, the second display includes a display in which the parking target position P2 is superimposed on the surrounding image captured by the camera 16. This allows the passenger to intuitively grasp the parking target position P2. The display screen of the second display may further include a screen in which an icon CP indicating the current position and attitude of the vehicle 1 is superimposed on the image 35.

[0070] Furthermore, when the vehicle 1 is at a third position P4 on the teacher route R1 that is a predetermined distance away from the first position P1 or that has traveled a predetermined time from the second position P3 as a starting point and is closer to the parking target position P2 than the second position P3, the control unit 11 may cause the display device 22 to display a second display indicating that autonomous driving to the parking target position P2 is possible, and may also cause the speaker 24 to emit a sound indicating that autonomous driving is possible. The sound is emitted in conjunction with the second display displayed by the display device 22.

[0071] (in autonomous driving mode) Next, the control of the control unit 11 in the autonomous driving mode will be described in detail.

[0072] When the operation device 20 receives a predetermined operation, the control unit 11 controls at least the steering based on at least the surrounding image to cause the vehicle to travel autonomously from a fourth position, which is closer to the parking target position P2 than the third position on the teacher path R1, to the parking target position P2. Specifically, the control unit 11 switches from manual travel to the autonomous travel mode. Then, when the operation device 20 receives a predetermined operation, the control unit 11 causes the display 22 to display a third display indicating that the automatic parking function has been activated, and controls at least the steering based on at least the surrounding image to cause the vehicle to travel autonomously from the fourth position on the teacher path R1 to the parking target position P2.

[0073] The third display will now be described with reference to Fig. 8. Fig. 8 is a schematic diagram showing an example of a display screen according to the embodiment. The display screen of the third display displayed by the display device 22 is, for example, a screen in which the parking target position P2, the fourth position P5, the teacher route R1, and the message M3 are superimposed on the image 36.

[0074] Fig. 8 shows an example in which the parking target position P2 and the fourth position P5 are displayed by placing an icon representing the vehicle 1 from the fourth position P5 on the teacher route R1 to the parking target position P2 on the image 36. Fig. 8 also shows an example in which an arrow image representing the teacher route R1 is superimposed on the image 36.

[0075] Furthermore, Figure 8 shows an example in which message M3 includes "Automatic driving has started," which is an example of a sentence that indicates that the automatic parking function has been activated. This allows the passenger to understand that the vehicle is traveling autonomously along the teacher route R1.

[0076] Furthermore, when the operation device 20 receives a predetermined operation, the control unit 11 may cause the display device 22 to display a third display indicating that the automatic parking function has been activated, and may cause the speaker 24 to emit a sound indicating that the automatic parking function has been activated. The sound may be emitted in conjunction with the third display displayed by the display device 22.

[0077] The vehicle 1 passes through the second position P3, the third position P4, and the fourth position P5 on the teacher route R1 toward the parking target position P2 at a speed that is a positive value and is greater than a predetermined value.

[0078] Furthermore, when the operation device 20 receives a predetermined operation, for example, if the predetermined operation is performed too slowly and the mode does not switch to the autonomous driving mode, the control unit 11 may display the fourth message indicating that the automatic parking function will not be activated.

[0079] Here, the fourth display indicating that the automatic parking function will not be activated will be described with reference to Fig. 9. Fig. 9 is a schematic diagram showing an example of a display screen according to the embodiment. The display screen of the fourth display displayed by the display device 22 is, for example, a screen in which a teacher route R1 and a message M4 are superimposed on the image 38.

[0080] FIG. 9 shows an example in which an arrow image representing the teacher route R1 is superimposed on the video 38. FIG. 9 also shows an example in which an icon representing the vehicle 1 that has deviated from the teacher route R1 is placed. Furthermore, FIG. 9 shows an example in which the message M4 includes "Switching to autonomous driving has failed," which is an example of a sentence indicating that the automatic parking function has not been activated. This allows the passenger to understand that switching to autonomous driving mode has failed.

[0081] Here, the operation device 20 receiving the predetermined operation means that the operation device 20 has acquired an input operation by the passenger. The predetermined operation is, for example, an input operation corresponding to a pushing operation by the passenger from an input device such as a push switch. Note that the predetermined operation is not limited to this.

[0082] Then, the control unit 11 executes the following process in the autonomous driving mode.

[0083] The control unit 11 reads the teacher route data D1 and the map data D2 from the storage device 18, and controls the mobile control device 12 to perform autonomous driving along the teacher route R1 represented by the teacher route data D1. The control unit 11 also estimates the current position of the vehicle 1 based on the map data D2 and images of the surroundings of the vehicle 1 captured by at least one camera 16.

[0084] FIG. 10 is an explanatory diagram of an example of a process for estimating the current position of the vehicle 1 executed by the control unit 11 in the autonomous driving mode according to the embodiment.

[0085] 10, S1, S2, and S3 represent three characteristic points extracted from the image captured by camera 16, and points Q1, Q2, and Q3 are characteristic points Q stored in map data D2 and represent the three-dimensional positions of characteristic points S1, S2, and S3 in real space. RP1 represents the imaging plane of camera 16. Point P' represents the position of camera 16 (i.e., the position of vehicle 1) determined from the three characteristic points S1, S2, and S3 extracted from the image captured by camera 16 and characteristic point Q (Q1, Q2, and Q3) stored in map data D2.

[0086] For example, the control unit 11 first compares characteristic points extracted from the image captured by the camera 16 with characteristic points Q stored in the map data D2 using pattern matching, feature amount search, etc. Then, the control unit 11 randomly selects several (e.g., 3 to 6) characteristic points from the characteristic points extracted from the image captured by the camera 16 that can be compared with the characteristic points Q stored in the map data D2.

[0087] Then, the control unit 11 estimates the current position of the vehicle 1 in the real space based on the positions of these several characteristic points in the captured image and the three-dimensional position in the real space of the characteristic point Q registered in the map data D2 corresponding to the several characteristic points.

[0088] In this case, the control unit 11 estimates the current position of the vehicle 1 by solving the PnP problem using a known method such as Lambda Twist (for example, see the literature: Mikael Persson et al. “Lambda Twist: An Accurate Fast Robust Perspective Three Point (P3P) Solver.”, ECCV 2018, pp. 334-349, published in 2018, http: / / openaccess.thecvf.com / content_ECCV_2018 / papers / Mikael_Persson_Lambda_Twist_An_ECCV_2018_paper.pdf).

[0089] When comparing the characteristic points extracted from the image captured by the camera 16 with the characteristic points Q stored in the map data D2, the control unit 11 may, for example, calculate the current position of the vehicle 1 as a tentative position based on the amount of movement of the vehicle 1 described above, and use the tentative position as a reference to narrow down the characteristic points Q stored in the map data D2 to be compared with the characteristic points extracted from the image captured by the camera 16.

[0090] Through these processes, the control unit 11 estimates current position information representing the current position of the vehicle 1 based on the map data D2 and the captured images of the surroundings of the vehicle 1 acquired by at least one camera 16, and the current position information includes information relating to the two-dimensional position (X coordinate, Y coordinate) of the vehicle 1 in real space and the orientation of the vehicle 1.

[0091] The control unit 11 then controls the movement control device 12 so that the estimated current position of the vehicle 1 is a position on the teacher route R1 represented by the teacher route data D1, thereby causing the vehicle 1 to autonomously travel along the teacher route R1 from the first position P1 toward the parking target position P2. The control unit 11 then stops the vehicle 1 at the parking target position P2.

[0092] When the vehicle 1 is autonomously traveling along the travel route R, the control unit 11 feedback controls the movement control device 12 so that the vehicle 1 moves along the teacher route R1 based on the estimated current position of the vehicle 1 and each position on the teacher route R1 represented by the teacher route data D1.

[0093] Next, there will be described an example of the flow of information processing executed by the control unit 11 of the vehicle control device 10. Fig. 11 is a flowchart showing an example of the flow of information processing executed by the control unit 11 when switching from manual driving to autonomous driving mode.

[0094] The control unit 11 manually drives the vehicle 1 based on the driving operation of the vehicle 1 by the passenger (step S111). Based on the movement longitude information of the teacher route R1 and the latitude and longitude detected by the position detection device 26, the control unit 11 determines whether the vehicle 1 is within a predetermined range from the second position P3 on the teacher route R1 while the vehicle 1 is being manually driven by the passenger (step S112).

[0095] Here, if the control unit 11 determines that the vehicle 1 is not within a predetermined range from the second position P3 on the teacher route R1 (step S112: No), the process returns to step S111. On the other hand, if the control unit 11 determines that the vehicle 1 is within a predetermined range from the second position P3 on the teacher route R1 (step S112: Yes), the process proceeds to step S113. In step S113, the control unit 11 causes the display 22 to display a first display indicating that the vehicle is near a location where automatic parking is possible (step S113).

[0096] The control unit 11 determines whether the vehicle 1 is located at a third position P4 on the teacher route R1, which is a predetermined distance away from the first position P1 or a predetermined time after traveling from the second position P3 as a starting point and is closer to the parking target position P2 than the second position P3 (step S114). If the control unit 11 determines that the vehicle 1 is not located at the third position P4 on the teacher route R1, which is a predetermined distance away from the first position P1 or a predetermined time after traveling from the second position P3 as a starting point and is closer to the parking target position P2 than the second position P3 (step S114: No), the control unit 11 returns to step S114 and executes the process again.

[0097] On the other hand, if the control unit 11 determines that the vehicle 1 is at a third position P4 on the teacher route R1 that is a predetermined distance away from the first position P1 or that has traveled a predetermined time from the second position P3 as a starting point and is closer to the parking target position P2 than the second position P3 (step S114: Yes), the process proceeds to step S115. In step S115, the control unit 11 causes the display 22 to display a second display indicating that autonomous driving to the parking target position P2 is possible (step S115).

[0098] The control unit 11 determines whether the operation device 20 has received a predetermined operation (step S116). If the control unit 11 determines that the operation device 20 has not received a predetermined operation (step S116: No), the control unit 11 returns to step S116 and executes the process again. On the other hand, if the control unit 11 determines that the operation device 20 has received a predetermined operation (step S116: Yes), the control unit 11 proceeds to step S117. In step S117, the control unit 11 switches from the manual driving mode to the autonomous driving mode.

[0099] The control unit 11 determines whether the switch from manual driving to autonomous driving mode has been successful (step S118). If the control unit 11 determines that the switch from manual driving to autonomous driving mode has failed (step S118: No), the process proceeds to step S119. On the other hand, if the control unit 11 determines that the switch from manual driving to autonomous driving mode has been successful (step S118: Yes), the process proceeds to step S120.

[0100] In step S119, the control unit 11 causes the display 22 to display a fourth display indicating that the automatic parking function has not been activated (step S119). In step S120, the control unit 11 causes the display 22 to display a third display indicating that the automatic parking function has been activated (step S120). The control unit 11 controls at least the steering based on at least the surrounding image to cause the vehicle 1 to autonomously travel from the fourth position P5 on the teacher path R1 to the parking target position P2 (step S121). Then, the control unit 11 controls the movement control device 12 to stop the vehicle 1 at the parking target position P2, and ends this routine.

[0101] As described above, the vehicle control device 10 of this embodiment is configured to be connected to an operating device 20 that accepts operations from the occupant, a camera 16 that acquires surrounding images, a display 22 that is visible to the occupant, and a mobile control device 12 that controls at least steering, and causes the vehicle 1 to perform autonomous driving to the parking target position P2 by controlling at least steering based on at least the surrounding images along the teacher route R1 obtained by teacher driving from the first position P1 to the parking target position P2.

[0102] When the vehicle 1 is manually driven by the passenger and the vehicle 1 is within a predetermined range from the second position P3 on the teacher route R1, the vehicle control device 10 causes the display 22 to display a first display indicating that the vehicle is near a location where automatic parking is possible. Next, when the vehicle 1 is at a third position P4 on the teacher route R1 that is a predetermined distance from the first position P1 or that has traveled a predetermined time from the second position P3 as a starting point and is closer to the parking target position P2 than the second position P3, the vehicle control device 10 causes the display 22 to display a second display indicating that autonomous driving to the parking target position P2 is possible. Next, when the operation device 20 receives a predetermined operation, the vehicle control device 10 controls at least the steering based on at least the surrounding image to cause the vehicle 1 to autonomously drive from a fourth position P5, which is closer to the parking target position P2 than the third position P4 on the teacher route R1, to the parking target position P2.

[0103] In this way, when the vehicle 1 is at the third position P4, which is closer to the parking target position P2 than the second position P3, the vehicle control device 10 of this embodiment displays on the display 22 a second display indicating that autonomous driving is possible to the parking target position P2, and when the operation device 20 receives a predetermined operation, displays on the display 22 a third display indicating that the automatic parking function has been activated, and controls at least the steering based on at least the surrounding image to allow the vehicle to drive autonomously from the fourth position P5 on the teacher route R1 to the parking target position P2.

[0104] Therefore, when the vehicle 1 is on the teacher route R1, the vehicle control device 10 of this embodiment controls at least the steering based on at least the surrounding image to cause the vehicle 1 to drive autonomously by performing a predetermined operation by the occupant. Therefore, when starting autonomous driving, the occupant can start autonomous driving without first stopping the vehicle 1 at the start position of autonomous driving.

[0105] Therefore, the vehicle control device 10 of this embodiment can improve the convenience for the passengers when performing parking assistance or automatic parking of the vehicle 1 compared to conventional devices.

[0106] Next, a description will be given of the hardware configuration of the vehicle control device 10 of this embodiment.

[0107] The vehicle control device 10 has a hardware configuration that utilizes a normal computer, with a CPU (Central Processing Unit) 11A, a ROM (Read Only Memory) 11B, a RAM (Random Access Memory) 11C, an I / F 11D for connecting to various devices, and the like interconnected by a bus 11E.

[0108] The CPU 11A is a calculation device that controls the overall processing of the vehicle control device 10. The RAM 11C stores data necessary for various processes by the CPU 11A. The ROM 11B stores programs and the like that realize various processes by the CPU 11A. The I / F 11D is an interface that is connected to an external device or external terminal via a communication line or the like and transmits and receives data to and from the connected external device or external terminal.

[0109] The programs for executing the various processes described above executed by the vehicle control device 10 are provided by being pre-installed in the ROM 11B, etc. The programs for executing the vehicle control method executed in this embodiment may be configured to be provided by being recorded on a computer-readable recording medium such as a CD-ROM, a flexible disk (FD), a CD-R, or a digital versatile disc (DVD) in a format that can be installed or executed by these devices.

[0110] The program for executing the vehicle control method according to the present embodiment may be stored on a computer connected to a network such as the Internet and provided by being downloaded via the network. The program for executing the vehicle control method according to the present embodiment may be provided or distributed via a network such as the Internet.

[0111] Although an embodiment of the present disclosure has been described, this embodiment is presented as an example and is not intended to limit the scope of the invention. This embodiment can be implemented in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. This embodiment is included in the scope and spirit of the invention, as well as in the invention described in the claims and their equivalents. [Explanation of symbols]

[0112] 1 vehicle 10 Vehicle control device 12 Movement control device 16 Camera 20 Operating device 22 Display 24 speakers 26 Position detection device

Claims

1. A vehicle control method is provided in a vehicle equipped with an operation device that receives operations from a passenger, a camera that captures surrounding images, a display that is visible to the passenger, and a movement control device that controls at least steering, the method controlling at least the steering based on at least the surrounding images to cause the vehicle to perform autonomous driving to a parking target position along a teacher route obtained by teacher driving from a first position to the parking target position, the method comprising: When the vehicle is manually driven by the passenger, if the vehicle is within a predetermined range from a second position on the instructor route, a first display indicating that the vehicle is near a location where automatic parking is possible is displayed on the display device; Next, when the vehicle is at a third position on the teacher route that is a predetermined distance away from the first position or that has traveled a predetermined time from the second position as a starting point and is closer to the parking target position than the second position, a second display indicating that autonomous driving to the parking target position is possible is displayed on the display device; Next, when the operation device receives a predetermined operation, the vehicle is caused to autonomously travel from a fourth position on the teacher path that is closer to the parking target position than the third position to the parking target position by controlling at least the steering based on at least the surrounding image. Vehicle control method.

2. 2. The vehicle control method according to claim 1, When the operation device receives the predetermined operation, a third display indicating that an automatic parking function has been activated is displayed on the display device; and controlling at least the steering based on at least the surrounding image to perform autonomous driving from the fourth position on the teacher path to the parking target position. Vehicle control method.

3. 2. The vehicle control method according to claim 1, the vehicle passes through the second position, the third position, and the fourth position on the teacher path toward the parking target position at a speed that is a positive value and is greater than a predetermined value; Vehicle control method.

4. 2. The vehicle control method according to claim 1, the first display includes a display in which at least a part of the teacher path is superimposed on the surrounding image acquired by the camera; Vehicle control method.

5. 5. The vehicle control method according to claim 4, The first display includes a display in which the parking target position is superimposed on the surrounding image acquired by the camera. Vehicle control method.

6. 2. The vehicle control method according to claim 1, The second display includes a display in which at least a part of the teacher path is superimposed on the surrounding image acquired by the camera. Vehicle control method.

7. 2. The vehicle control method according to claim 1, The second display includes a display indicating the parking target position. Vehicle control method.

8. 7. A vehicle control method according to claim 6, The second display includes a display in which the parking target position is superimposed on the surrounding image acquired by the camera. Vehicle control method.

9. 2. The vehicle control method according to claim 1, The vehicle further includes a speaker that emits sound toward the passenger, When the vehicle is manually driven by the passenger and the vehicle is located within the predetermined range from the second position on the instructor route, the display device displays the first display indicating that the vehicle is near a location where automatic parking is possible, and the speaker emits a sound indicating that the vehicle is near a location where automatic parking is possible. Vehicle control method.

10. 2. The vehicle control method according to claim 1, the vehicle further includes a position detection device that detects at least latitude and longitude; the information relating to the teacher route includes latitude and longitude information of the teacher route, determining whether the vehicle is located within a predetermined range from the second position on the teacher route while the vehicle is being manually driven by the passenger, based on the latitude and longitude information of the teacher route and the latitude and longitude detected by the position detection device; Vehicle control method.

11. A vehicle control device configured to be mounted on a vehicle including an operating device that accepts operations by a passenger, a camera that acquires surrounding images, a display that can be seen by the passenger, and a movement control device that controls at least steering, the vehicle is configured to perform autonomous driving to the parking target position by controlling at least the steering based on at least the surrounding image along a teacher route obtained by teacher driving from the first position to the parking target position, When the vehicle is manually driven by the passenger, if the vehicle is within a predetermined range from a second position on the instructor route, a first display indicating that the vehicle is near a location where automatic parking is possible is displayed on the display device; Next, when the vehicle is at a third position on the teacher route that is a predetermined distance away from the first position or that has traveled a predetermined time from the second position as a starting point and is closer to the parking target position than the second position, a second display indicating that autonomous driving to the parking target position is possible is displayed on the display device; Next, when the operation device receives a predetermined operation, the vehicle is caused to autonomously travel from a fourth position on the teacher path that is closer to the parking target position than the third position to the parking target position by controlling at least the steering based on at least the surrounding image. Vehicle control device.

12. The vehicle control device according to claim 11, When the operation device receives the predetermined operation, a third display indicating that an automatic parking function has been activated is displayed on the display device; and controlling at least the steering based on at least the surrounding image to perform autonomous driving from the fourth position on the teacher path to the parking target position. Vehicle control device.

13. The vehicle control device according to claim 11, the vehicle passes through the second position, the third position, and the fourth position on the teacher path toward the parking target position at a speed that is a positive value and is greater than a predetermined value; Vehicle control device.

14. The vehicle control device according to claim 11, the first display includes a display in which at least a part of the teacher path is superimposed on the surrounding image acquired by the camera; Vehicle control device.

15. The vehicle control device according to claim 14, The first display includes a display in which the parking target position is superimposed on the surrounding image acquired by the camera. Vehicle control device.

16. The vehicle control device according to claim 11, The second display includes a display in which at least a part of the teacher path is superimposed on the surrounding image acquired by the camera. Vehicle control device.

17. The vehicle control device according to claim 11, The second display includes a display indicating the parking target position. Vehicle control device.

18. The vehicle control device according to claim 16, The second display includes a display in which the parking target position is superimposed on the surrounding image acquired by the camera. Vehicle control device.

19. The vehicle control device according to claim 11, The vehicle further includes a speaker that emits sound toward the passenger, When the vehicle is manually driven by the passenger and the vehicle is located within the predetermined range from the second position on the instructor route, the display device displays the first display indicating that the vehicle is near a location where automatic parking is possible, and the speaker emits a sound indicating that the vehicle is near a location where automatic parking is possible. Vehicle control device.

20. The vehicle control device according to claim 11, the vehicle further includes a position detection device that detects at least latitude and longitude; the information relating to the teacher route includes latitude and longitude information of the teacher route, determining whether the vehicle is located within a predetermined range from the second position on the teacher route while the vehicle is being manually driven by the passenger, based on the latitude and longitude information of the teacher route and the latitude and longitude detected by the position detection device; Vehicle control device.

Images