Parking support method and parking support device

Abstract

To support parking of a vehicle properly in an intended parking mode.SOLUTION: A parking support method according to the disclosure includes: storing information related to a teacher path of teacher traveling in which a vehicle is moved and parked in a first parking mode; receiving a change instruction of the parking mode which specifies a second parking mode different from the first parking mode; and correcting the teacher path based on the second parking mode.SELECTED DRAWING: Figure 7

Description

[Technical field]

[0001] The present disclosure relates to a parking assistance method and a parking assistance device. [Background technology]

[0002] Conventionally, parking assistance technologies that move a vehicle by automatic driving when parking the vehicle are known. One such parking assistance technology is a technology that learns a driving route based on teacher driving by the driver and performs parking assistance using the learning results. This technology is used when parking in a fixed parking position repeatedly, such as a parking lot at the user's home or workplace. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent No. 6022447 Summary of the Invention [Problem to be solved by the invention]

[0004] However, in such parking assistance, even if the parking position or attitude during teacher driving is not as desired by the user, for example, the vehicle is parked at an angle to the parking space, the vehicle is always parked in the undesired parking manner during automatic driving. On the other hand, redoing teacher driving until the vehicle can be parked in the desired parking manner is a burden for the user.

[0005] One of the problems to be solved by the present disclosure is to appropriately assist parking of a vehicle in a desired parking manner. [Means for solving the problem]

[0006] The parking assistance method of the present disclosure includes storing information regarding a teacher route, which is a driving route in a teacher drive in which a vehicle is moved and stopped in a first parking mode, accepting an instruction to change the parking mode specifying a second parking mode different from the first parking mode, and correcting the teacher route based on the second parking mode. Effect of the Invention

[0007] According to the present disclosure, it is possible to appropriately assist in parking a vehicle in a desired parking position. [Brief description of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram illustrating an example of a vehicle equipped with a parking assistance device according to an embodiment. [Diagram 2] FIG. 2 is a diagram showing an example of a configuration in the vicinity of a driver's seat of a vehicle according to an embodiment. [Diagram 3] FIG. 3 is a diagram illustrating an example of a hardware configuration of the parking assistance device according to the embodiment. [Figure 4] FIG. 4 is a diagram illustrating an example of a functional configuration of the parking assistance device according to the embodiment. [Diagram 5] FIG. 5 is a diagram showing an example of a display screen displayed in the support process according to the embodiment. [Figure 6] FIG. 6 is a diagram for explaining a route correction in response to a designated parking mode according to the embodiment. [Figure 7] FIG. 7 is a diagram for explaining a route correction in response to a designated parking mode according to the embodiment. [Figure 8] FIG. 8 is a diagram showing an example of a display screen displayed in the support process according to the embodiment. [Figure 9] FIG. 9 is a flowchart showing an example of the flow of the route correction process executed by the parking assistance device according to the embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, embodiments of a parking assistance device, a vehicle, a parking assistance method, a program, and a recording medium according to the present disclosure will be described with reference to the drawings.

[0010] In the description of the present disclosure, components having the same or substantially the same functions as those described above with respect to the previously-mentioned drawings may be given the same reference numerals, and the description may be omitted as appropriate. In addition, even when the same or substantially the same parts are shown, the dimensions and ratios of the components may be different depending on the drawing. In addition, for example, from the viewpoint of ensuring the visibility of the drawings, reference numerals may be given only to the main components in the description of each drawing, and reference numerals may not be given to components having the same or substantially the same functions as those described above with respect to the previously-mentioned drawings.

[0011] In the description of the present disclosure, components having the same or substantially the same functions may be distinguished by adding an alphanumeric character to the end of the reference symbol. Alternatively, when multiple components having the same or substantially the same functions are not distinguished, they may be collectively described by omitting the alphanumeric character at the end of the reference symbol.

[0012] Fig. 1 is a diagram illustrating an example of a vehicle 1 equipped with a parking assistance device 3 according to an embodiment. As shown in Fig. 1, the vehicle 1 has a vehicle body 12 and two pairs of wheels 13 arranged along a predetermined direction on the vehicle body 12. The two pairs of wheels 13 include a pair of front tires 13f and a pair of rear tires 13r.

[0013] Here, the front tire 13f according to the embodiment is an example of a first wheel. Also, the rear tire 13r according to the embodiment is an example of a second wheel. Note that, although FIG. 1 illustrates a vehicle 1 having four wheels 13, the present invention is not limited to this. The vehicle 1 may have at least one front tire 13f and at least one rear tire 13r. The number of wheels 13 of the vehicle 1 may be two, three, five or more.

[0014] The orientation of at least one of the wheels 13 of the vehicle 1 (steered wheel) is electrically or mechanically linked to the rotation angle, i.e., steering angle, of a steering wheel disposed in front of the driver's seat 130a. In other words, the vehicle 1 can turn right or left by steering. The steered wheel may be the rear tire 13r, or both the front tire 13f and the rear tire 13r.

[0015] The vehicle body 12 is supported by wheels 13. The vehicle 1 has a driving machine (not shown) and can move by driving at least one wheel (driving wheel) of the wheels 13 of the vehicle 1 with the power of the driving machine. Any driving machine can be used as the driving machine, such as an engine fueled by gasoline or hydrogen, a motor using power from a battery, or a combination of an engine and a motor. In this case, the predetermined direction in which the two pairs of wheels 13 are arranged is the traveling direction of the vehicle 1. The vehicle 1 can move forward or backward by switching gears (not shown) or the like.

[0016] The vehicle body 12 has a front end portion F which is an end portion on the front tire 13f side, and a rear end portion R which is an end portion on the rear tire 13r side. The vehicle body 12 has a substantially rectangular shape when viewed from above, and the four corners of the substantially rectangular shape are sometimes referred to as ends.

[0017] A pair of bumpers 14 are provided at the front and rear ends F, R of the vehicle body 12 near the lower end of the vehicle body 12. Of the pair of bumpers 14, a front bumper 14f covers the entire front surface and part of the side surface near the lower end of the vehicle body 12. Of the pair of bumpers 14, a rear bumper 14r covers the entire rear surface and part of the side surface near the lower end of the vehicle body 12.

[0018] Here, a configuration in the vicinity of the driver's seat of the vehicle 1 according to this embodiment will be described. Fig. 2 is a diagram showing an example of a configuration in the vicinity of the driver's seat 130a of the vehicle 1 according to this embodiment.

[0019] The vehicle 1 has at least one seat 130, as illustrated in Fig. 2. Fig. 2 illustrates a driver's seat 130a and a passenger seat 130b as the seat 130. In addition, in front of the driver's seat 130a, a steering wheel 140, a windshield 180, a dashboard 190, a display device 221, and operation buttons 222a and 222b are provided.

[0020] The steering wheel 140 is provided in front of the driver's seat 130a and can be operated by the driver. The rotation angle of the steering wheel 140, i.e., the steering angle, and the change in the direction of the front tires 13f, which are the steered wheels, are electrically or mechanically linked. The steered wheels may be the rear tires 13r, or both the front tires 13f and the rear tires 13r.

[0021] 1, the vehicle 1 is equipped with a parking assistance device 3. The parking assistance device 3 is an information processing device that can be installed in the vehicle 1, and is realized by, for example, an ECU (Electronic Control Unit) or an OBU (On Board Unit) provided inside the vehicle 1. Alternatively, the parking assistance device 3 may be an external computer installed near the dashboard 190 of the vehicle 1, or may also function as another device such as a car navigation device.

[0022] Fig. 3 is a diagram showing an example of a hardware configuration of the parking assistance device 3 according to the embodiment. As shown in Fig. 3, the parking assistance device 3 has a CPU (Central Processing Unit) 31, a ROM (Read Only Memory) 32, a RAM (Random Access Memory) 33, a HDD (Hard Disk Drive) 34, and an I / F (Interface) 35. The CPU 31, the ROM 32, the RAM 33, the HDD 34, and the I / F (Interface) 35 are interconnected by a bus 39 or the like, and have a hardware configuration using a normal computer.

[0023] The CPU 31 is a calculation device that controls the entire parking assistance device 3. The CPU 31 loads programs stored in the ROM 32 or the HDD 34 into the RAM 33 and executes them to realize various processes that will be described later.

[0024] The CPU 31 according to the embodiment is an example of a processor in the parking assistance device 3. As the processor, another processor may be provided instead of the CPU 31 or in addition to the CPU 31. As the other processor, various processors such as a GPU (Graphics Processing Unit) and a DSP (Digital Signal Processor), a dedicated arithmetic circuit realized by an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array), etc. can be appropriately used.

[0025] The ROM 32 stores programs and parameters for implementing various processes by the CPU 31.

[0026] The RAM 33 is, for example, a main storage device of the parking assistance device 3, and temporarily stores data necessary for various processes performed by the CPU 31.

[0027] The HDD 34 stores various data, programs, and the like used by the parking assistance device 3. As an example, the HDD 34 stores information obtained from on-board sensors mounted on the vehicle 1, such as the sonar 211, the all-around camera 212, the orientation sensor 213, and the GNSS sensor 214. Note that instead of the HDD 34 or in addition to the HDD 34, various storage media and storage devices, such as an SSD (Solid State Drive) and a Flash memory, can be appropriately used.

[0028] The I / F 35 is an interface for transmitting and receiving data. The I / F 35 receives data from other devices provided in the vehicle 1, such as an on-board sensor of the vehicle 1 and the HMI 22. The I / F 35 also transmits data to other devices provided in the vehicle 1, such as the HMI 22, the steering control device 23, and the drive / brake control device 24. The I / F 35 also acquires signals from an accelerator sensor (not shown) that detects the amount of accelerator pedal operation by the driver and a brake sensor (not shown) that detects the amount of brake pedal operation by the driver, or the amount of operation based on these signals.

[0029] The I / F 35 may transmit and receive information to and from other ECUs mounted on the vehicle 1 via an in-vehicle network including a Controller Area Network (CAN) or Ethernet (registered trademark) in the vehicle 1, or may communicate with an information processing device outside the vehicle 1 via a network such as the Internet. For example, the parking assistance device 3 is connected to the sonar 211, the all-around camera 212, the orientation sensor 213, the GNSS sensor 214, the HMI 22, the steering control device 23, and the drive / brake control device 24 of the vehicle 1, respectively, via the in-vehicle network.

[0030] As an example, I / F35 may acquire information regarding the state of vehicle 1, such as vehicle speed pulses, various speeds including yaw rate, acceleration, position information, and shift information, from other ECUs or various on-board sensors of vehicle 1, for example via an on-board network.

[0031] The vehicle 1 is equipped with a number of on-board sensors. Figures 1 and 3 illustrate a sonar 211 and an all-around camera 212. Figure 3 further illustrates an orientation sensor 213 and a GNSS sensor 214.

[0032] The sonar 211 is provided, for example, at a predetermined end of the vehicle body 12, and transmits and receives sound waves such as ultrasonic waves. The sonar 211 includes wave transmitting and receiving units 211f, 211r. For example, one or more wave transmitting and receiving units 211f are disposed on the front bumper 14f, and one or more wave transmitting and receiving units 211r are disposed on the rear bumper 14r. The number and / or positions of the wave transmitting and receiving units 211f, 211r are not limited to the example shown in FIG. 1 and can be changed as appropriate. For example, the vehicle 1 may be provided with the wave transmitting and receiving units 211f, 211r on the left and right sides.

[0033] Based on the results of transmitting and receiving sound waves, the sonar 211 detects obstacles around the vehicle 1. Also, based on the results of transmitting and receiving sound waves, the sonar 211 measures the distance between the vehicle 1 and obstacles around the vehicle 1.

[0034] In the present embodiment, the sonar 211 uses sound waves such as ultrasonic waves, but is not limited to this. For example, the vehicle 1 may have a radar that transmits and receives electromagnetic waves instead of or in addition to the sonar 211.

[0035] The all-around camera 212 is provided on the vehicle 1 so as to be able to capture images of the surroundings of the vehicle 1. As an example, the vehicle 1 has, as the all-around camera 212, a front camera 212a that captures images in front, a rear camera 212b that captures images in the rear, a left side camera 212c that captures images on the left side, and a right side camera (not shown) that captures images on the right side.

[0036] The all-around camera 212 captures images of the surroundings of the vehicle 1. The all-around camera 212 is, for example, a camera that captures images based on visible light and / or infrared light. Note that the images captured by the all-around camera 212 may be moving images or still images.

[0037] The position and / or number of the all-around camera 212 is not limited to the example shown in FIG. 1 and can be changed as appropriate. For example, the vehicle 1 may have only two cameras, the front camera 212a and the rear camera 212b. Alternatively, the vehicle 1 may have other cameras in addition to the above-mentioned examples. For example, a part or all of the all-around camera 212 may be provided in the vehicle cabin space (inside the vehicle) of the vehicle 1. Furthermore, the all-around camera 212 may be a camera built into the vehicle 1, or may be a camera of a drive recorder attached to the vehicle 1.

[0038] The orientation sensor 213 detects the heading of the traveling direction of the vehicle 1. The orientation sensor 213 is a sensor that measures the heading of the traveling direction of the vehicle 1 from, for example, a rotation difference between the left and right wheels 13 of the vehicle 1, geomagnetism, a gas rate gyro, an optical fiber gyro, or the like. Note that the method for determining the heading of the traveling direction of the vehicle 1 can be a known method and is not particularly limited.

[0039] The GNSS sensor 214 detects position information indicating the current position of the vehicle 1. As an example, the GNSS sensor 214 is a Global Navigation Satellite System (GNSS) sensor such as a Global Positioning System (GPS) sensor that outputs position information of the vehicle 1. The GNSS sensor 214 includes a GNSS antenna that receives radio waves (GNSS signals) from satellites, and a GNSS circuit that obtains position information (GNSS coordinates) based on the radio waves from at least two satellites received by the GNSS antenna.

[0040] The vehicle 1 may have other on-board sensors not shown. As an example, the vehicle 1 may have a steering angle sensor that outputs a signal corresponding to the amount of operation of the steering wheel 140 by the driver, i.e., the steering angle. As an example, the vehicle 1 may have a wheel speed sensor that outputs a signal corresponding to the rotation speed and rotation direction of the wheels 13. As an example, the vehicle 1 may have a brake sensor that detects the amount of operation of the brake pedal by the driver. As an example, the vehicle 1 may have an accelerator sensor that detects the amount of operation of the accelerator pedal by the driver. As an example, the vehicle 1 may have an acceleration sensor that outputs a signal corresponding to the acceleration applied to the vehicle 1. As an example, the vehicle 1 may have a gyro sensor that outputs a signal corresponding to the angular velocity applied to the vehicle 1.

[0041] The HMI 22 is an interface for outputting various types of information, such as notifications and warnings, to the driver of the vehicle 1. The HMI 22 is also an interface for accepting input of various types of information by the driver of the vehicle 1. The HMI 22 only needs to be able to output notifications and warnings recognizable by the driver of the vehicle 1 and accept various operations by the driver of the vehicle 1, and is provided, for example, around the driver's seat of the vehicle 1, but may also be provided in another portion around the driver's seat, such as the back seat.

[0042] The HMI 22 includes a display device that outputs various types of information, such as notifications and warnings, to the driver of the vehicle 1. As an example, the HMI 22 includes a display device 221 that is provided on the dashboard 190 or console of the vehicle 1 and is configured to be able to output images. Fig. 2 illustrates the display device 221 disposed in the center of the dashboard 190. The display device 221 is, for example, a liquid crystal display (LCD) or an organic EL (Electro Luminescence) display.

[0043] Here, the HMI 22 as a display device according to the embodiment is an example of a display unit. The HMI 22 as a display unit displays various screens based on various display information from an output control unit 309, which will be described later.

[0044] The display device 221 may be configured as a touch panel display. The display device 221 may be a part of a car navigation device mounted on the vehicle 1. The display device may be a projection-type display device such as a Head Up Display (HUD) that projects an image (virtual image) onto a display area provided in front of the driver, for example, on the windshield or dashboard (console).

[0045] The HMI 22 is not limited to a display device, and may include other output devices such as a speaker configured to output notification sounds, warning sounds, and audio.

[0046] The HMI 22 also includes an input device that accepts various information input by the driver of the vehicle 1. As an example, the HMI 22 includes, as an input device, a touch panel of the display device 221 configured as a touch panel display. As an example, the HMI 22 includes operation buttons 222a and 222b as input devices. FIG. 2 illustrates an operation button 222a arranged on an instrument panel and an operation button 222b provided on a console. The operation buttons 222a and 222b may be provided in other positions such as the steering wheel 140 or the dashboard 190. In addition, when another input device is present, such as when the touch panel of the display device 221 can be used as an input device for the HMI 22, the operation buttons 222a and 222b may not be provided.

[0047] Here, the HMI 22 as an input device according to the embodiment is an example of an operation unit. The HMI 22 as an operation unit outputs signals corresponding to various operations from the user to a later-described reception unit 302 in response to various operations on the touch panel or operation buttons 222a and 222b of the display device 221 from the user.

[0048] The HMI 22 may include other input devices such as other buttons, dials, switches, a microphone, etc. These input devices are arranged, for example, on the dashboard 190, the instrument panel, the steering wheel 140, or a console of the vehicle 1.

[0049] In addition, the output device, input device or input / output device of HMI22 may be an operating terminal capable of receiving, transmitting or transmitting signals to vehicle 1 from outside vehicle 1, such as a tablet terminal, smartphone, remote controller, or electronic key.

[0050] The steering control device 23 controls the steering of the vehicle 1. For example, the steering control device 23 deflects the wheels 13 in a direction according to a control signal according to an amount of operation of the steering wheel 140 by the driver or a control signal from the parking assistance device 3. The steering control device 23 may have a steering actuator (not shown) that changes the rotation angle of the steering wheel 140 according to a control signal from the parking assistance device 3.

[0051] The drive / brake control device 24 controls the acceleration / deceleration of the vehicle 1. The drive / brake control device 24 has, for example, a brake actuator (not shown) and an engine controller (not shown). The brake actuator applies brakes, changes a shift (gear ratio), and controls the output of a drive machine such as an engine or a motor based on the detection result of a brake sensor (not shown) that detects the amount of operation of the brake pedal by the driver or a control signal from the parking assistance device 3, thereby braking the vehicle 1 or decelerating the vehicle 1. The accelerator controller accelerates the vehicle 1 by controlling the output of a drive machine such as an engine or a motor based on the detection result of an accelerator sensor (not shown) that detects the amount of operation of the accelerator pedal by the driver or a control signal from the parking assistance device 3.

[0052] 3 illustrates a case in which the sonar 211, the all-around camera 212, the orientation sensor 213, the GNSS sensor 214, the HMI 22, the steering control device 23, and the drive / brake control device 234 are not included in the parking assistance device 3, but the present invention is not limited to this. Some or all of these may be included in the parking assistance device 3.

[0053] Fig. 4 is a diagram showing an example of a functional configuration of the parking assistance device 3 according to the embodiment. The parking assistance device 3 realizes functions of an acquisition unit 301, a reception unit 302, a map information generation unit 303, a driving route detection unit 304, a storage unit 305, a self-position estimation unit 306, a route following calculation unit 307, a parking mode correction unit 308, an output control unit 309, and a vehicle control unit 310 by executing a program loaded in the RAM 33 by the CPU 31.

[0054] Here, the acquisition unit 301 and the reception unit 302 according to the embodiment are each an example of a reception unit. Also, the acquisition unit 301, the map information generation unit 303, and the driving route detection unit 304 according to the embodiment are each an example of a learning unit. Also, the self-position estimation unit 306, the route following calculation unit 307, and the vehicle control unit 310 according to the embodiment are each an example of a vehicle control unit. Also, the parking mode correction unit 308 according to the embodiment is an example of a correction unit. Also, the output control unit 309 according to the embodiment is an example of a display control unit.

[0055] The acquisition unit 301 acquires data from other devices provided in the vehicle 1, such as on-board sensors such as the sonar 211, the all-around camera 212, the orientation sensor 213, and the GNSS sensor 214, via the I / F 35, for example.

[0056] As an example, the acquisition unit 301 acquires outputs from external sensors that detect spatial information around the vehicle 1, such as the sonar 211 and the omnidirectional camera 212.

[0057] As an example, the acquisition unit 301 acquires outputs from vehicle sensors, such as the direction sensor 213 and the GNSS sensor 214, that detect the distance, speed, and direction traveled by the vehicle 1.

[0058] The reception unit 302 receives various operations from the user based on the output of the HMI 22, such as the display device 221 and the operation buttons 222a and 222b, in response to various operations from the user.

[0059] As an example, when the operation button 222a is pressed while the vehicle 1 is stopped, the reception unit 302 receives the operation button 222a as an instruction to call a parking assistance function, which is a menu display on a display such as the display device 120. Note that, instead of pressing the operation button 222a, pressing the operation button 222b or touching an icon on the touch panel of the display device 221 configured as a touch panel display may be received as an instruction to call a parking assistance function.

[0060] As an example, when a touch operation is performed on an operation icon on the display screen of the assistance information or operation buttons 222a, 222b are pressed after an instruction to call up the parking assistance function is received, the reception unit 302 receives the operation as an instruction to start teacher driving, an instruction to start route correction, an instruction to change the parking mode, or an instruction to start playback driving.

[0061] For example, the reception unit 302 receives a parking mode change instruction that specifies a parking mode 601b (see FIG. 7) different from the parking mode 601a (see FIG. 7) of the teacher route based on the output of the HMI 22 in response to a user operation.

[0062] In the present disclosure, the parking assistance function includes a teacher driving function for acquiring a teacher route by teacher driving, a route correction function for correcting the teacher route, and a playback driving function for moving the vehicle 1 along the teacher route. The teacher driving function is realized by a teacher driving process executed in response to an instruction to start teacher driving. The route correction function is realized by a route correction process executed in response to an instruction to end teacher driving or to start route correction. In the route correction process, when an instruction to change the parking mode by the user is accepted, the teacher route is corrected according to the parking mode specified by the user. The playback driving function is realized by a playback driving process executed in response to an instruction to start playback driving.

[0063] In this disclosure, teacher driving refers to manual or automatic driving in which the driver moves the vehicle 1 from the parking start position to the parking end position and parks the vehicle 1 in a desired parking manner. The parking manner refers to at least one of the parking position and the parking attitude (orientation) of the vehicle 1. The teacher route refers to the driving route of the vehicle 1 detected in teacher driving, that is, the driving trajectory of the vehicle 1 in teacher driving. In other words, the teacher route includes information indicating the parking manner. The playback driving refers to automatic driving in which the vehicle 1 moves along the teacher route.

[0064] During teacher driving, the map information generating unit 303 acquires a surrounding environment map that defines the environment around the vehicle 1, i.e., the surrounding environment, which is a space that spreads along the driving route of the vehicle 1, as a map. For example, the map information generating unit 303 extracts feature points from each of the captured images of the front, rear, left, and right of the vehicle 1, i.e., the surrounding images. The map information generating unit 303 defines the surrounding environment of the vehicle 1 based on the extracted feature points, and acquires it as a surrounding environment map. Here, the surrounding environment map is an example of information about the driving route. The surrounding environment map is also information about the surrounding environment, which is a space that spreads along the driving route, and is an example of spatial information.

[0065] The method for extracting the feature points is not particularly limited, and any known method may be applied. For example, the map information generating unit 303 extracts feature points from the surrounding image using a method such as FAST (Features from Accelerated Segment Test) or ORB (Oriented FAST and Rotated BRIEF). The map information generating unit 303 may preferentially record feature points that satisfy a specified condition among the feature points extracted from the surrounding image. For example, among a plurality of surrounding images consecutive in time series, the feature points extracted from the image in which the vehicle 1 traveled a longer distance during shooting may be preferentially selected as feature points.

[0066] For example, the map information generating unit 303 uses a Simultaneous Localization and Mapping (SLAM) technique for the extracted feature points to detect coordinates indicating the position of each feature point and create a surrounding environment map (local map).

[0067] When detecting the position coordinates of each feature point, the position of the vehicle 1 is detected in parallel, but the position detection of the vehicle 1 may be performed by combining the SLAM technology with the Dead-Reckoning (DR) technology. The position, speed, and moving direction of the vehicle 1 may be calculated using an Intelligent Transport Systems (ITS).

[0068] The surrounding environment map may be defined by spatial information based on the output of the sonar 15, in addition to spatial information based on the captured images.

[0069] The travel route detection unit 304 detects a teacher route of the vehicle 1, i.e., a travel route in teacher travel, based on a self-position estimation result detected by, for example, SLAM technology and dead reckoning technology during teacher travel. When detecting the travel route, the travel route is represented as a set of points (waypoints) that exist at approximately equal distance intervals, and each point is detected as route point cloud data that holds, as parameters, coordinates indicating a position, a direction indicating the direction of the vehicle 1 at that position, and a travel speed. The detected teacher route is stored in the storage unit 305.

[0070] As an example, the travel route detection unit 304 detects the teacher route of the vehicle 1 based on the outputs of the orientation sensor 213 and the GNSS sensor 214 acquired by the acquisition unit 301. As an example, the travel route detection unit 304 detects the teacher route of the vehicle 1 based on the rotation direction and rotation speed of each wheel 13 acquired by the acquisition unit 301. Note that the speed and moving direction of the vehicle 1 may be acquired using vehicle speed pulses and shift information flowing through an in-vehicle network such as CAN.

[0071] The storage unit 305 is composed of, for example, the ROM 32, the RAM 33, or the HDD 34. The storage unit 305 stores the surrounding environment map generated by the map information generation unit 303 and the teacher route detected by the travel route detection unit 304. In other words, the storage unit 305 stores information about the teacher route, which is the travel route in the teacher travel in which the vehicle 1 is moved and stopped in the parking mode 601a (see FIG. 7). In addition, when the parking mode correction unit performs a route correction, the storage unit 305 stores the corrected teacher route. In addition, the storage unit 305 stores a constraint condition regarding the change amount of the vehicle mode between predetermined adjacent waypoints. The constraint condition is a condition that specifies the difference in the position (coordinate) that can be corrected between adjacent waypoints and the difference in the angle (yaw angle) that can be corrected between adjacent waypoints. In addition, the storage unit 305 stores correspondence information indicating the correspondence between the number of waypoints calculated to require correction and the number of waypoints to be corrected by the optimization process. The correspondence information may be information indicating a constant to be added to the calculated number of waypoints, or information indicating a coefficient to be multiplied by the calculated number of waypoints. Furthermore, these constants and coefficients may have multiple values ​​defined according to the value or category of the calculated number of waypoints.

[0072] The self-position estimation unit 306 estimates the position and orientation of the vehicle 1 when the vehicle 1 performs playback driving based on the teacher route by the vehicle control unit 310 described later. As an example, the self-position estimation unit 306 reads out feature point information of the surrounding environment map stored in the storage unit 305, and compares it with feature points extracted from the current surrounding image acquired during playback driving, thereby estimating the position and orientation of the vehicle 1.

[0073] The position and orientation of the vehicle 1 estimated by the self-position estimation unit 306 may be a relative position and relative direction with respect to a reference position and reference direction such as the starting position and starting direction in the acquired driving route, but an absolute position and absolute direction based on the output of the orientation sensor 213 or the GNSS sensor 214 may also be used.

[0074] In addition, in each of the teacher driving and playback driving, the position of vehicle 1 is not limited to being obtained using dead reckoning technology based on the amount of movement of vehicle 1 from a reference position, but may also be obtained using known map matching or GNSS sensor 214.

[0075] In playback traveling, the route following calculation unit 307 calculates a junction route, which is a traveling route from the current position of the vehicle 1 to junction with the teacher route, based on the teacher route stored in the storage unit 305 and the position of the vehicle 1 estimated by the self-position estimation unit 306. The calculated junction route is stored in the storage unit 305.

[0076] The parking mode correction unit 308 corrects the teacher path based on the parking mode specified by the user. The correction of the teacher path based on the parking mode specification will be described later. The corrected teacher path is stored in the memory unit 305.

[0077] The output control unit 309 generates support information related to the parking mode. Moreover, the output control unit 309 presents the generated support information to the user via the HMI 22. Specifically, the output control unit 309 outputs information for displaying the support information related to the parking mode via the HMI 22 as support information to the HMI 22. Presentation of the support information by the HMI 22 will be described later (see Figs. 5 and 8).

[0078] The vehicle control unit 310 controls the playback driving to move the vehicle 1 along the teacher route and stop the vehicle 1 in a predetermined parking mode based on the information on the teacher route. Specifically, the vehicle control unit 310 controls the steering, braking, and acceleration / deceleration of the vehicle 1 to follow the teacher route. As an example, the vehicle control unit 310 detects a difference between the merging route and the teacher route and the position and direction of the vehicle 1 based on the merging route and the teacher route stored in the storage unit 305 and the position and direction of the vehicle 1 estimated by the self-position estimation unit 306, and controls at least one of the control amounts of the steering, braking, and acceleration / deceleration of the vehicle 1 so as to reduce the detected difference, thereby making the vehicle 1 follow the merging route and the teacher route and controlling the playback driving.

[0079] Here, a description will be given of the presentation of support information by the HMI 22. Fig. 5 is a diagram showing an example of a display screen 401 displayed in the parking support process according to the embodiment. Fig. 5 illustrates an example of the display of support information related to a parking space 405 defined by a wall 403.

[0080] The display screen 401 includes a representation 407 of the parking behavior before correction.

[0081] As an example, the display screen 401 is displayed when the teacher driving ends. The display 407 on the display screen 401 may be a surround view image showing the current parking state. For example, the output control unit 309 recognizes the end of the teacher driving when the reception unit 302 receives a user operation instructing the end of the teacher driving, or when the stopping time of the vehicle 1 during the teacher driving exceeds a predetermined threshold value.

[0082] Here, the surround view image is an overhead image of the vehicle 1 generated by synthesizing images of the surroundings of the vehicle 1 (for example, images of the front, rear, left and right) obtained by the omnidirectional camera 212.

[0083] As an example, the display screen 401 is displayed when a user operation to instruct the start of route correction is accepted by the accepting unit 302. The display 407 on the display screen 401 may be a surround view image showing the parking state at the end of the teacher driving, which is acquired at the end of the teacher driving and stored in the storage unit 305.

[0084] In addition, there may be cases where the teacher route has already been corrected in response to the designation of the parking mode. In this case, the display 407 on the display screen 401 may be a surround view image showing the parking mode designated in the latest route correction, i.e., the parking mode of the latest teacher route, stored in the storage unit 305. In addition, the surround view image may be an image in which the parking mode of the latest teacher route is superimposed on the surround view image before correction, similar to the display 409 for designating the parking mode described later. Alternatively, in the case where playback travel based on the latest teacher route has already been performed, the surround view image may be a surround view image acquired at the end of the playback travel and stored in the storage unit 305.

[0085] Moreover, the display screen 401 further includes a display 409 for designating the parking mode.

[0086] As an example, the display 409 on the display screen 401 is a display that simulates the parking state after correction. Fig. 5 illustrates a case in which the position and attitude of the vehicle 1 after correction are pseudo-expressed by displaying a frame.

[0087] In this manner, the output control unit 309 according to the embodiment outputs display information to the HMI 22 for displaying the display screen 401, which is a display screen for specifying the parking mode desired by the user and includes a display 407 showing the parking mode of the teacher route and a display 409 simulating the desired parking mode as the operation target.

[0088] The display 409 may include displays of input elements such as sliders, dials, and numerical input fields for indicating the corrected position and attitude of the vehicle 1. In other words, the parking mode may be specified by moving or rotating a display simulating the corrected parking mode, such as a frame, on the touch panel, or by operating the display of the input elements displayed on the touch panel.

[0089] Each of the displays 407 and 409 is not limited to the example in Fig. 5, and may be a CG image or icon of the vehicle, or may be a frame. The display colors, line types, and transparency may be different between the displays 407 and 409. By changing the display mode in this way, it is possible to prevent the display contents from becoming difficult to see, and to facilitate the designation of the parking mode, for example, by allowing the user to easily understand which display to operate to designate the ideal parking mode.

[0090] As an example, the output control unit 309 may output support information for displaying the parking state before correction by a CG image of the vehicle and pseudo-displaying the parking state after correction by a frame. As an example, the output control unit 309 may output support information for displaying the pseudo-display of the parking state after correction in a color different from the parking state before correction. As an example, the output control unit 309 may output support information for displaying the parking state before correction semi-transparently and pseudo-displaying the parking state after correction opaquely.

[0091] Here, correction of the teacher route based on the designation of the parking mode will be described. Figures 6 and 7 are diagrams for explaining the route correction according to the designation of the parking mode according to the embodiment.

[0092] Fig. 6 relates to parking assistance for parking a vehicle 1 in a parking space 503 defined by a wall 501. Fig. 6 illustrates, with solid lines, a parking start position 511a, a forward section 521a, a turning back position 513a, a reversing section 523a, and a parking end position 515a for a teacher path before correction. Fig. 6 also illustrates, with dashed lines, a parking start position 511b, a forward section 521b, a turning back position 513b, a reversing section 523b, and a parking end position 515b for a teacher path after correction in accordance with a designated parking mode.

[0093] FIG. 7 illustrates some waypoints Pa0, Pa1, Pa2, Pa3, Pa4, and Pa5 among the multiple waypoints that define the retreat portion 523a for the teacher route before correction (dashed line in FIG. 7). The waypoint Pa0 corresponds to the parking end position 515a on the teacher route before correction and represents the parking mode 601a. ​​Here, the waypoint Pa0 according to the embodiment is an example of the first waypoint for the teacher route detected by the teacher driving. Also, the parking mode 601a according to the embodiment is an example of the first parking mode. Note that, in a case where a parking mode is further specified for the teacher route that has been subjected to the route correction, the waypoint Pa0 and the parking mode 601a are examples of the second waypoint and the second parking mode, respectively. Also, FIG. 7 illustrates some waypoints Pb0, Pb1, Pb2, Pb3, Pb4, and Pb5 among the multiple waypoints that define the retreat portion 523b for the teacher route after correction in accordance with the specification of the parking mode (solid line in FIG. 7). The waypoint Pb0 corresponds to the parking end position 515b on the corrected teacher route, and represents the parking mode 601b specified by the user. Here, the waypoint Pb0 according to the embodiment is an example of the second waypoint with respect to the teacher route detected by the teacher driving. Also, the parking mode 601b according to the embodiment is an example of the second parking mode. Note that, with respect to the case where a parking mode is further specified for the teacher route that has been subjected to the route correction, the waypoint Pb0 and the parking mode 601b are examples of the fourth waypoint and the third parking mode, respectively.

[0094] Here, we will mainly exemplify a case where a parking mode change instruction is given based on a user operation that specifies a desired parking mode with respect to the teacher route detected by teacher driving. Specifically, we will exemplify a case where a user specifies a parking mode 601b by moving and rotating the display 409 on the display screen 401 of Fig. 5 including the display 407 showing the parking mode 601a. ​​Also, we will exemplify a case where some of the waypoints that define the retreat portion 523a of the teacher route are specified as the waypoint 701 to be corrected.

[0095] The reception unit 302 receives the designation of the parking mode 601b, that is, the instruction to change the parking mode, based on the output of the HMI 22 in response to the user's operation. By designating the ideal parking mode by the user after the teacher driving is completed, the position (coordinates) on the local map is uniquely determined.

[0096] In general, it is assumed that the teacher driving by the user reflects the user's ideal, such as places that the user does not want to pass through in that environment. For this reason, the parking mode correction unit 308 corrects the teacher route together with the parking mode.

[0097] If the corrected teacher path includes a sudden change in posture, it becomes difficult for the vehicle 1 to follow the teacher path during automatic parking. For this reason, the parking mode correction unit 308 corrects the teacher path gradually, i.e. gently, until the parking end position 515b of the corrected teacher path reaches the parking mode 601b specified by the user.

[0098] Specifically, the parking mode correction unit 308 corrects the teacher route within a range in which the change in the position and attitude of the vehicle 1 between adjacent waypoints, that is, the vehicle mode, satisfies a predefined constraint condition.

[0099] For example, the parking mode correction unit 308 calculates the number of waypoints required for the correction amount of the parking mode within a range that satisfies the constraint conditions, i.e., the number of waypoints that need correction, based on the correction amount of the parking mode, which is the difference between the position and attitude of the parking modes 601a and 601b, i.e., the difference between the waypoints Pa0 and Pb0, and the predefined constraint conditions.

[0100] Also, for example, the parking mode correction unit 308 may specify the waypoint that is the number of waypoints calculated from the parking end position 515a as the waypoint to be corrected. In other words, the parking mode correction unit 308 may specify the number of waypoints that need to be corrected from the waypoint Pa0 as the starting point as the waypoint 701 to be corrected, among the multiple waypoints that define the teacher route.

[0101] In this case, referring to the example of FIG. 7, since it is calculated that correction is required for the number of waypoints of four points, the waypoints 701 to be corrected are the waypoints Pa0, Pa1, Pa2, and Pa3. Here, the waypoint Pa3 according to the embodiment is an example of a third waypoint with respect to the teacher route detected in the teacher driving. Note that, with respect to the case where a parking mode is further specified for the teacher route to which the route correction has been applied, the waypoint Pa3 is an example of a fifth waypoint. On the other hand, the waypoints Pa4 and Pa5 are non-correction target waypoints 703 that do not need to be corrected. The non-correction target waypoints 703 are treated as fixed constraint conditions in the subsequent optimization process.

[0102] Also, for example, the parking mode correction unit 308 performs optimization processing on the waypoint 701 to be corrected so that the parking end position 515b of the corrected teacher route becomes the parking mode 601b specified by the user, and obtains waypoints Pb0, Pb1, Pb2, and Pb3. As the optimization processing, for example, Pose Graph Optimization (PGO) that optimizes only the trajectory by adjusting the posture based on the posture graph can be used, but other known optimization methods may be used as appropriate.

[0103] Also, for example, the parking mode correction unit 308 uses the original waypoint as is for the waypoint 703 that is not to be corrected. That is, the parking mode correction unit 308 uses the waypoints Pa4 and Pa5 as the waypoints Pb4 and Pb5. Therefore, in correcting the teacher route, the parking mode correction unit 308 specifies the teacher route by a plurality of waypoints including the waypoint 701 that is to be corrected after correction from the waypoint Pb0 to the waypoint Pb3 and the waypoint 703 that is not to be corrected.

[0104] In correcting the teacher path, the parking mode correction unit 308 may adjust the distance of the waypoint at which the correction starts, that is, the correction start position from the parking end position 515a, depending on the amount of correction of the parking mode.

[0105] For example, the parking mode correction unit 308 may specify the waypoint 701 to be corrected by adding an arbitrary number of waypoints according to the correspondence information to the number of waypoints that need to be corrected, which is calculated based on the correction amount of the parking mode and the constraint conditions that are specified in advance. In other words, the parking mode correction unit 308 may specify, as the waypoint 701 to be corrected, a waypoint that is the number of waypoints that need to be corrected or more, starting from the waypoint Pa0 as the starting point, among the multiple waypoints that define the teacher route.

[0106] In this case, referring to the example of Fig. 7, for example, it is calculated that correction is required for the number of waypoints of three, and four waypoints Pa0, Pa1, Pa2, and Pa3 are identified as waypoints 701 to be corrected according to the correspondence information. Note that, for simplicity of explanation, it has been described here that one waypoint is added, but this is not limited to this. The number of points to be increased may be a number of points equal to or greater than two. As an example, the number of waypoints may be appropriately determined, such as adding 10 waypoints to waypoints spaced 20 cm apart.

[0107] Here, the presentation of support information by the HMI 22 will be described again. Fig. 8 is a diagram showing an example of the display screen 411 displayed in the parking support process according to the embodiment. Fig. 8 illustrates an example of the display of support information related to a corrected teacher route (corrected route) corresponding to the parking mode 601b specified by the user.

[0108] For example, the output control unit 309 outputs, to the HMI 22, display information for displaying, for example, the display screen 411 as information indicating a corrected path corrected based on the parking mode 601b specified by the user. As an example, the display screen 411 is displayed when generation of a corrected path corresponding to the specified parking mode 601b is completed. As an example, the display screen 411 includes an overhead image in which the corrected path (broken line in FIG. 8) is superimposed on the surrounding image of the vehicle 1. FIG. 8 illustrates, as the surrounding image of the vehicle 1, a wall 413 and a parking space 415 defined by the wall 413. Note that the parking space 415 in FIG. 8 may correspond to the parking space 405 in FIG. 5. That is, the wall 403 in FIG. 5 may represent a part of the wall 413 in FIG. 8.

[0109] The display screen 411 also includes a representation 419 of the specified parking configuration 601b along with the corrected path. This representation 419 may be a surround view image showing the specified parking configuration 601b, or it may be another representation such as a frame representation, similar to the representation 407 of FIG.

[0110] As shown in FIG. 8, the display screen 411 may include a display of the teacher route before correction (solid line in FIG. 8) together with the corrected route. Also, the display screen 411 may display the parking mode 401a before correction similarly to the display 407 in FIG. 5. Alternatively, the display screen 411 may accept a designation of a parking mode for the display 419, and for example, a display for designating a parking mode may be displayed for the display 419 similarly to the display 409 in FIG. 5. That is, a screen similar to the display screen 411 may be used as a designation screen for a parking mode instead of FIG. 5. In other words, the display screen 401 in FIG. 5 may include a display of the teacher route.

[0111] As an example, the display screen 411 may be a screen that displays only the corrected route (dashed line in FIG. 8) that is the teacher route after correction. In other words, the display screen 411 does not necessarily have to display an image of the surroundings of the vehicle 1 including the wall 413, and may display, for example, an image in which the corrected route is superimposed on a black background. Of course, even if the image of the surroundings of the vehicle 1 is not displayed, the teacher route before correction (solid line in FIG. 8) may be displayed together with the corrected route.

[0112] In addition, in the playback travel processing that is performed after the route correction, a screen that displays the corrected route may be displayed similarly to the display screen 411 in FIG.

[0113] Next, a flow of the parking assistance process executed by the parking assistance device 3 configured as above will be described. As described above, the parking assistance process executed by the parking assistance device 3 includes the teacher driving process, the route correction process, and the playback driving process. Here, the route correction process executed after at least one teacher driving is described.

[0114] Fig. 9 is a flowchart showing an example of the flow of the route correction process executed by the parking assistance device 3 according to the embodiment. The flow in Fig. 9 illustrates the route correction process executed as a series of flows in conjunction with the teacher driving process. That is, the flow in Fig. 9 starts in a state where the vehicle 1 is stopped at the parking end position in the teacher driving process.

[0115] When the teacher driving ends, the output control unit 309 displays a screen for specifying the parking mode. For example, as shown in Fig. 5, the output control unit 309 generates display information for displaying a display screen 401 including a display 408 such as a surround view image at the end of the teacher driving and a display 409 for specifying the parking mode, and outputs the display information as support information.

[0116] The parking mode correction unit 308 judges whether a parking mode has been specified based on, for example, an output from the HMI 22 in response to a user operation on the display 409 (S102). Here, the "specification of a parking mode" refers to the result of a user operation that changes at least one of the position and the posture of the parking mode 601b simulated by the display 409 on the display screen 401 from the parking mode 601a at the end of the teacher driving.

[0117] If it is not determined that the parking mode is specified (S102: No), the parking mode correction unit 308 stores the driving route in the teacher driving, that is, the driving route for parking the vehicle 1 in the parking mode 601a at the end of the teacher driving, as the teacher route in the storage unit 305 (S103). After that, the flow of FIG. 9 ends.

[0118] On the other hand, if it is determined that the parking mode has been specified (S102: Yes), the parking mode correction unit 308 reads out the constraint conditions between adjacent waypoints from the storage unit 305 (S104).

[0119] In addition, the parking mode correction unit 308 calculates the number of waypoints that require correction based on the amount of correction to the specified parking mode and the read constraint conditions (S105).

[0120] In addition, the parking mode correction unit 308 executes an optimization process for the waypoint 701 to be corrected (S106).

[0121] In addition, the parking mode correction unit 308 stores the non-correction target waypoint 703 and the corrected correction target waypoint 701 as a corrected teacher route in the storage unit 305 (S107). After that, the flow of FIG. 9 ends.

[0122] Conventionally, a parking assistance system is known in which, during teacher driving, the driving route of the vehicle 1 by the user is stored as a teacher route, and during automatic parking, a merging route that merges with the stored teacher route is generated, and the vehicle 1 is moved to the parking end position during teacher driving and parked. In such a route-storing type automatic parking, a local map and route information expressed in the coordinate system of the map are created and stored at the end of teacher driving. Here, the parking position of the vehicle 1 during teacher driving is expressed by the waypoint at the end of the route information.

[0123] In route memory type automatic parking, as a result of performing teacher driving, the vehicle 1 may be parked in a parking manner that is not as desired by the user, such as being parked at an angle to the parking space. If the parking manner during teacher driving is not as desired by the user, the vehicle 1 will be parked in a parking manner that is not as desired every time automatic parking is performed. On the other hand, redoing teacher driving until the vehicle can be parked in the parking manner as desired is a burden for the user.

[0124] In this situation, the parking assistance device 3 according to the embodiment is configured so that, for example, when the teacher driving ends, the user can specify the ideal parking position and parking posture through a UI such as a surround view image, regarding the automatic correction function of the teacher driving route. By specifying the ideal parking posture after the teacher driving ends, the ideal position (coordinates) on the local map is uniquely determined.

[0125] Furthermore, the parking assistance device 3 according to the embodiment generates a teacher route that finally reaches the specified parking mode by correcting the stored teacher route in response to the designation of the parking mode. This allows the route correction to be based on a driving route that reflects the user's ideal, such as places the user does not want to pass through in the environment where the teacher driving was performed. The user will be able to park in the ideal parking mode in the next and subsequent playback driving without having to redo the teacher driving. Therefore, the parking assistance according to the embodiment can improve the usability of automatic parking for the end user.

[0126] The parking assistance device 3 according to the embodiment corrects the waypoints going back as far as necessary so that the final end point of the teacher driving is the designated ideal parking mode and satisfies the predefined constraint conditions. This makes it possible to suppress the teacher route from including abrupt attitude changes, and to facilitate the vehicle 1 following the route during automatic parking. Therefore, unnecessary deceleration is suppressed, and the user's discomfort during automatic parking can be reduced. In addition, according to the configuration for adjusting the position at which the correction is started depending on the correction amount of the parking mode, the corrected teacher route can be generated so as to follow the driving route that reflects the user's ideal as much as possible, such as places that the user does not want to pass in the environment where the teacher driving is performed.

[0127] In the above embodiment, the number of waypoints that require correction is calculated based on the correction amount, but the present invention is not limited to this. For example, a plurality of waypoints from the parking end position 515a to the first turning position 513a starting from the parking end position 515a may be set as fixed waypoints 701 to be corrected. Also, for example, a fixed number of waypoints may be used regardless of the correction amount. That is, a fixed number of waypoints from the waypoint Pa0 at the parking end position 515a to the waypoint Pa3 at the fixed number of waypoints may be set as the waypoints 701 to be corrected.

[0128] In addition, the output control unit 309 according to the embodiment described above may generate notification information for notifying the user when the waypoint that requires correction calculated based on the correction amount exceeds the turning position 513a, and output the notification information to the HMI 22. In other words, the parking assistance device 3 according to the embodiment described above may notify the user when the correction cannot be made within a range that satisfies the constraint condition using only the reverse path 523. This notification may be to notify an error, or may be to prompt the user to redo the teacher driving.

[0129] In the above embodiment, a part of the multiple waypoints that define the retreat portion 523a of the teacher route, that is, a waypoint from the parking position to the turning position, is set as the waypoint 701 to be corrected, but this is not limited thereto. That is, in the above embodiment, the waypoint Pa3 is a waypoint that defines the vehicle state at the first turning position 513a starting from the waypoint Pa0 among the multiple waypoints that define the teacher route, or a waypoint on the waypoint Pa0 side from the waypoint. On the other hand, the parking mode correction unit 308 according to the embodiment may set multiple waypoints including a waypoint that defines the forward portion 521a as the waypoint 701 to be corrected. That is, the waypoint 701 to be corrected may include a waypoint that is farther away from the first turning position starting from the parking position.

[0130] The parking mode correction unit 308 according to the embodiment described above may perform a route correction according to the vehicle mode of the turning position 513b, i.e., the turning mode, specified by the user. In this route correction according to the specification of the turning mode, the turning positions 513a and 513b are treated in the same manner as the parking end positions 515a and 515b according to the embodiment described above. For example, the reception unit 302 may receive a specification of the turning mode, i.e., an instruction to change the turning mode, based on the output of the HMI 22 according to the user's operation. For example, the output control unit 309 may output display information to the HMI 22 for displaying a display screen for specifying a user's desired turning mode, the display screen including a display showing the turning mode of the teacher route and a display simulating a desired turning mode as an operation target. For example, the parking mode correction unit 308 may correct the waypoint backward from the turning position 513b according to the correction amount to the turning mode specified by the user. The correction amount may be the difference between a waypoint (sixth waypoint) that defines a first vehicle behavior related to the turning position 513a and a waypoint (seventh waypoint) that defines a second vehicle behavior related to the turning position 513b designated by the user. In this case, the waypoints to be corrected may be multiple waypoints from the waypoint that defines the first vehicle behavior related to the turning position 513a to a waypoint (eighth waypoint) that is equal to or greater than the number of waypoints required for correction.

[0131] In the above embodiment, the case where the parking mode 601b is specified by the user is illustrated, but the present invention is not limited to this. The reception unit 302 may receive the specification of the parking mode 601b based on the output from an on-board sensor such as the sonar 211 or the omnidirectional camera 212. For example, the acquisition unit 301 may acquire information on objects around the vehicle 1, such as walls, obstacles, and white lines defining a parking space, for the vehicle 1 stopped in the parking mode 601a. ​​For example, the reception unit 302 may accept a parking mode along the parking space of the vehicle 1, which is defined by information on objects around the vehicle 1, as an instruction to change the parking mode 601b. As an example, the parking mode correction unit 308 determines whether the longitudinal direction of the vehicle 1 is parallel to the wall, obstacle, or white line on the side of the vehicle 1 stopped in the parking mode 601a, based on the information on the objects around the vehicle 1 acquired by the acquisition unit 301. If the parking mode correction unit 308 determines that the parking mode is not parallel, the reception unit may receive a parking mode 601b along a wall, an obstacle, or a white line as an instruction to change the parking mode. With this configuration, parking in an appropriate parking mode according to the surrounding environment of the parking space can be easily achieved. In addition, there may be a mode in which the parking mode along the parking space of the vehicle 1 is presented to the user on the display screen 401. For example, the parking mode along the parking space of the vehicle 1 may be used as the initial display of the display 409 on the display screen 401.

[0132] In the above-mentioned embodiment, "determining whether it is A" may mean "determining that it is A", or "determining that it is not A", or "determining whether it is A or not".

[0133] The program executed by the parking assistance device 3 of the above-described embodiment is provided by being recorded in a computer-readable recording medium such as a CD-ROM, FD, CD-R, or DVD in the form of an installable or executable file.

[0134] The program executed by the parking assistance device 3 of the above embodiment may be stored on a computer connected to a network such as the Internet and provided by being downloaded via the network. The program executed by the parking assistance device 3 may be provided or distributed via a network such as the Internet.

[0135] Furthermore, the programs executed by the parking assistance device 3 of the above-described embodiment may be provided by being pre-installed in a ROM or the like.

[0136] In addition, the program executed by the parking assistance device 3 of the above-mentioned embodiment has a modular configuration including each of the above-mentioned functional units (acquisition unit 301, reception unit 302, map information generation unit 303, driving route detection unit 304, memory unit 305, self-position estimation unit 306, route following calculation unit 307, parking mode correction unit 308, output control unit 309, and vehicle control unit 310), and in terms of actual hardware, the CPU 31 reads out and executes the program from the ROM 32 or HDD 34, whereby each of the above-mentioned functional units is loaded onto the RAM 33, and each of the above-mentioned functional units is generated on the RAM 33.

[0137] According to at least one of the embodiments described above, parking of the vehicle 1 in a desired parking manner can be appropriately assisted.

[0138] Although some embodiments of the present invention have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These embodiments can be implemented in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their modifications are included in the scope of the invention and its equivalents described in the claims, as well as in the scope and spirit of the invention.

[0139] (Additional Note) The above description of the embodiments discloses the following techniques. (1) storing information about a teacher route, which is a travel route in teacher travel in which the vehicle is moved and parked in the first parking mode; Receiving a parking mode change instruction specifying a second parking mode different from the first parking mode; and correcting the teacher path based on the second parking mode. Parking assistance methods. (2) Each of the first parking mode and the second parking mode is at least one of a parking position and a parking posture of the vehicle. The parking assistance method according to (1) above. (3) Further, the method includes outputting information indicating a corrected path corrected based on the second parking mode to a display device. 2. A parking assistance method according to claim 1, wherein the parking assistance method is a parking assistance method for parking a vehicle. (4) based on a parking mode correction amount, which is a difference between a first waypoint that defines the first parking mode and a second waypoint that defines the second parking mode, correcting waypoints to be corrected from the first waypoint to a third waypoint among the plurality of waypoints that define the teacher route, and performing a route correction that defines the teacher route using a plurality of waypoints including the corrected waypoints from the second waypoint to the third waypoint. A parking assistance method according to any one of claims (1) to (3). (5) storing a constraint condition regarding a change amount of a vehicle behavior of the vehicle between predetermined adjacent waypoints; calculating the number of waypoints required to correct the correction amount within a range that satisfies the constraint condition, and specifying, as the third waypoint, a waypoint that is equal to or greater than the first waypoint as a starting point among a plurality of waypoints that define the teacher route; The parking assistance method according to (4) above. (6) The third waypoint is a waypoint that is a predetermined number of waypoints from the first waypoint as a starting point among a plurality of waypoints that define the teacher route. The parking assistance method according to (4) above. (7) The third waypoint is a waypoint that defines a vehicle state of the vehicle at a first turning point starting from the first waypoint among a plurality of waypoints that define the teacher route, or a waypoint that is closer to the first waypoint than the first waypoint. A parking assistance method according to (5) or (6) above. (8) outputting notification information for notifying a user when a number of waypoints starting from the first waypoint among the plurality of waypoints defining the teacher route exceed a waypoint defining the vehicle state at a first turning point starting from the first waypoint among the plurality of waypoints defining the teacher route, A parking assistance method according to any one of (5) to (7) above. (9) The instruction to change the parking mode is based on a user operation to specify the second parking mode, A parking assistance method according to any one of claims (1) to (8). (10) Further comprising: outputting, to a display device, display information for displaying a display screen including a display showing the first parking mode and a display simulating the second parking mode; The parking assistance method according to (9) above. (11) The instruction to change the parking mode is based on information about objects around the vehicle parked in the first parking mode from an on-board sensor mounted on the vehicle. A parking assistance method according to any one of claims (1) to (10). (12) Accepting a parking mode change instruction specifying a third parking mode different from the second parking mode; and correcting waypoints to be corrected from the second waypoint to a fifth waypoint among the plurality of waypoints defining the teacher route based on a correction amount of at least one of the position and attitude of the vehicle, which is a difference between the second waypoint and a fourth waypoint defining the third parking mode, and performing a route correction to define the teacher route using a plurality of waypoints including the corrected waypoints from the fourth waypoint to the fifth waypoint. A parking assistance method according to any one of claims citing (4) above directly or indirectly among (4) to (11) above. (13) receiving a change instruction for a turning pattern that specifies a vehicle pattern of the vehicle at a first turning motion starting from the first waypoint among a plurality of waypoints that define the teaching route; and further comprising: modifying correction target waypoints from the sixth waypoint to the eighth waypoint among the plurality of waypoints defining the teacher route based on a correction amount of the vehicle behavior at the first turning, which is a difference between a sixth waypoint defining a first vehicle behavior at the first turning on the teacher route and a seventh waypoint defining a specified second vehicle behavior, and defining the teacher route by a plurality of waypoints including the corrected waypoints from the seventh waypoint to the eighth waypoint. A parking assistance method according to any one of claims citing (4) above directly or indirectly among (4) to (12) above. (14) and controlling a regenerative travel to move the vehicle along the teacher route and stop the vehicle in the second parking mode based on information about the teacher route. A parking assistance method according to any one of claims (1) to (13). Parking assistance methods. (15) A storage unit that stores information about a teacher route that is a travel route in teacher travel in which the vehicle is moved and parked in a first parking mode; A reception unit that receives a parking mode change instruction that specifies a second parking mode different from the first parking mode; A correction unit that corrects the teacher path based on the second parking mode. Parking assistance device. (16) Each of the first parking mode and the second parking mode is at least one of a parking position and a parking posture of the vehicle. The parking assistance device according to (15) above. (17) Further, an output control unit outputs information indicating a corrected path corrected based on the second parking mode to a display device. A parking assistance device according to (15) or (16) above. (18) the correction unit corrects waypoints to be corrected from the first waypoint to a third waypoint among the plurality of waypoints defining the teacher route based on a correction amount of the parking mode, the waypoint being a difference between a first waypoint defining the first parking mode and a second waypoint defining the second parking mode, and performs a route correction to define the teacher route using a plurality of waypoints including the corrected waypoint from the second waypoint to the third waypoint; A parking assistance device according to any one of claims (15) to (17). (19) The storage unit stores a constraint condition regarding an amount of change in a vehicle behavior of the vehicle between predetermined adjacent waypoints; the correction unit calculates the number of waypoints required to correct the correction amount within a range that satisfies the constraint condition, and specifies, as the third waypoint, a waypoint that is equal to or greater than the first waypoint, among a plurality of waypoints that define the teacher route, starting from the first waypoint. The parking assistance device according to (18) above. (20) The third waypoint is a waypoint that is a predetermined number of waypoints from the first waypoint as a starting point among a plurality of waypoints that define the teacher route. The parking assistance device according to (18) above. (twenty one) The third waypoint is a waypoint that defines a vehicle state of the vehicle at a first turning point starting from the first waypoint among a plurality of waypoints that define the teacher route, or a waypoint that is closer to the first waypoint than the first waypoint. A parking assistance device according to (19) or (20) above. (twenty two) a display control unit that outputs notification information for notifying a user when a number of waypoints, starting from the first waypoint among the plurality of waypoints defining the teacher route, exceed a waypoint that defines the vehicle state at a first turning point starting from the first waypoint among the plurality of waypoints defining the teacher route, A parking assistance device according to any one of claims (19) to (21). (twenty three) The instruction to change the parking mode is based on a user operation to specify the second parking mode, A parking assistance device according to any one of claims (15) to (22). (twenty four) Further comprising a display control unit that outputs display information for displaying a display screen including a display showing the first parking mode and a display simulating the second parking mode on a display device; The parking assistance device according to (23) above. (twenty five) The instruction to change the parking mode is based on information about objects around the vehicle parked in the first parking mode from an on-board sensor mounted on the vehicle. A parking assistance device according to any one of claims (15) to (24). (26) The reception unit receives a parking mode change instruction that specifies a third parking mode different from the second parking mode, the correction unit modifies waypoints to be corrected from the second waypoint to a fifth waypoint among the plurality of waypoints defining the teacher route based on a correction amount of at least one of the position and attitude of the vehicle, which is a difference between the second waypoint and a fourth waypoint defining the third parking mode, and performs a route correction to define the teacher route using a plurality of waypoints including the corrected waypoint from the fourth waypoint to the fifth waypoint. A parking assistance device according to any one of claims citing (18) directly or indirectly among (18) to (25). (27) the reception unit receives a change instruction for a turning behavior that specifies a vehicle behavior of the vehicle at a first turning behavior starting from the first waypoint among a plurality of waypoints that define the teacher route; the correction unit corrects waypoints to be corrected from the sixth waypoint to the eighth waypoint among the plurality of waypoints defining the teacher route based on a correction amount of the vehicle behavior at the first turning, which is a difference between a sixth waypoint defining a first vehicle behavior at the first turning on the teacher route and a seventh waypoint defining a specified second vehicle behavior, and defines the teacher route by a plurality of waypoints including the corrected waypoints from the seventh waypoint to the eighth waypoint. A parking assistance device according to any one of claims citing (18) directly or indirectly among (18) to (26). (28) a vehicle control unit that controls regenerative driving to move the vehicle along the teacher route and stop the vehicle in the second parking mode based on information about the teacher route, A parking assistance device according to any one of claims (15) to (27). (29) A program for causing a computer to execute the parking assistance method according to any one of (1) to (14) above. (30) A recording medium (Computer Program Product) having recorded thereon the program described in (29) above, which is executed by a computer. (31) A parking assistance device according to any one of (15) to (28) above; An operation unit that accepts an operation related to a user's designation of a parking mode and / or a vehicle mode during a turn. vehicle. (32) A parking assistance device according to any one of (15) to (28) above; A display device that displays a display screen for accepting user operations. vehicle. (33) The parking assistance device according to (28) above; and a control device that controls at least one of a steering angle, a drive, and a brake based on a control signal from the vehicle control unit. vehicle. (34) A parking assistance device according to any one of (15) to (28) above; At least one of a sonar, a 360-degree camera, a steering angle sensor, a wheel speed sensor, a direction sensor, and a GNSS sensor is provided. vehicle. [Explanation of symbols]

[0140] 1 vehicle 12 Body 13 wheels 14 Bumper 22 HMI (display section, operation section) 23 Steering control device 24 Drive and brake control device 3. Parking Assistance Device 31 CPU 32 ROM 33 RAM 34 HDD 35 Interfaces 39 Bus 130 seats 140 Steering wheel (operating part) 180 Windshield 190 Dashboard 211 Sonar 212 All-around camera 213 Orientation Sensor 214 GNSS Sensor 221 Display device (display section, operation section) 222a, 222b Operation button (operation part) 301 Acquisition Department 302 Reception 303 Map information generation unit 304 Travel route detection unit 305 Storage section 306 Self-position estimation unit (vehicle control unit) 307 Path following calculation unit (vehicle control unit) 308 Parking mode correction unit (correction unit) 309 Output control section (display control section) 310 Vehicle control unit

Claims

1. A parking assistance method for automatically driving a vehicle based on a teacher driving in which the vehicle is moved and stopped in a first parking mode, comprising: generating an overhead image by combining a front image of the vehicle, a rear image of the vehicle, and a side image of the vehicle; storing first feature point information during the teacher driving in which the vehicle is moved and parked in the first parking mode; comparing the stored first feature point information with second feature point information extracted from an image of the surroundings of the vehicle; displaying the overhead image, information indicating the first parking mode, and information simulating a second parking mode different from the first parking mode; the second parking mode includes a parking position and a parking posture of the vehicle, The information simulating the second parking mode is displayed after the instructor driving. receiving a parking mode change instruction specifying the second parking mode different from the first parking mode; The instruction to change the parking mode is based on a user operation to specify the second parking mode, the user operation is performed by moving and rotating the information simulating the second parking mode, automatically driving the vehicle based on the second parking mode; The parking position and the parking posture of the vehicle in the first parking mode are different from the parking position and the parking posture of the vehicle in the second parking mode, The travel route of the teacher travel is different from the travel route of the automatic travel, Parking assistance methods.

2. The first parking mode and the second parking mode include a parking position and a parking posture of the vehicle. The parking assistance method according to claim 1 .

3. The method further includes outputting information indicating the corrected path corrected based on the second parking mode to a display device. The parking assistance method according to claim 1 .

4. based on a parking mode correction amount which is a difference between a first waypoint that defines the first parking mode and a second waypoint that defines the second parking mode, correcting waypoints to be corrected from the first waypoint to a third waypoint among the plurality of waypoints that define a teacher route in the teacher traveling, and performing a route correction that defines the teacher route using a plurality of waypoints including the corrected waypoints from the second waypoint to the third waypoint. The parking assistance method according to claim 1 .

5. storing constraints regarding the amount of change in vehicle behavior of the vehicle between predetermined adjacent waypoints; calculating the number of waypoints required to correct the correction amount within a range that satisfies the constraint condition, and specifying, as the third waypoint, a waypoint that is equal to or greater than the first waypoint as a starting point from among the plurality of waypoints that define the teacher route; The parking assistance method according to claim 4.

6. the third waypoint is a waypoint that is a predetermined number of waypoints from the first waypoint as a starting point among a plurality of waypoints that define the teacher route; The parking assistance method according to claim 4.

7. the third waypoint is a waypoint that defines the vehicle behavior of the vehicle at the first turning point starting from the first waypoint among the plurality of waypoints that define the teacher route, or a waypoint that is closer to the first waypoint than the first waypoint; 7. The parking assistance method according to claim 5 or 6.

8. outputting notification information for notifying a user when a second or more waypoints, starting from the first waypoint among the plurality of waypoints defining the teacher route, exceed a waypoint that defines the vehicle state at a first turn starting from the first waypoint among the plurality of waypoints defining the teacher route, The parking assistance method according to claim 5.

9. outputting, to a display device, display information for displaying a display screen including a display showing the first parking mode and a display simulating the second parking mode; The parking assistance method according to claim 1 .

10. The instruction to change the parking mode is based on information regarding objects around the vehicle parked in the first parking mode from an on-board sensor mounted on the vehicle. The parking assistance method according to claim 1 .

11. Accepting a parking mode change instruction specifying a third parking mode different from the second parking mode; modifying waypoints to be corrected from the second waypoint to a fifth waypoint among the plurality of waypoints defining the teacher route based on a correction amount of at least one of the position and attitude of the vehicle, which is a difference between the second waypoint and a fourth waypoint defining the third parking mode, and performing a route correction to define the teacher route using a plurality of waypoints including the corrected waypoints from the fourth waypoint to the fifth waypoint. The parking assistance method according to claim 4.

12. receiving a change instruction for a turning pattern that specifies the vehicle pattern of the vehicle at a first turning starting from the first waypoint among the plurality of waypoints that define the teacher route; and further comprising: correcting waypoints to be corrected from the sixth waypoint to an eighth waypoint among the plurality of waypoints defining the teacher route based on a correction amount for the vehicle behavior at the first turning, which is a difference between a sixth waypoint that defines a first vehicle behavior at the first turning on the teacher route and a seventh waypoint that defines a specified second vehicle behavior; and defining the teacher route using a plurality of waypoints including the corrected waypoints from the seventh waypoint to the eighth waypoint. The parking assistance method according to claim 4.

13. A parking assistance device that automatically drives a vehicle based on a teacher driving that moves the vehicle and stops the vehicle in a first parking mode, a storage unit that stores first feature point information during the teacher driving in which the vehicle is moved and parked in the first parking mode; a display control unit that generates an overhead image by combining a front image of the vehicle, a rear image of the vehicle, and a side image of the vehicle; a display unit that displays the overhead image, information indicating the first parking mode, and information simulating a second parking mode different from the first parking mode; a reception unit that receives a parking mode change instruction that specifies the second parking mode different from the first parking mode; a vehicle control unit that compares the stored first feature point information with second feature point information extracted from an image of the surroundings of the vehicle, and causes the vehicle to automatically drive based on the second parking mode, the second parking mode includes a parking position and a parking posture of the vehicle, The information simulating the second parking mode is displayed after the instructor driving. The instruction to change the parking mode is based on a user operation to specify the second parking mode, the user operation is performed by moving and rotating the information simulating the second parking mode, The parking position and the parking posture of the vehicle in the first parking mode are different from the parking position and the parking posture of the vehicle in the second parking mode, The travel route of the teacher travel is different from the travel route of the automatic travel, Parking assistance device.

Images