Parking assistance procedure and parking assistance device

Abstract

A parking assistance method according to the present disclosure performs an automated drive of a vehicle (1) based on a previously performed parking maneuver. The parking assistance method comprises capturing a learning path based on a driving path in the previously performed parking maneuver, capturing vehicle operation information relating to the operation of at least one vehicle accessory, such as a turn signal and a side mirror, in the vehicle (1) during the previously performed parking maneuver, storing learning data containing the learning path and the vehicle operation information, and reproducing an operation of the vehicle accessory caused by the operation captured during the previously performed parking maneuver, based on the learning data, during automated parking, in which the vehicle (1) is moved along the learning path.

Description

AREA

[0001] The embodiments described here generally relate to a parking assistance method and a parking assistance device. TECHNICAL BACKGROUND

[0002] Existing technology includes parking assistance techniques that move a vehicle through automated driving when the vehicle's parking position is known. One of these parking assistance techniques is a regenerative driving technique that learns a driving path based on a driver's learning drive and moves a vehicle along the driving path learned during the learning drive. Such a technique, related to path memory automatic parking, is used, for example, in a case where parking is repeatedly performed at a specific parking position, such as a parking lot at a user's home or workplace. Conventional technologies are described, for example, in Japanese Patent No. 6022447.

[0003] However, the parking assistance technology for automatic parking with path storage still needs improvement in terms of assistance content. For example, the parking assistance technology needs improvement from a safety perspective, as it is difficult for people and vehicles in the surrounding area to detect the vehicle's behavior during regeneration driving. Furthermore, the parking assistance technology needs improvement from a convenience perspective, for example, as a driver must operate the vehicle in a situation where the vehicle's surroundings during regeneration driving are different from those during the learning drive.

[0004] One of the objectives to be achieved by the present disclosure is to adequately support automated parking of the path storage type. SUMMARY

[0005] A parking assistance method according to an embodiment of the present disclosure performs automatic travel of a vehicle based on a past parking operation. The method includes: acquiring a teaching path based on a travel path in the past parking operation; acquiring vehicle operation information related to an operation of at least one vehicle accessory of a turn signal and a side mirror in the vehicle during the past parking operation; storing teaching data including the teaching path and the vehicle operation information; and reproducing an operation of the vehicle accessory caused by the operation acquired during the past parking operation based on the teaching data in automatic parking in which the vehicle is moved along the teaching path. BRIEF DESCRIPTION OF THE DRAWINGS Fig. 1 is a diagram schematically showing an example of a vehicle equipped with a parking assist device according to an embodiment; Fig. 2 is a diagram showing an example of a configuration near a driver's seat of the vehicle of Fig. 1 shows; Fig. 3 is a diagram showing an example of a hardware configuration of the parking assist device of Fig. 1 shows; Fig. 4 is a diagram showing an example of a functional configuration of the parking assist device of Fig. 1 shows; Fig. 5 is a flowchart showing an example of the flow of a teaching driving procedure performed by the parking assist device of Fig. 1 is executed; and Fig. 6 is a flowchart showing an example of the flow of a regeneration process performed by the parking assist device of Fig. 1 is executed. DETAILED DESCRIPTION

[0006] Hereinafter, embodiments of a parking assistance device, a vehicle, a parking assistance method, a computer program product, and a recording medium according to the present disclosure will be described with reference to the accompanying drawings.

[0007] It should be noted that in the description of the present disclosure, the components that have the same or substantially the same functions as those described above with reference to the previous drawings are designated by the same reference numerals, and the description of these elements may be omitted as appropriate. Furthermore, even when illustrating the same or substantially the same part, the dimensions and proportions may be represented differently depending on the drawings. Furthermore, in the description of each drawing, only the main components are provided with reference numerals to ensure the visibility of the drawings, and components that have the same or substantially the same functions as those described in the previous drawings may not be provided with reference numerals.

[0008] It should be noted that in the description of the present disclosure, components having the same or substantially the same function may be distinguished and designated by adding alphanumeric characters to the end of the reference numerals. Alternatively, in a case where multiple components having the same or substantially the same function are not distinguished, the components may be integrated and designated by omitting the alphanumeric characters at the end of the reference numerals.

[0009] Fig. 1 is a diagram schematically showing an example of a vehicle 1 equipped with a parking assist device 3 according to an embodiment. As shown in Fig. As shown in Figure 1, the vehicle 1 includes a vehicle body 12 and two pairs of wheels 13 arranged along a predetermined direction on the vehicle body 12. The two pairs of wheels 13 include a pair of front tires 13f and a pair of rear tires 13r.

[0010] Here, the front tire 13f according to the embodiment is an example of a first wheel. Moreover, the rear tire 13r according to the embodiment is an example of a second wheel. It should be noted that although Fig. While Figure 1 shows the vehicle 1 with four wheels 13, the present invention is not limited thereto. The vehicle 1 only needs to have at least one front tire 13f and at least one rear tire 13r. The number of wheels 13 of the vehicle 1 can be two, three, five, or more.

[0011] The direction of at least one wheel (steered wheel) of the wheels 13 of the vehicle 1 is electrically or mechanically coupled, for example, to a rotation angle of the steering wheel arranged in front of a driver's seat 130a, i.e., a steering angle. That is, the vehicle 1 can be turned right or left by steering. Note that the steered wheel may be the rear tire 13r or both the front tire 13f and the rear tire 13r.

[0012] The vehicle body 12 is supported by the wheels 13. The vehicle 1 includes a prime mover (not shown) and is movable by driving at least one of the wheels 13 of the vehicle 1 by the power of the prime mover. Note that any prime mover, such as an engine that uses gasoline, hydrogen, or the like as fuel, a motor that uses electric power from a battery, or a combination of a motor and a motor, can be used as the prime mover. In this case, a predetermined direction in which the two pairs of wheels 13 are arranged is a traveling direction of the vehicle 1. The vehicle 1 can move forward or backward by shifting gears (not shown) or the like.

[0013] Furthermore, the vehicle body 12 has a front end portion F, which is an end portion on the side of the front tire 13f, and a rear end portion R, which is an end portion on the side of the rear tire 13r. The vehicle body 12 has a substantially rectangular shape in plan view, and the four corners of the substantially rectangular shape can be referred to as end portions.

[0014] A pair of bumpers 14 are provided near the lower end of the vehicle body 12 at the front and rear end portions F and R of the vehicle body 12. The front bumper 14f of the pair of bumpers 14 covers the entire front surface and part of the side surface near the lower end portion of the vehicle body 12. The rear bumper 14r of the pair of bumpers 14 covers the entire rear surface and part of the side surface near the lower end portion of the vehicle body 12.

[0015] Here, a configuration near the driver's seat of the vehicle 1 in the present embodiment will be described. Fig. 2 is a diagram showing an example of a configuration near the driver's seat 130a of the vehicle 1 according to the embodiment.

[0016] As in Fig. 2, the vehicle 1 comprises at least one seat 130. In Fig. 2, the driver's seat 130a and a passenger seat 130b are shown as seat 130. Furthermore, a steering wheel 140, a windshield 180, an instrument panel 190, a display device 221, and control buttons 222a and 222b are provided in front of the driver's seat 130a.

[0017] The steering wheel 140 is arranged in front of the driver's seat 130a and can be operated by the driver. The angle of rotation of the steering wheel 140, i.e., the steering angle, is electrically or mechanically linked to a change in the direction of the front wheel 13f, which is a steered wheel. Note that the steered wheel can be the rear tire 13r or both the front tire 13f and the rear tire 13r.

[0018] As in Fig. 1, the vehicle 1 is equipped with the parking assistance device 3. The parking assistance device 3 is an information processing device that can be mounted on the vehicle 1 and is implemented, for example, by an electronic control unit (ECU) or an on-board unit (OBU) provided inside the vehicle 1. Note that the parking assistance device 3 can be implemented by a domain control unit (DCU) such as a cockpit domain controller (CDC) in which multiple ECUs are integrated. The CDC is configured to be capable of executing processes such as in-vehicle infotainment (IVI), meter control, display device control of a head-up display (HUD), an electronic mirror, or the like, and advanced driver assistance systems (ADAS).Alternatively, the parking assistance device 3 may be an external computer installed near the dashboard 190 of the vehicle 1, or it may also serve as another device, such as a vehicle navigation device. Note that the parking assistance device 3 is integrally integrated with an HMI 22 (see FIG. Fig. 3) may be configured or configured to be capable of cooperating with the HMI 22 through communication, and that it may be realized as an information display device that may be mounted on the vehicle 1.

[0019] Fig. 3 is a diagram showing an example of a hardware configuration of the parking assist device 3 according to the embodiment. As shown in Fig. 3, the parking assist device 3 includes a central processing unit (CPU) 31, a read-only memory (ROM) 32, a random access memory (RAM) 33, a hard disk drive (HDD) 34, and an interface (I / F) 35. The CPU 31, the ROM 32, the RAM 33, the HDD 34, and the I / F 35 are connected to each other by a bus 39 or the like, and have a hardware configuration using a normal computer.

[0020] The CPU 31 is a computing unit that controls the entire parking assist device 3. The CPU 31 loads a program stored in the ROM 32 or the HDD 34 into the RAM 33 and executes the program, thereby implementing each process described later.

[0021] Note that the CPU 31 according to the embodiment is an example of at least one processor in the parking assist device 3. As the at least one processor, another processor may be provided instead of the CPU 31 or in addition to the CPU 31. As other processors, various processors such as a CPU, a graphics processing unit (GPU), a digital signal processor (DSP), a dedicated arithmetic circuit implemented by an application-specific integrated circuit (ASIC), and a field-programmable gate array (FPGA), and the like can be used.

[0022] The ROM 32 stores programs, parameters, and the like that execute various processes of the CPU 31. The ROM 32 according to the embodiment is an example of at least one memory in the parking assistance device 3.

[0023] The RAM 33, for example, is a main memory of the parking assistance device 3 and temporarily stores data necessary for various processes of the CPU 31. The RAM 33 according to the embodiment is an example of at least one memory in the parking assistance device 3.

[0024] The hard disk 34 stores various types of data, programs, and the like used by the parking assistance device 3. For example, the HDD 34 stores information obtained from various in-vehicle sensors mounted on the vehicle 1, such as a sonar 211, a surround-view camera 212, an azimuth sensor 213, and a GNSS sensor 214. Note that, instead of or in addition to the HDD 34, various storage media and storage devices such as a solid-state drive (SSD) and a flash memory may be used. Various storage media and storage devices such as the HDD 34 according to the embodiment are examples of at least one memory in the parking assistance device 3.

[0025] The I / F 35 is an interface for sending and receiving data. The I / F 35 receives data from another device in vehicle 1, e.g., from an on-board sensor of vehicle 1 or the HMI 22. Furthermore, the I / F 35 transmits data to other devices in vehicle 1, e.g., to the HMI 22, a steering control unit 23, and a drive / brake control unit 24.

[0026] Note that the I / F 35 can send and receive information to and from another control device mounted in the vehicle 1 via an in-vehicle network including a Controller Area Network (CAN), Ethernet (registered trademark), Universal Serial Bus (USB, registered trademark), or the like in the vehicle 1, or communicate with an information processing device external to the vehicle 1 via a network such as the Internet. For example, the parking assist device 3 is connected to the sonar 211, the surround camera 212, the azimuth sensor 213, the GNSS sensor 214, the HMI 22, the steering control device 23, and the drive / brake control device 24 of the vehicle 1 via the in-vehicle network.

[0027] For example, the I / F 35 may receive information about the state of the vehicle 1, such as a vehicle speed pulse, each speed including a yaw rate, an acceleration, position information, and shift information, from another control unit or an on-board sensor via the on-board network.

[0028] For example, the I / F 35 can receive information about the surrounding situation of the vehicle 1, such as a detection result of a surrounding obstacle and a surrounding image, from another control unit or an on-board sensor via the on-board network. The obstacle is, for example, an object in the surroundings of the vehicle 1, e.g., a moving body such as a person or a vehicle, or a structure.

[0029] For example, the I / F 35 can receive information (vehicle operation information) about the vehicle operation of the vehicle 1 from another control unit, the HMI 22, or an on-board sensor via the on-board network. The vehicle operation information will be described later.

[0030] Vehicle 1 is equipped with a variety of on-board sensors. Fig. 1 and Fig. 3 shows the sonar 211 and the all-round camera 212. Fig. 3 also shows the azimuth sensor 213 and the GNSS sensor 214.

[0031] The sonar 211 is provided, for example, at a predetermined end portion of the vehicle body 12 and transmits and receives a sound wave, such as an ultrasonic wave. The sonar 211 includes wave transmitting / receiving modules 211f and 211r. For example, at least one wave transmitting / receiving module 211f is arranged on the front bumper 14f, and at least one wave transmitting / receiving module 211r is arranged on the rear bumper 14r. Furthermore, the number and / or positions of the wave transmitting / receiving modules 211f and 211r are not limited to the Fig. 1 and can be modified as needed. For example, vehicle 1 may have wave transmitting / receiving modules 211f and 211r on the left and right sides.

[0032] The sonar 211 detects an obstacle in the vicinity of the vehicle 1 based on the transmission / reception result of the sound wave. Furthermore, the sonar 211 measures a distance between an obstacle around the vehicle 1 and the vehicle 1 based on the transmission / reception result of the sound wave.

[0033] Note that in the present embodiment, the sonar 211 using a sound wave such as an ultrasonic wave is exemplified, but the present invention is not limited thereto. For example, the vehicle 1 may include a radar that transmits and receives electromagnetic waves instead of the sonar 211 or in addition to the sonar 211.

[0034] The surround-view camera 212 is provided in the vehicle 1 to image the surroundings of the vehicle 1. For example, the vehicle 1 includes, as the surround-view camera 212, a front camera 212a that images the front, a rear camera 212b that images the rear, a left side camera 212c that images the left side, and a right side camera (not shown) that images the right side.

[0035] The surround-view camera 212 records the surroundings of the vehicle 1. The surround-view camera 212 is, for example, a camera that captures an image based on visible light and / or infrared light. It should be noted that the image captured by the surround-view camera 212 can be a moving image or a still image.

[0036] It should be noted that the position and / or number of the surround cameras 212 are not limited to the Fig. 1 is limited and can be modified as needed. For example, the vehicle 1 may have only two cameras, namely the front camera 212a and the rear camera 212b. Alternatively, the vehicle 1 may include another camera in addition to the example described above. For example, some or all of the surround-view cameras 212 may be arranged in a vehicle interior (inside the vehicle) of the vehicle 1. Furthermore, the surround-view camera 212 may be a camera built into the vehicle 1, a camera of a drive recorder retrofitted to the vehicle 1, or the like.

[0037] The azimuth sensor 213 detects the orientation of the traveling direction of the vehicle 1. The azimuth sensor 213 is, for example, a sensor that measures the orientation of the traveling direction of the vehicle 1 from a rotation difference between the left and right wheels 13 of the vehicle 1, geomagnetism, a gas rate gyroscope, an optical fiber gyroscope, or the like. Note that a known method can be appropriately adopted as a method for obtaining the azimuth of the traveling direction of the vehicle 1, and the present invention is not particularly limited.

[0038] The GNSS sensor 214 acquires position information indicating the current position of the vehicle 1. The GNSS sensor 214 is, for example, a GNSS (Global Navigation Satellite System) sensor that outputs position information of the vehicle 1, such as a GPS (Global Positioning System) sensor. Note that the GNSS sensor 214 includes a GNSS antenna that receives a radio wave (GNSS signal) from a satellite, and a GNSS circuit that obtains position information (GNSS coordinates) based on radio waves received from at least two satellites by the GNSS antenna.

[0039] It should be noted that the vehicle 1 may include other in-vehicle sensors (not shown). For example, the vehicle 1 may include a steering angle sensor that outputs a signal corresponding to an operation amount of the steering wheel 140 by the driver, i.e., a steering angle. The vehicle 1 may, for example, include a wheel speed sensor that outputs a signal corresponding to the rotational speed and direction of the wheels 13. The vehicle 1 may, for example, include a brake sensor that detects an operation of a brake pedal by the driver. The vehicle 1 may, for example, include an accelerator pedal sensor that detects the operation of an accelerator pedal by the driver. The vehicle 1 may, for example, include an acceleration sensor that outputs a signal corresponding to an acceleration exerted on the vehicle 1.The vehicle 1 may, for example, contain a gyro sensor that outputs a signal corresponding to an angular velocity acting on the vehicle 1.

[0040] The HMI 22 is an interface for outputting various types of information, such as notifications and warnings, to the driver of vehicle 1. The HMI 22 is an interface for receiving vehicle operations or inputting various types of information by the driver of vehicle 1. The HMI 22 only needs to be able to output a notification or warning in order to be recognizable to the driver of vehicle 1 and to receive various operations from the driver of vehicle 1. The HMI 22 is arranged, for example, around the driver's seat of vehicle 1, but can also be arranged in another area around the driver's seat, for example, on a rear seat.

[0041] The HMI 22 includes a display device that displays various types of information, such as notifications and warnings, to the driver of the vehicle 1. For example, the HMI 22 includes a display device 221 provided on the dashboard 190 or a console of the vehicle 1 and configured to output a video. Fig. 2 shows the display device 221 arranged in the center of the instrument panel 190. The display device 221 is, for example, a liquid crystal display (LCD) or an organic electroluminescence (EL) display. Here, the HMI 22 as a display device according to the embodiment is an example of a display unit. The HMI 22 as a display unit displays various screens based on various types of display information from an output control module 310, which will be described later.

[0042] It should be noted that the display device 221 may be configured as a touch panel display. Furthermore, the display device 221 may be part of a navigation device mounted on the vehicle 1. Furthermore, the display device may be a projection device such as a head-up display (HUD) that projects a video (virtual image) in front of the driver, e.g., in a display area on the windshield or dashboard (console).

[0043] It should be noted that the HMI 22 is not limited to the display device, but may also include another output device, such as a speaker configured to emit a notification sound, a warning sound, or a voice.

[0044] Furthermore, the HMI 22 includes an input device that receives vehicle operation and input of various types of information from the driver of the vehicle 1. For example, the HMI 22 includes a touch panel of the display device 221, configured as a touch panel display, as an input device. For example, the HMI 22 includes the operation buttons 222a and 222b as an input device. Fig. 2 shows the operation button 222a arranged on an instrument panel and the operation button 222b provided on the console. Note that the operation buttons 222a and 222b may also be provided in other locations, such as the steering wheel 140 and the instrument panel 190. Furthermore, the operation buttons 222a and 222b need not be provided if another input device is provided, such as the touch panel of the display device 221 being used as the input device of the HMI 22. Here, the HMI 22 as the input device according to the embodiment is an example of an operation unit. The HMI 22 as an operation unit outputs a signal corresponding to various user operations on the touch panel of the display device 221 and the operation buttons 222a and 222b, as well as various user operations on a detection module 301 or a reception module 302, which will be described later.

[0045] It should be noted that the HMI 22 may also contain other input devices such as other buttons, knobs, switches, and microphones. These input devices are located, for example, on the dashboard 190, the instrument panel, the steering wheel 140, the console, or the like of the vehicle 1.

[0046] It should be noted that an operator terminal capable of receiving, transmitting, or sending / receiving a signal from the outside of the vehicle 1, such as a tablet terminal, a smartphone, a remote control, or an electronic key, can be used as the output device, input device, or input / output device of the HMI 22.

[0047] The steering control unit 23 controls the steering of the vehicle 1. The steering control unit 23 steers the wheels 13 in a direction corresponding, for example, to a control signal representing an amount of operation of the steering wheel 140 by the driver or a control signal from the parking assist device 3. Note that the steering control unit 23 may include a steering actuator (not shown) that changes the rotation angle of the steering wheel 140 in accordance with a control signal from the parking assist device 3.

[0048] The drive / brake control unit 24 controls the acceleration / deceleration of the vehicle 1. The drive / brake control unit 24 includes, for example, a brake actuator (not shown) and an accelerator pedal controller (not shown). The brake actuator brakes the vehicle 1 or decelerates the vehicle 1 by operating a brake, changing a gear shift (gear ratio), or controlling an output of the drive machine, such as the engine or the internal combustion engine, based on a detection result of the brake sensor (not shown) that detects the amount of operation of the brake pedal by the driver, or a control signal from the parking assist device 3. The accelerator pedal controller accelerates the vehicle 1 by controlling the output of the drive machine, such asof the engine, based on the detection result of the accelerator pedal sensor (not shown) which detects the driver's operation of the accelerator pedal or a control signal from the parking assist device 3.

[0049] It should be noted that Fig. 3 shows a case where the sonar 211, the surround camera 212, the azimuth sensor 213, the GNSS sensor 214, the HMI 22, the steering control unit 23, and the drive / brake control unit 24 are not provided in the parking assist device 3, but the present invention is not limited thereto. Some or all of the above components may be provided in the parking assist device 3.

[0050] Fig. 4 is a diagram showing an example of a functional configuration of the parking assist device 3 according to the embodiment. The CPU 31 executes the program loaded into the RAM 33, whereby the parking assist device 3 implements functions such as the detection module 301, the reception module 302, a map information generation module 303, a travel path detection module 304, a vehicle operation detection module 305, a storage unit 306, a self-position estimation module 307, a path following calculation module 308, a vehicle operation playback module 309, an output control module 310, and a vehicle control module 311.

[0051] Here, each of the detection module 301 and the reception module 302 according to the embodiment is an example of a reception module. Furthermore, each of the detection module 301, the map information generation module 303, the travel path recognition module 304, and the vehicle operation detection module 305 according to the embodiment is an example of a learning module. Furthermore, each of the self-position estimation module 307, the path following calculation module 308, the vehicle operation playback module 309, and the vehicle control module 311 is an example of a regeneration module. Furthermore, the output control module 310 according to the embodiment is an example of a display control unit.

[0052] The acquisition module 301 acquires data from at least one in-vehicle sensor, including other devices present in the vehicle 1, e.g., the sonar 211, the surround camera 212, the azimuth sensor 213, and the GNSS sensor 214, e.g., via the I / F 35.

[0053] As an example, the detection module 301 detects an output from an external sensor (vehicle sensor) that detects spatial information around the vehicle 1, such as the sonar 211 and the surround camera 212.

[0054] As an example, the acquisition module 301 acquires an output from the vehicle sensor (vehicle sensor) that acquires a distance, a speed, and an azimuth in which the vehicle 1 is moving, such as the azimuth sensor 213 and the GNSS sensor 214.

[0055] As an example, the sensing module 301 captures an output from a turn signal lever or a switch (HMI 22) on the steering wheel that receives a vehicle command instructing the on / off of a turn signal (turn signals and vehicle accessories). Note that the sensing module 301 may capture information indicating an operation state of the turn signal.

[0056] The receiving module 302 receives various operations of a user, such as the driver, based on an output of the HMI 22 corresponding to various operations of the user.

[0057] For example, in a case where the operation button 222a is pressed in a state where the vehicle 1 is stopped, the reception module 302 receives this operation as a parking assist function call command to request a parking assist function call, which is a menu display on the display such as the display device 221. Note that, instead of pressing the operation button 222a, in a case where the operation button 222b is pressed or a touch operation is performed on an icon on the touch panel of the display device 221 configured as the touch panel display, this operation may be received as a parking assist function call instruction.

[0058] For example, in a case where the parking assist function call command is received and then a touch of an operation icon on the assistance information display screen or a press of the operation buttons 222a and 222b is performed, the receiving module 302 receives the parking assist function call command as a start command for a teaching drive mode or as a start command for an automated drive mode (regenerative drive) according to the operation.

[0059] In the present disclosure, the parking assist function is a function of the parking assist device 3 that implements a parking assist method for automatically moving a vehicle based on a past parking operation. The parking assist function includes the teaching drive mode in which a teaching path is acquired through teaching drive, and the regeneration drive mode in which the vehicle 1 is moved along the teaching path. The teaching drive mode is implemented by a control process (teaching drive process) executed in response to the teaching drive mode start command. The automated drive mode is implemented by a control process (regeneration drive process) executed in response to the automated drive mode start command.

[0060] In the present disclosure, the teaching drive is a manual drive or an automated drive in which the driver moves the vehicle 1 from a parking start position (teaching drive start position) to a parking end position (teaching drive end position) and parks the vehicle 1 in a desired parking shape. The parking shape includes at least a parking position and a parking attitude (azimuth) of the vehicle 1. Furthermore, the teaching path is a travel path of the vehicle 1 acquired in the teaching drive, i.e., a travel trajectory of the vehicle 1 in the teaching drive. That is, the teaching path includes information indicating the parking shape. Furthermore, the regeneration drive is an automated drive in which the vehicle 1 is moved along a travel path (teaching path) in a parking operation performed in the past.That is, the regeneration drive is an automated parking of the path memory type, in which the vehicle 1 is moved from the parking start position to the parking end position by automated driving.

[0061] It should be noted that the training drive can be a manual drive or an automated drive, in which the vehicle 1 is moved from the parking end position (training drive start position) to the parking start position (training drive end position) in order to park the vehicle 1 in a desired parking configuration. This means that in the regeneration drive according to the invention, the training drive can be regenerated sequentially or reversibly.

[0062] The map information generation module 303 acquires a surroundings map, which defines, as a map, an environment around the vehicle 1, that is, an environment that is a space extending along the travel path of the vehicle 1, in the teaching drive. For example, the map information generation module 303 extracts feature points from each of the captured images of the front, rear, left, and right sides of the vehicle 1, that is, an environment image. The map information generation module 303 creates the surroundings map by defining the surroundings of the vehicle 1 based on the extracted feature points. Here, the surroundings map is an example of travel path information. Furthermore, the surroundings map is information about the surroundings, which is a space extending along the travel path, and is an example of space information.

[0063] Note that the method for extracting feature points is not particularly limited, and any known method can be adopted. For example, the map information generation module 303 extracts feature points from the surrounding image by a method such as the features from the accelerated segment test (FAST) or the oriented FAST and rotated BRIEF (ORB). Further, the map information generation module 303 may preferably record a feature point that satisfies a predetermined condition among the feature points extracted from the surrounding image. For example, among a plurality of surrounding images that are continuous in time, a feature point extracted from a surrounding image showing a longer distance traveled by the vehicle 1 during shooting may preferably be selected as the feature point.

[0064] The map information generation module 303 uses, for example, the SLAM (Simultaneous Localization and Mapping) technique for the extracted feature points to obtain coordinates indicating the position of each feature point and create the surrounding map (local map).

[0065] It should be noted that when detecting the position coordinates of the individual feature points, the position of vehicle 1 is detected in parallel. However, the position of vehicle 1 can be detected by combining the SLAM technique with a dead reckoning (DR) technique. Furthermore, the position, speed, and heading of vehicle 1 can be calculated using an intelligent transportation system (ITS).

[0066] It should be noted that in addition to the spatial information based on the captured image, the environment map can also be defined by spatial information based on the output of the Sonar 15.

[0067] During the teaching run, the travel path acquisition module 304 acquires the travel path of the vehicle 1 during the teaching run based on the result of self-position estimation acquired, for example, by the SLAM technique and the dead reckoning technique, and acquires the travel path as the teaching path. When the travel path is acquired, the travel path is represented by a set of points (waypoints) located at substantially equal distance intervals, and each point is acquired as waypoint group data, which includes, as parameters, coordinates indicating a position, an azimuth indicating a direction of the vehicle 1 at that position, and a traveling speed.

[0068] For example, the travel path detection module 304 detects the teaching path of the vehicle 1 based on the outputs of the azimuth sensor 213 and the GNSS sensor 214 detected by the detection module 301. For example, the travel path detection module 304 detects the teaching path of the vehicle 1 based on the rotation direction and rotation speed of each wheel 13 detected by the detection module 301. Note that the speed and traveling direction of the vehicle 1 can be detected using a vehicle speed pulse or switching information flowing through an on-board network such as CAN.

[0069] The vehicle operation detection module 305 detects an operation of another vehicle accessory in addition to an operation of a vehicle accessory (vehicle component) for traveling with respect to the vehicle 1, and detects the operation as vehicle operation information.

[0070] Here, the vehicle operation information is, for example, information regarding a vehicle operation of the vehicle 1 by the driver, the information indicating the presence or absence, an operation timing, a target, a content, a type, and an operation amount. Note that the vehicle operation information may be information regarding the control of the vehicle 1 according to the operation, the information indicating the presence or absence, a control timing, a target, a content, a type, and a control amount. Furthermore, the vehicle operation information may be information regarding the behavior change of the vehicle 1 associated with the control, the information indicating the presence or absence, an operation timing, a target, a content, a type, and a change amount.It should be noted that the vehicle operation information is not limited to the driver, but may also be information about the operation by a passenger of the vehicle 1 or an operator who operates the vehicle 1 from outside the vehicle.

[0071] The vehicle operation detection module 305 detects, for example, a vehicle operation related to the turn signal during the teaching drive based on, for example, information from the HMI 22 or the turn signal of the vehicle 1 detected by the detection module 301, and detects the detected vehicle operation as vehicle operation information related to a turn signal operation.

[0072] It should be noted that the acquisition of vehicle operation information is not limited to a driving period during which the driver moves the vehicle 1 from the learning start position to the learning end position and parks the vehicle 1 in a desired parking position, but can also be performed in periods before and after the driving period. In the present disclosure, the term "during the learning drive" means a period including a driving period of the learning drive and periods before and after the driving period.

[0073] It should be noted that the vehicle operation information may be acquired during the regeneration drive. In this case, the vehicle operation information acquired during the regeneration drive can be used in the subsequent regeneration drive. That is, in the parking assist device according to the present invention, each regeneration drive can be treated as a teaching drive (past parking operation) for the subsequent regeneration drive. In other words, in the parking assist device according to the present invention, the teaching data can be relearned (updated) based on the regeneration drive.

[0074] The storage unit 306 includes, for example, the ROM 32, the RAM 33, or the HDD 34. The storage unit 306 stores the teaching data generated by the teaching drive process.

[0075] For example, the teaching data includes information indicating the surrounding map generated by the map information generation module 303 and information indicating the travel path (teaching path) during the teaching run, which is acquired by the travel path acquisition module 304. Note that in a case where the vehicle operation is detected during the teaching run, the teaching data also includes the vehicle operation information acquired by the vehicle operation acquisition module 305.

[0076] The self-position estimation module 307 estimates the position and direction (azimuth) of the vehicle 1 in the regeneration travel of the vehicle 1 based on the teaching path by the vehicle control module 311 of the vehicle 1, which will be described later.

[0077] For example, the self-position estimation module 307 reads information about feature points included in the surrounding area map of the teaching data stored in the storage unit 306, and compares the read feature point with the feature point extracted from the current surrounding area image of the vehicle 1 acquired during the regeneration drive, thereby estimating the position and direction of the vehicle 1.

[0078] Note that the position and direction of the vehicle 1 estimated by the self-position estimation module 307 may be a relative position and a relative direction with respect to a reference position and a reference direction, such as a starting position and a starting direction on the detected travel path. However, an absolute position and an absolute direction based on the outputs of the azimuth sensor 213 and the GNSS sensor 214 may also be used.

[0079] In both the teaching run and the regeneration run, the position of the vehicle 1 is not limited to the case where it is determined using the dead reckoning technique based on the amount of movement of the vehicle 1 from the reference position, but may be determined using a known map matching or the GNSS sensor 214.

[0080] During the regeneration travel, the path trajectory calculation module 308 acquires a travel path (teaching path) in the parking operation (teaching travel) performed in the past from the storage unit 306 and calculates a merging path, which is a travel path from the current position of the vehicle 1 to the merging with the teaching path, based on the teaching path and the position of the vehicle 1 estimated by the self-position estimation module 307. The calculated merging path is stored in the storage unit 306.

[0081] The vehicle operation reproduction module 309 reproduces the vehicle behavior during the teaching run in the regeneration run (automated parking) based on the vehicle operation information acquired during the past parking operation (during the teaching run), that is, the vehicle operation information included in the information (teaching data) regarding the teaching path. Specifically, the vehicle operation reproduction module 309 reproduces the operation (vehicle behavior) of the vehicle accessories caused by the operation acquired during the teaching run during the regeneration run based on the vehicle operation information stored in the storage unit 306 and the position of the vehicle 1 estimated by the self-position estimation module 307.In other words, during the regeneration drive, the vehicle operation reproduction module 309 reproduces the vehicle behavior during the teaching drive by using the position (position coordinates) of the vehicle 1 as a trigger. For example, during the automated drive (regeneration drive), the vehicle operation reproduction module 309 reproduces the vehicle behavior during the teaching drive at a time point where it is located at the same position coordinates as the position coordinates where the vehicle operation is started in the teaching drive.

[0082] For example, the vehicle operation reproduction module 309 reproduces the result of the turn signal operation detected by the vehicle operation detection module 305 during the training drive, ie, the vehicle behavior related to the turn signal.

[0083] It should be noted that the reproduction of the vehicle behavior during the training drive in the regeneration drive may mean the reproduction of the vehicle operation itself for vehicle 1, which is detected during the training drive, such as the movement (behavior) of the turn signal lever. Alternatively, the reproduction may also mean the reproduction of the control of vehicle 1 according to the vehicle operation, such as the simulation of a signal from the turn signal lever (HMI 22). Furthermore, the reproduction of the vehicle behavior is not limited to the case of being realized by the same vehicle controller as during the training drive, but may also be realized by a different vehicle controller, for example, by generating a control signal supplied to the turn signal so that the behavior of the turn signal is reproduced.

[0084] Furthermore, the vehicle behavior reproduced during the regeneration run may be the same as the vehicle behavior detected during the training run or may correspond to the vehicle behavior detected during the training run. That is, the vehicle behavior reproduced during the regeneration run may be different from the vehicle behavior detected during the training run. In other words, the reproduction of the operation of the vehicle accessory includes the operation of the vehicle accessory of the vehicle 1 by the vehicle controller according to the vehicle controller after changing the vehicle controller based on the operation detected during the training run.

[0085] For example, in a case where the vehicle operation detected by the vehicle operation detection module 305 during the teaching drive does not correspond to a predetermined reference stored in the storage unit 306, the vehicle operation reproduction module 309 may distinguish the turn signal operation to be reproduced from that during the teaching drive. For example, in a case where the time for which the turn signal flashes during the teaching drive—that is, the operation time based on the turn signal operation detected during the teaching drive—is a short time that does not meet a predetermined time (reference), the vehicle operation reproduction module 309 may extend the operation time of the turn signal so that the turn signal flashes for a predetermined time or longer.In other words, the vehicle operation playback module 309 sets the operation time of the turn signal during the automatic parking to be longer than the operation time of the turn signal during the teaching drive.

[0086] Note that extending the turn signal operation time means advancing the operation start time or delaying the operation end time relative to the turn signal operation time. Furthermore, when the time the turn signal flashes during the teaching run—that is, the operation time based on the turn signal operation acquired during the teaching run—is long and exceeds the predetermined time (reference). In other words, the vehicle operation playback module 309 can shorten the turn signal operation time during automatic parking compared to the turn signal operation time during the teaching run.

[0087] For example, the vehicle operation playback module 309 may vary the vehicle operations detected by the vehicle operation detection module 305 during the teaching drive depending on the environmental situation of the vehicle 1 during the regeneration drive, based on the information regarding the environmental situation of the vehicle 1.

[0088] For example, the vehicle operation playback module 309 may change the presence or absence of the turn signal operation from the time of the learning drive based on the width of the roadway, for example, when parallel parking is performed from the traveling road (lane) of the vehicle 1 to a parking space along the roadway. For example, in a case where the width of the roadway is larger than a predetermined reference stored in the storage unit 306, the vehicle operation playback module 309 may change the turn signal behavior so that the turn signal flashes in a case where the turn signal operation is not detected during the learning drive, or may change the turn signal behavior so that the turn signal flashes for a longer time than during the learning drive or at an earlier time.Alternatively, the vehicle operation playback module 309 may change the behavior of the turn signal so that the turn signal flashes in a case where the width of the roadway is smaller than a predetermined reference stored in the storage unit 306.

[0089] For example, if a person or another vehicle is detected in the vicinity of the vehicle 1 during automatic parking, the vehicle operation playback module 309 may change the operation timing of the vehicle accessory or add the operation of the vehicle accessory. For example, in a case where a person or another vehicle is detected in the vicinity, the vehicle operation playback module 309 may change the behavior of the turn signal so that the turn signal flashes even in a case where the turn signal operation is not detected during the teaching drive. For example, in a case where a person or another vehicle is detected in the vicinity, the vehicle operation playback module 309 may change the behavior of the turn signal so that the turn signal flashes for a longer operation time than that during the teaching drive time or at an earlier operation time.

[0090] The output control module 310 generates assistance information related to the parking assistance function. Furthermore, the output control module 310 presents the generated assistance information to the user through the HMI 22. Specifically, the output control module 310 outputs to the HMI 22, as the assistance information, assistance information related to the parking assistance function and information for displaying an operation icon by the HMI 22 upon receipt of an operation input from the driver.

[0091] As an example, the output control module 310 outputs information for displaying a screen with an operation icon for an instruction to call the parking assistance function to the HMI 22 as assistance information.

[0092] For example, the output control module 310 outputs information for displaying a menu screen for displaying each menu of the parking assist function to the HMI 22 as the assistance information. For example, the menu screen includes an operation icon for an instruction to start the teaching drive mode and an operation icon for an instruction to start the automated driving mode (regeneration drive).

[0093] As an example, the output control module 310 outputs information for displaying a screen containing a top view image to the HMI 22 as the assistance information in the teaching drive mode and / or the automated driving mode. In this case, the top view image is a surrounding view image that displays information about a surrounding space of the vehicle 1. For example, the top view image is a bird's-eye view image of the vehicle 1 generated by combining surrounding images of the surroundings (e.g., front, rear, left, and right) of the vehicle 1 obtained from the surround camera 212.

[0094] For example, the display screen including the top view image further includes displaying information indicating the state of the vehicle 1. Here, the display of information indicating the state of the vehicle 1 is intended to display the vehicle behavior of the vehicle 1 in a pseudo-manner. That is, the assistance information of the output control module 310 may include information for displaying the operation of the vehicle accessories of the vehicle 1 in a pseudo-manner. This display allows the driver to easily grasp the reproduced vehicle behavior. Therefore, the driver can take measures such as stopping an unnecessary operation of the vehicle accessories or manually adding an operation of the vehicle accessories that has not been reproduced.

[0095] For example, the pseudo-display of the vehicle behavior of the vehicle 1, that is, the operation of the vehicle accessories, can be realized by superimposing and displaying an icon simulating the vehicle 1 on the top view image of the vehicle 1. That is, the output control module 310 can output information for displaying a screen on which the icon simulating the vehicle 1 is superimposed and displayed on the top view image of the vehicle 1 to the HMI 22 as the support information in the teaching drive mode and / or the automated drive mode. Here, the icon simulating the vehicle 1 can be, for example, a CG image simulating the vehicle 1, such as a 3D vehicle model, or it can be an icon indicating the vehicle accessories mounted on the vehicle 1.

[0096] For example, the output control module 310 may change the display of the vehicle accessory by superimposing and displaying the icon simulating the vehicle 1 on the top view image in conjunction with the reproduction of the operation of the vehicle accessory based on the teaching data. For example, in a case where the turn signal is turned on based on the vehicle operation information during automatic parking, the turn signal is also turned on on the icon in conjunction with the reproduction. For example, in a case where the side mirror is stored (folded) based on the vehicle operation information during automatic parking, the side mirror is also stored on the icon in conjunction with the reproduction.

[0097] It should be noted that the information indicating the status of vehicle 1 may be information indicating an operation plan of the vehicle accessories. For example, a display screen is displayed on the HMI 22, on which icons of pins or flags or text are superimposed and displayed at corresponding positions on the instruction path, so that an event (operation of the vehicle accessory) at that position can be displayed to the driver. In addition, the operation plan of the vehicle accessories can be edited on the screen.

[0098] It should be noted that the information indicating the status of the vehicle 1 is not limited to being overlaid and displayed in the top view. For example, only a CG image or icon simulating the vehicle 1 may be displayed on the display screen, or only an icon or text indicating the operation of the vehicle accessories may be displayed on the HMI 22.

[0099] It should be noted that in the automated driving mode, the display screen containing the top view image may also contain information about the teaching path, such as the parking start position, the parking end position, and the driving path (path).

[0100] The vehicle control module 311 controls the regeneration drive, in which the vehicle 1 moves along the teaching path and stops in a predetermined parking position based on the information (teaching data) regarding the teaching path. Specifically, the vehicle control module 311 controls the steering, braking, and acceleration / deceleration of the vehicle 1 to follow the teaching path.

[0101] For example, the vehicle control module 311 determines a difference between the merging path and the teaching path, as well as the position and direction of the vehicle 1, based on the merging path and the teaching path stored in the storage unit 306 and the position and direction of the vehicle 1 estimated by the self-position estimation module 307. Then, the vehicle control module 311 controls at least one of the steering, braking, and acceleration / deceleration control variables of the vehicle 1 to reduce the detected difference, thereby causing the vehicle 1 to follow the merging path and the teaching path and controlling the regeneration drive.

[0102] Next, the flow of a parking assistance process executed by the parking assistance device 3 configured as described above will be described. As described above, the parking assistance process executed by the parking assistance device 3 includes the teaching drive process and the regeneration drive process.

[0103] Fig. Fig. 5 is a flowchart showing an example of the flow of the teaching driving process performed by the parking assist device 3 of Fig. 1 is executed. The sequence of Fig. 5 is started, for example, after the instruction to call the parking assist function is received in a state where the vehicle 1 has been stopped at the teaching start position (parking start position).

[0104] The parking assistance device 3 determines whether there is an instruction from the driver to start the learning mode (S101). If there is no instruction to start the learning mode (S101: No), the process returns from Fig. 5 returns to the process of S101. For example, the parking assist device 3 repeats the process of S101 at predetermined intervals (e.g., at an interval of 100 ms).

[0105] If the instruction to start the teaching drive mode is given (S101: Yes), the parking assistance device 3 determines whether further driving operation of the vehicle by the driver has been detected. If no vehicle operation is detected (S102: No), the parking assistance device 3 temporarily stores the position of the vehicle 1 and the surrounding map in the storage unit 306 (S103). In a case where the vehicle operation is detected (S102: Yes), the parking assistance device 3 temporarily stores the vehicle operation information together with the position of the vehicle 1 and the surrounding map in the storage unit 306 (S104). This vehicle operation information can be stored, for example, in conjunction with the position of the vehicle 1.

[0106] The parking assistance device 3 determines whether there is an instruction from the driver to end the learning mode (S105). If there is no instruction to end the learning mode (S105: No), the process returns from Fig. 5 returns to the process of S102. As described above, the position of the vehicle 1 is stored in chronological order, and the position data in chronological order is data indicating the movement path during the teaching run. In a case where there is an instruction to end the teaching run mode (S105: Yes), the parking assist device 3 generates teaching data to be referred to during the automated driving mode (regenerative driving process) based on the travel path (teaching path) indicated by the position data stored in chronological order during the teaching run, the surrounding map along the travel path, and the vehicle operation information, and stores the generated teaching data in the storage unit 306 (S106). Thereafter, the process of Fig. 5.

[0107] Fig. Fig. 6 is a flowchart showing an example of the flow of the regeneration process performed by the parking assist device 3 of Fig. 1 is executed. The sequence of Fig. 6 is started, for example, after the parking assist function call command is received in a state where the vehicle 1 has been stopped at the parking start position.

[0108] The parking assistance device 3 determines whether there is a driver instruction to start the automated driving mode (S201). If there is no instruction to start the automated driving mode (S201: No), the process returns from Fig. 6 returns to the process of S201. For example, the parking assist device 3 repeats the process of S201 at predetermined intervals (e.g., at an interval of 100 ms).

[0109] When the instruction to start the automated driving mode is given (S201: Yes), the parking assist device 3 reads the teaching data from the storage unit 306 (S202). After that, while estimating its own vehicle position (S203), the parking assist device 3 moves the vehicle 1 by performing feedback control to conform to a target path (teaching path) of the teaching data (S204), and reproduces the vehicle operation (vehicle behavior) during the teaching drive based on the vehicle operation information of the teaching data (S205).

[0110] Thereafter, the parking assistance device 3 determines whether the vehicle 1 has reached the target position (parking completion position) of the target path specified by the teaching data (S206). In a case where the vehicle has not reached the parking completion position (S206: No), the flow returns from Fig. 6 returns to the process of S203. When the vehicle 1 has reached the parking position (S206: Yes), the parking assist device 3 stops the vehicle 1, and the process of Fig. 6 ends.

[0111] As described above, the parking assist device 3 according to the embodiment is configured to acquire the vehicle operation information related to the automated parking with path storage in the teaching drive mode and the operation of the turn signal (vehicle accessory) of the vehicle 1 along the teaching path during the teaching drive. Furthermore, in the automated driving mode, the parking assist device 3 is configured to move the vehicle 1 along the travel path (teaching path) during the past parking operation and reproduce the turn signal operation caused by the operation acquired during the past parking operation (teaching drive).

[0112] According to this configuration, in automatic parking with path memory, in which the vehicle 1 automatically drives based on the parking operation performed in the past, it is possible to perform automatic control of vehicle accessories such as the turn signal, and improve safety and convenience for the surroundings. Thus, according to the above configuration, it is possible to appropriately support automatic parking with path memory.

[0113] It should be noted that in the above-described embodiment, the turn signal (direction indicator) mounted on the vehicle 1 was exemplified as the vehicle accessory of a target for acquiring the vehicle operation information, that is, the target for reproducing the operation, but the present invention is not limited to this. The target vehicle operation acquired as the vehicle operation information only needs to be a vehicle operation that can improve the safety of the environment during regeneration driving, the safety of the vehicle 1 or the driver, or the driver's convenience by recording and reproducing it. For example, the target vehicle operation is not limited to the turn signal operation, but may consist of various vehicle accessory operations such as:Operating the side mirror, operating the hazard warning lights (emergency flashing lights), operating the headlights, operating the position lights (outer contour lights), and operating the doors. By reproducing these processes, it is possible to improve the safety around vehicle 1 as well as the safety and comfort of vehicle 1 (the driver).

[0114] The vehicle accessory of the target for reproducing the operation in the parking assist device 3 according to the above-described embodiment may be, for example, the side mirror mounted on the vehicle 1 in addition to or instead of the turn signal.

[0115] For example, the parking assistance device 3 can detect that the side mirror is folded (stored) during the learning drive, such as when parking in a tight parking space. For example, the vehicle operation detection module 305 can detect the vehicle operation related to the development / storage of the side mirror during the learning drive, for example, based on the information from the HMI 22 and the side mirror acquired by the detection module 301, and record the vehicle operation as the vehicle operation information related to the side mirror operation.

[0116] For example, the vehicle operation playback module 309 may change the operation (vehicle behavior) of the side mirror displayed during the regeneration drive compared to the vehicle behavior detected during the training drive, based on the information about the surrounding situation of the vehicle 1.

[0117] For example, the vehicle operation playback module 309 may change the presence or absence of the operation of the side mirror from the time of the teaching run based on the width of the parking space of the vehicle 1, that is, the size of the surrounding space of the vehicle 1 at the parking completion position. For example, if the width of the parking space is smaller than a predetermined width (reference) determined in advance and stored in the storage unit 306, the vehicle operation playback module 309 may change the behavior of the side mirror to fold even if the operation of the side mirror is not detected during the teaching run, or may advance the operation timing of the side mirror to fold at a time before the time of the teaching run (for example, before the start of automatic parking).Note that the predetermined width used as a reference is obtained, for example, by adding a predetermined margin considering the risk of collision or contact to the size of the vehicle 1. As described above, according to the configuration in which automatic parking is performed with a margin provided according to the environmental situation of the vehicle 1, it is possible to reduce the risk of collision or contact and improve safety during automatic parking.

[0118] For example, when a person or another vehicle is near the vehicle, the vehicle operation playback module 309 may add an operation to fold the side mirror even if the operation of the side mirror is not detected during the teaching drive, or advance the operation timing of the side mirror so that it is folded at an earlier time than during the teaching drive.

[0119] For example, the vehicle accessory of the target for reproducing the operation in the parking assist device 3 according to the above-described embodiment may be a hazard warning light mounted on the vehicle 1 in addition to or instead of the turn signal and / or the side mirror. The vehicle accessory such as the hazard warning light may be the same vehicle accessory as the turn signal or a different vehicle accessory.

[0120] The vehicle accessory of the target for simulating the operation in the parking assistance device 3 according to the above-described embodiment may, for example, be a lamp mounted on the vehicle 1 in addition to or instead of the turn signal and / or the side mirror. The lamp includes, for example, at least a headlight and a position lamp (outer contour lamp).

[0121] For example, the parking assistance device 3 may detect that the hazard warning lights or lights are turned on / off during the training run. For example, the vehicle operation detection module 305 may detect a vehicle operation related to turning the hazard warning lights or lights on / off during the training run, for example, based on the information from the HMI 22, the hazard warning lights, or the lights detected by the detection module 301, and detect the vehicle operation as vehicle operation information related to the hazard warning light operation or the light operation.

[0122] For example, the vehicle operation playback module 309 may change the operation (vehicle behavior) of the hazard warning lights or the light played back during the regeneration drive based on the information about the surrounding situation of the vehicle 1 from the vehicle behavior acquired during the teaching drive.

[0123] For example, if a person or another vehicle is nearby, even if the operation of the hazard warning lights or lights is not detected during the learning trip, the vehicle operation playback module 309 may add an operation to turn on, advance the operation timing to turn on at an earlier time than the learning trip time, or delay the operation timing to turn off at a later time than the learning trip time.

[0124] Furthermore, the parking assist device 3 may acquire, for example, an output from an in-vehicle sensor, such as a raindrop sensor mounted on the vehicle 1, a sensor that detects the operation of the windshield wiper, or an illumination sensor, as information about the weather or illuminance. Furthermore, the parking assist device 3 may acquire information about the weather on the Internet, for example, by communicating with the exterior of the vehicle 1 via the I / F 35. Furthermore, the parking assist device 3 may acquire information about the illuminance in the surroundings of the vehicle 1, for example, in a tunnel or a multi-story parking garage, based on an output of the GNSS sensor 214 and map data or an output of a car navigation device mounted on the vehicle 1.

[0125] For example, the vehicle operation playback module 309 may change the vehicle behavior detected during the training run based on at least one of the weather and illuminance factors during automatic parking. For example, the vehicle operation playback module 309 may change the vehicle behavior from the vehicle behavior detected during the training run when the weather or illuminance meets a predetermined condition (reference) determined in advance and stored in the storage unit, such as during rainy or cloudy weather, at night, in a tunnel, or in a multi-story parking garage.For example, in a case where the weather or the illuminance satisfies the predetermined condition (reference), even in a case where the operation of the hazard warning lights or the light is not detected during the teaching run, the vehicle operation playback module 309 may add an operation to turn on, advance the operation timing to turn on at an earlier time than the teaching run, or delay the operation timing to turn off at a later time than the teaching run.

[0126] As an example, the vehicle accessory of the target for reproducing the operation in the parking assist device 3 according to the above-described embodiment may be a door mounted on the vehicle 1 in addition to or instead of at least one of the turn signals and the side mirror.

[0127] For example, the parking assist device 3 may detect the door opening degree, such as the opening / closing amount and the opening / closing speed, during the teaching run, for example, at the parking completion position. For example, the vehicle operation detection module 305 may detect a vehicle operation related to the door opening degree during the teaching run, for example, based on information from the HMI 22 or an opening / closing sensor (in-vehicle sensor) provided on the door and detected by the detection module 301, and detect the vehicle operation as vehicle operation information related to the door opening degree.

[0128] For example, the vehicle operation playback module 309 may limit the operation of the door during the regeneration run based on vehicle operation information related to the opening / closing amount and / or the opening / closing speed of the door of the vehicle 1. For example, in a case where the door is only opened halfway during the teaching run, the vehicle operation playback module 309 may limit the opening / closing amount of the door so that the door is only opened halfway during the regeneration run. For example, in a case where the door is opened slowly during the teaching run, the vehicle operation playback module 309 may limit the opening / closing speed of the door so that the door is only opened at a speed equal to or lower than the speed in the teaching run during the regeneration run.

[0129] In a case where the vehicle 1 is on a road such as a public road (in other words, excluding the parking lot and the private road), the parking assist device 3 does not need to receive a teaching mode start instruction or an automated driving mode start instruction (regeneration drive) based on, for example, the output of the GNSS sensor 214 and the map data or the output of the vehicle navigation device mounted on the vehicle 1. For example, the output control module 310 may change the operation icon for an instruction to start the teaching mode or the operation icon for an instruction to start the automated driving mode to be grayed out on the menu screen for displaying each menu of the parking assist function, and output the grayed out icon to the HMI 22.Since the vehicle is on a public road, the output control module 310 may alternatively notify the user that the start of the learning drive mode is not received or that the start of the regeneration drive is not received. Note that the output control module 310 may output a notification on the menu screen, a screen transition from the menu screen to a notification, a notification through a pop-up display, or a notification through voice or a notification tone.

[0130] Note that in the above embodiment, “determine whether it is A” may mean “determine that it is A,” “determine that it is not A,” or “determine whether it is A.”

[0131] The program executed by the parking assist device 3 in the above-described embodiment may be provided by recording on a computer-readable recording medium such as a CD-ROM, an FD, a CD-R, a DVD, or an SD card as a file in an installable format or an executable format.

[0132] Furthermore, the program executed by the parking assistance device 3 in the above-described embodiment may be stored on a computer connected to a network such as the Internet and provided by downloading over the network. Furthermore, the program executed by the parking assistance device 3 may be provided or distributed over a network such as the Internet.

[0133] Moreover, the program executed by the parking assist device 3 in the above-described embodiment may be configured to be stored in advance in the ROM or the like.

[0134] Furthermore, the program executed by the parking assist device 3 in the above-described embodiment has a module configuration including the above-described functional units (acquisition module 301, reception module 302, map information generation module 303, travel path acquisition module 304, vehicle operation acquisition module 305, storage unit 306, self-position estimation module 307, path following calculation module 308, vehicle operation reproduction module 309, output control module 310, and vehicle control module 311). As the actual hardware, the CPU 31 reads the program from the ROM 32 or the HDD 34 and executes it, loading the functional units into the RAM 33, and generating the functional units in the RAM 33.

[0135] According to at least one of the embodiments described above, it is possible to support automatic parking in a suitable manner in the manner of a path memory.

[0136] According to the present disclosure, it is possible to appropriately support automatic parking in the form of a path memory.

[0137] Although specific embodiments have been described, these embodiments have been presented only by way of example and are not intended to limit the scope of the inventions. Indeed, the novel methods and systems described herein may be embodied in a variety of other forms; further, various omissions, substitutions, and changes may be made in the form of the methods and systems described herein without departing from the spirit of the inventions. The appended claims and their equivalents are intended to cover such forms or changes as fall within the scope and spirit of the inventions. Additional notes

[0138] The following techniques are disclosed by the above description of the above embodiment. (1) A parking assistance method for performing automatic driving of a vehicle based on a parking operation performed in the past, the method comprising: Creating an educational trail based on a driving path during a past parking session; Collecting vehicle operation information relating to operation of at least one vehicle accessory of a turn signal and a side mirror in the vehicle during the parking operation performed in the past; Storing teaching data including the teaching path and vehicle operating information; and Reproducing an operation of the vehicle accessory caused by the operation detected during the parking operation performed in the past based on the teaching data in automated parking in which the vehicle is moved along the teaching path. (2) In the parking assistance method according to (1), reproducing the operation of the vehicle accessory includes operating the vehicle accessory after changing the vehicle control based on the operation detected during the parking operation performed in the past. (3) In the parking assist method according to (2), the changing of the vehicle control includes extending an operation time of the turn signal to be equal to or longer than a predetermined time when an operation time based on the operation of the turn signal detected during the parking operation performed in the past is shorter than the predetermined time. (4) In the parking assist method according to (2) or (3), changing the vehicle control includes changing the presence or absence of the operation of the turn signal based on a width of a lane of the vehicle during the parking operation performed in the past. (5) In the parking assist method according to any one of (2) to (4), changing the vehicle control includes changing the presence or absence of the operation of the side mirror based on a width of a parking space of the vehicle (1) during the parking operation performed in the past. (6) In the parking assist method according to any one of (2) to (5), the changing of the vehicle control includes advancing an operation timing of the side mirror when a width of a parking space of the vehicle during the parking operation performed in the past is narrower than a predetermined width. (7) In the parking assist method according to any one of (2) to (6), changing the vehicle control includes changing an operation timing or adding an operation for the turn signal and / or the side mirror when a person or another vehicle is detected near the vehicle during automatic parking. (8) In the parking assistance method according to any one of (1) to (7), the vehicle accessories of the vehicle (1) also include at least one hazard warning light, one headlight, one position light and one door. (9) In the parking assistance method according to any one of (1) to (8), further detecting the vehicle operation information relating to the operation of a headlight of the vehicle (1) during the parking operation carried out in the past; and include reproducing the operation of the headlight during automatic parking caused by the operation detected during the parking operation performed in the past, based on the teaching data, wherein reproducing the operation of the headlight includes operating the headlight after changing the vehicle control based on the operation detected during the parking operation performed in the past, based on at least one of weather and illuminance during automatic parking. (10) In the parking assist method according to any one of (1) to (9), further acquiring the vehicle operation information relating to at least one of an opening / closing amount and an opening / closing speed of a door of the vehicle during the parking operation performed in the past; and a limitation of door operation based on the teaching data in the automatic parking system. (11) In the parking assistance method according to any one of (1) to (10), there is further provided an output of assistance information for displaying a display screen on which an icon simulating the vehicle is superimposed and displayed on a bird's-eye view image of the vehicle generated by combining surrounding images of the vehicle; and a change of a vehicle accessory display by the symbol in connection with the operation of the vehicle accessory during automatic parking. (12) A parking assistance device that performs automatic driving of a vehicle based on a parking operation performed in the past, the parking assistance device comprising a learning module that acquires a learning path based on a travel path during the past parking operation and acquires vehicle operation information regarding an operation of at least one vehicle accessory of a turn signal and a side mirror in the vehicle during the past parking operation; a storage unit that stores teaching data including the teaching path and vehicle operation information; and a regeneration module that reproduces an operation of the vehicle accessory caused by the operation detected during the parking operation performed in the past, based on the teaching data during automatic parking in which the vehicle is moved along the teaching path. (13) A parking aid with at least one processor and a memory in which at least one program is stored which is executed by at least one processor, in which at least one processor is configured to carry out the parking assistance method according to one of points (1) to (11) by executing at least one program. (14) A program that causes a computer to carry out the parking assistance procedure according to any one of items (1) to (11). (15) A computer program product in which the program according to (14) is recorded and which is executed by a computer. (16) A vehicle with the parking assistance device according to (12) or (13); and a control unit that receives operation of the vehicle accessory by a user. (17) A vehicle with the parking assistance device according to (13), in which at least one processor is configured to execute the parking assistance method according to (11); and a display device that displays the display screen based on the assistance information from the parking assistance device. (18) A vehicle with the parking assistance device according to (12) or (13); and a control device that controls at least a steering angle, driving, and braking based on a control signal from the regeneration module. (19) A vehicle with the parking assistance device according to (12) or (13); and at least one of a sonar, an all-round camera, a steering angle sensor, a wheel speed sensor, an azimuth sensor and a GNSS sensor.

Claims

[1] Parking assistance method for carrying out an automatic drive of a vehicle (1) on the basis of a parking operation carried out in the past, comprising: Creating an educational trail based on a driving path during a past parking session; Acquiring vehicle operation information relating to an operation of at least one vehicle accessory of a turn signal and a side mirror in the vehicle (1) during the parking operation performed in the past; Storing teaching data including the teaching path and vehicle operating information; and Reproducing an operation of the vehicle accessory caused by the operation detected during the parking operation performed in the past based on the teaching data in automated parking in which the vehicle (1) is moved along the teaching path. [2] The parking assist method according to claim 1, wherein reproducing the operation of the vehicle accessory comprises operating the vehicle accessory after changing the vehicle control based on the operation detected during the parking operation performed in the past. [3] The parking assist method according to claim 2, wherein changing the vehicle control includes extending an operation time of the turn signal to be equal to or longer than a predetermined time when an operation time based on the operation of the turn signal detected during the parking operation performed in the past is shorter than the predetermined time. [4] The parking assist method according to claim 2 or 3, wherein changing the vehicle control includes changing the presence or absence of the operation of the turn signal based on a width of a lane of the vehicle (1) during the parking operation performed in the past. [5] The parking assist method according to any one of claims 2 to 4, wherein changing the vehicle control includes changing the presence or absence of the operation of the side mirror based on a width of a parking space of the vehicle (1) during the parking operation performed in the past. [6] A parking assistance method according to any one of claims 2 to 5, wherein changing the vehicle control includes advancing an operation timing of the side mirror when a width of a parking space of the vehicle (1) during the parking operation performed in the past is narrower than a predetermined width. [7] A parking assistance method according to any one of claims 2 to 6, wherein changing the vehicle control comprises changing an operation timing or adding an operation for the turn signal and / or the side mirror when a person or another vehicle (1) is detected near the vehicle (1) during automatic parking. [8] Parking assistance method according to one of claims 1 to 7, wherein the vehicle accessory of the vehicle (1) further comprises at least one hazard warning light, one headlight, one position light and one door. [9] Parking assistance method according to one of claims 1 to 8, further comprising: further recording the vehicle operation information relating to the operation of a headlight of the vehicle (1) during the parking operation carried out in the past; and Reproducing the headlight operation during automatic parking caused by the operation detected during the parking operation performed in the past based on the teaching data, where reproducing the operation of the headlight includes operating the headlight after changing the vehicle control based on the operation detected during the parking operation performed in the past, based on at least one of weather and illuminance during automatic parking. [10] Parking assistance method according to one of claims 1 to 9, further comprising: further detecting the vehicle operation information relating to at least one of an opening / closing amount and an opening / closing speed of a door of the vehicle (1) during the parking operation performed in the past; and Limiting door operation based on teaching data in automatic parking system. [11] Parking assistance method according to one of claims 1 to 10, further comprising: Outputting support information for displaying a display screen on which a symbol simulating the vehicle (1) is superimposed and displayed on a bird's-eye view image of the vehicle (1) generated by combining surrounding images of the vehicle (1); and Changing a vehicle accessory display by the symbol in conjunction with the vehicle accessory operation during automatic parking. [12] Parking assistance device (3) which carries out an automatic drive of a vehicle (1) on the basis of a parking operation carried out in the past, comprising: a learning module (301, 303, 304, 305) that acquires a teaching path based on a travel path during the parking operation performed in the past and acquires vehicle operation information regarding an operation of at least one vehicle accessory of a turn signal and a side mirror in the vehicle (1) during the parking operation performed in the past; a storage unit (306) that stores teaching data including the teaching path and the vehicle operation information; and a regeneration module (307, 308, 309, 311) that reproduces an operation of the vehicle accessory caused by the operation detected during the parking operation performed in the past, based on the teaching data during automatic parking in which the vehicle (1) is moved along the teaching path.

Images