Parking Assistance Device

The parking assistance device addresses the inefficiencies of conventional systems by using a parking frame detection unit and occupant-based instructions to automatically park, ensuring quick and disturbance-free parking.

JP7578372B2Active Publication Date: 2024-11-06PANASONIC AUTOMOTIVE SYST CO LTD
View PDF 5 Cites 0 Cited by

Patent Information

Application Number
JP2023122398
Authority / Receiving Office
JP · JP
Patent Type
Patents
Current Assignee / Owner
Priority Date
2022-10-28
Filing Date
2023-07-27
Publication Date
2024-11-06
Estimated Expiration
2043-07-27

AI Technical Summary

Technical Problem

Conventional parking assist devices require a long time for parking due to the need to detect a parking space and request safety confirmation, often leading to honking from vehicles behind.

Method used

A parking assistance device that includes a parking frame detection unit, storage unit, instruction detection unit, and vehicle control unit to automatically park based on the vehicle's operation history and occupant instructions, such as releasing the steering wheel or hazard lamps, allowing for quick detection and execution of parking instructions.

Benefits of technology

Enables smooth automatic parking in a shorter time without the need for additional safety confirmations, reducing the overall parking time and minimizing disturbances from following vehicles.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 0007578372000001
    Figure 0007578372000001
  • Figure 0007578372000002
    Figure 0007578372000002
  • Figure 0007578372000003
    Figure 0007578372000003
Patent Text Reader

Abstract

To provide a parking support apparatus and a parking support method that enable smooth automatic parking in a short period of time.SOLUTION: A parking support apparatus includes: a parking frame detection unit that detects a parking frame on the basis of a peripheral image showing the periphery of a vehicle; a storage unit that stores the history of any operation of an occupant of the vehicle or of any movement of the vehicle; an instruction detection unit that detects a parking instruction of the occupant on the basis of the history; and a vehicle control unit that automatically parks the vehicle after the instruction detection unit detects the parking instruction. The instruction detection unit determines that the parking instruction has been detected when the history indicates that the vehicle has stopped after turning by steering from straight traveling, and the occupant has performed a prescribed parking instruction operation.SELECTED DRAWING: Figure 4
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present disclosure relates to a parking assistance device and a parking assistance method. [Background technology]

[0002] Conventionally, a parking assistance device is known that can automatically park a vehicle in a parking space by turning the vehicle at a predetermined position around the parking space and driving backward from the position turned at a predetermined angle. For example, Patent Document 1 discloses a configuration in which a vehicle is parked next to a parking space, a parking assistance device is activated, and a parking position of the vehicle in the parking space is uniquely determined. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent Publication No. 2022-69161 Summary of the Invention [Problem to be solved by the invention]

[0004] However, in the conventional technology, after the parking assistance device is activated, it is necessary to detect the parking space and to confirm the safety of the area through which the vehicle passes when parking, which results in the vehicle staying in the passage for a long time. As a result, the time required for parking is longer overall, and the following vehicle may honk the horn while the parking assistance device is being operated.

[0005] An object of the present disclosure is to provide a parking assistance device and a parking assistance method that are capable of performing automatic parking smoothly in a short amount of time. [Means for solving the problem]

[0006] The parking assistance device according to the present disclosure is A parking space detection unit that detects a parking space based on a surrounding image showing the surroundings of the vehicle; A storage unit that stores a history of either an operation by a passenger of the vehicle or a movement of the vehicle; an instruction detection unit that detects a parking instruction from the occupant based on the history; a vehicle control unit that performs automatic parking of the vehicle after the instruction detection unit detects the parking instruction; Equipped with The instruction detection unit determines that the parking instruction has been detected when the history indicates that the vehicle has stopped after being steered from a straight line to a turn and the occupant has performed a predetermined parking instruction operation. A parking assistance device, The predetermined parking instruction operation includes at least one of releasing the steering wheel, releasing the brake, turning on the hazard lights, and operating a button corresponding to a parking instruction. .

[0007] The parking assistance method according to the present disclosure includes: Detecting a parking space based on a surrounding image showing the surroundings of the vehicle; storing a history of either the operation of an occupant of the vehicle or the motion of the vehicle; detecting a parking instruction from the occupant based on the history; after detecting the parking instruction, automatically parking the vehicle; having In the step of detecting the parking instruction, if the history indicates that the vehicle has stopped after steering from a straight line and turning, and if the occupant has performed a specified parking instruction operation, it is determined that the parking instruction has been detected. Effect of the Invention

[0008] According to the present disclosure, automatic parking can be performed smoothly in a short time. [Brief description of the drawings]

[0009] [Figure 1] FIG. 1 is a diagram for explaining automatic parking by a vehicle to which a parking assistance device according to a first embodiment of the present disclosure is applied. [Diagram 2] 1 is a diagram showing a vehicle to which a parking assistance device according to an embodiment of the present invention can be applied. [Diagram 3]FIG. 2 is a diagram showing a network configuration of a system to which a parking assistance device is applied. [Figure 4] FIG. 2 is a block diagram showing hardware that implements the functions of the parking assistance device. [Diagram 5] FIG. 2 is a block diagram showing a parking assistance device. [Figure 6] 10A and 10B are diagrams for explaining setting of a parking space detection area; [Figure 7] 1A and 1B are diagrams illustrating an example of parking space detection using a rear camera. [Figure 8] FIG. 13 is a diagram showing an example in which the camera cannot capture the entire parking space; [Figure 9] FIG. 13 is a diagram showing an example in which the entire parking space is captured by the front camera. [Figure 10] FIG. 11 is a diagram illustrating an example in which a plurality of parking spaces are detected. [Figure 11] 4 is a flowchart showing an example of the operation of parking assistance control in the parking assistance device according to the first embodiment. [Figure 12] 4 is a flowchart showing an example of the operation of parking assistance control in the parking assistance device according to the first embodiment. [Figure 13] 4 is a flowchart showing an example of the operation of parking assistance control in the parking assistance device according to the first embodiment. [Figure 14] FIG. 11 is a diagram for explaining an example of parking a vehicle in the second embodiment. [Figure 15] 10 is a flowchart showing an example of the operation of parking assistance control in the parking assistance device according to the second embodiment. [Figure 16] 10 is a flowchart showing an example of the operation of parking assistance control in the parking assistance device according to the second embodiment. [Figure 17] 10 is a flowchart showing an example of the operation of parking assistance control in the parking assistance device according to the second embodiment. [Figure 18] FIG. 13 is a diagram for explaining an example of a steering action of a vehicle in the third embodiment. [Figure 19] FIG. 13 is a diagram for explaining an example of a steering action of a vehicle in the third embodiment. [Figure 20] FIG. 13 is a diagram for explaining an example of a steering action of a vehicle in the third embodiment. [Figure 21] FIG. 13 is a diagram for explaining an example of a steering action of a vehicle in the third embodiment. [Figure 22] FIG. 13 is a diagram for explaining an example of a steering action of a vehicle in the third embodiment. [Diagram 23] FIG. 13 is a diagram for explaining an example of a steering action of a vehicle in the third embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] (First embodiment) Hereinafter, the first embodiment of the present disclosure will be described in detail with reference to the drawings. FIG. 1 is a diagram for explaining automatic parking by a vehicle 1 to which a parking assistance device 100 according to the first embodiment of the present disclosure is applied. FIG. 2 is a diagram showing a vehicle 1 to which the parking assistance device 100 according to the present embodiment can be applied. In the explanation of FIG. 1 and the like, a Cartesian coordinate system (X, Y) is used. The Cartesian coordinate system (X, Y) is also used in the drawings described later. For example, the X direction indicates the left-right direction relative to the parking position (the position of the vehicle 1 in the parking space), and the Y direction indicates the front-rear direction relative to the parking position. In addition, when expressing the position of the vehicle 1, it is expressed as the position of a representative point of the vehicle 1 in principle. The representative point of the vehicle 1 is the midpoint of the ground contact points of the two rear wheels. In addition, when expressing the trajectory of the movement of the vehicle 1, it is expressed as the trajectory of the representative point of the vehicle 1.

[0011] As shown in Fig. 1, a vehicle 1 is equipped with a camera 2 (see Fig. 2, etc.) that monitors the periphery of the vehicle, and is configured to be able to perform automatic parking in a parking lot with one or more parking spaces by a parking assistance device 100 shown in Fig. 3. A parking space is an area sandwiched between two approximately parallel parking frame lines. Each parking frame line is arranged at an interval greater than the width of the vehicle 1.

[0012] In the example shown in FIG. 1, a parking lot is shown in which three parking spaces S1, S2, and S3 are arranged in order from the negative side in the X direction. The parking space S1 is a parking space sandwiched between the parking space lines L1 and L2, the parking space S2 is a parking space sandwiched between the parking space lines L2 and L3, and the parking space S3 is a parking space sandwiched between the parking space lines L3 and L4. The left side is the negative side in the X direction, and the right side is the positive side in the X direction, with a line parallel to the parking space lines L2 and L3 and equidistant from both the parking space lines L2 and L3 as the boundary. The boundary line between the positive and negative sides in the Y direction is the short side of the parking space that is closer to the vehicle 1.

[0013] Here, an example of the case where the vehicle 1 automatically parks in the parking frame S2 will be described. First, the vehicle 1 goes straight in the direction from the - side to the + side in the X direction in the portion on the + side in the Y direction of the three parking frames S1, S2, and S3. The vehicle 1 in the straight-moving state turns at a position P1 in front of the parking frame line L2 located on the - side in the X direction among the parking frame lines L2 and L3 of the parking frame S2, turns while moving forward to the + side in the Y direction, and stops at a position P2. The position P2 is a position on the + side in the X direction of the parking frame line L3 and on the + side in the Y direction, and is a position where the vehicle 1 can park between the parking frame lines L2 and L3 by turning while retreating from the position P2. The vehicle 1 travels to the position P2 by manual operation of the occupant.

[0014] Then, when the occupant performs a parking instruction operation such as putting the gear in R (reverse), the vehicle 1 performs automatic parking. The vehicle 1 turns while backing up from position P2, enters the parking space S2, and stops at position P3 in the parking space S2 while driving backward. During automatic parking, the hazard lamps may be turned on. In addition, since it is possible to determine which parking space the vehicle will be parked in based on the steering position and the steering direction, the occupant does not need to specify the parking position during automatic parking. In the following explanation, it is assumed that the vehicle 1 performs the above-mentioned behavior during automatic parking.

[0015] As shown in Fig. 2, cameras 2 are provided at four locations on the front, rear, left and right sides of the body of vehicle 1. Each camera 2 is equipped with a fisheye lens and has a horizontal viewing range of 180 degrees or more (see dashed lines). Each camera 2 is mounted at a depression angle in order to capture the road surface, so when the range in which the road surface is captured is converted into a horizontal viewing range, a range of about 240 degrees of the road surface is captured by each camera 2. For example, the front and rear wheels and the side of the vehicle body are captured in an image captured by side cameras 2A provided on the left and right of the vehicle body.

[0016] As shown in FIG. 3, the vehicle 1 includes an operation device 10, an HMI (human machine interface) device 20, a vehicle control device 30, and a parking assistance device 100 in addition to the four cameras 2. The operation device 10 is for a driver (passenger) to manually operate, and includes a physical switch on a panel at the driver's seat, a software switch displayed on a touch panel, and other devices for driving operations such as a steering wheel, pedals, and gears. The HMI device 20 is used as an HMI for an occupant to input operations to the parking assistance device 100, such as a touch panel attached to a navigation device 40 provided in the vehicle 1. The touch panel may be included in the operation device 10, and various switches grounded to the driver's seat may also be included in the operation device 10, so the operation device 10 and the HMI device 20 may be overlapping.

[0017] The vehicle control device 30 includes a central processing unit (CPU), a read only memory (ROM), a random access memory (RAM), and an input / output circuit (not shown). When an occupant operates the operation device 10, the vehicle control device 30 accepts the operation. In a normal driving mode, the vehicle control device 30 drives a motor (not shown) in response to operation information to control the steering angle and vehicle speed, and simultaneously outputs the operation information and vehicle information such as the steering angle and vehicle speed to a local area network (LAN: in-vehicle LAN, etc.). In a parking assistance mode, the vehicle control device 30 receives commands for speed and steering angle from the parking assistance device 100 via the LAN, and controls the speed and steering angle in response to the commands.

[0018] The parking assistance device 100 obtains operation information of the operation device 10 via the LAN because the vehicle control device 30 monitors the operation of the operation device 10. The occupant can operate the parking assistance device 100 via the HMI device 20. The parking assistance device 100 may receive the position information of the vehicle 1 output by the navigation device 40 via the LAN, or may obtain the position information directly from the navigation device 40. The camera 2 always outputs the captured image to the parking assistance device 100, and even when parking assistance is not performed, the parking assistance device 100 generates and outputs a display image showing the surroundings of the vehicle from the captured image. In addition, the above-mentioned touch panel also functions as a display that outputs the display image generated by the parking assistance device 100, so the parking assistance device 100 and the HMI device 20 are directly connected. The parking assistance device 100 may directly receive information on the operation performed on the touch panel, or may receive the information via the LAN. In addition, the parking assistance device 100 can receive the position information of the vehicle 1 output by the navigation device 40 via the LAN.

[0019] The functions of the parking assistance device 100 may be implemented in the hardware shown in FIG. 4. The parking assistance device 100 includes a CPU 101, a ROM 102, a RAM 103, an I / O (input / output interface) 104, and an IMP (Image Processor) 105. The parking assistance device 100 may be a computer in which each element is connected by a bus. Furthermore, a plurality of elements may be accommodated in one chip, or one element may be composed of a plurality of chips. The bus may not be a single one, but may be a combination of a plurality of types of buses. For example, the CPU 101, the ROM 102, the RAM 103, and the IMP 105 accommodated in one chip may be connected by a parallel bus, and the I / O 104 composed of a plurality of chips may be connected to the chip accommodating the CPU 101, etc., by a serial bus.

[0020] The CPU 101 controls the entire parking assistance device 100. The ROM 102 is an electrically rewritable memory that stores the programs executed by the CPU 101 and also functions as a non-volatile data storage area, and retains data and the like even when the parking assistance device 100 is turned off. The RAM 103 is used for temporary storage as a working area for the CPU 101. For example, data that only needs to be temporarily stored, such as the latest surrounding image, is stored in the RAM 103. The RAM 103 has a capacity that can store multiple surrounding images. The IMP 105 is a processor that is specialized for image processing and has improved processing performance, and executes the processing of the image acquisition unit 130, the parking frame detection unit 140, and the display image output unit 180 shown in FIG. 5.

[0021] The above-mentioned camera 2 constantly outputs the captured camera image to the parking assistance device 100. Even when parking assistance is not being performed, the parking assistance device 100 generates and outputs a display image showing the periphery of the vehicle 1 from the camera image. As shown in FIG. 5, the parking assistance device 100 has an operation reception unit 110, a state management unit 120, an image acquisition unit 130, a parking space detection unit 140, a route calculation unit 150, a travel control unit 160, a storage unit 170, and a display image output unit 180.

[0022] The operation receiving unit 110 receives operation information indicating operations by a driver, including a driving operation.

[0023] The state management unit 120 manages the parking assistance state according to the operation information and the state of the vehicle. There are a plurality of parking assistance states, for example, state 0 to state 8.

[0024] State 0 is the initial state, and is the state when the parking assistance function is not activated. State 1 is the pause state, and is the state when the vehicle speed information and position information are acquired, and when the vehicle speed drops below a threshold, it is determined whether the vehicle position is on a road. The road is a road on a map, for example, a public road that is not a parking lot with parking spaces.

[0025] State 2 is a monitoring state, in which white line detection is performed at a low frequency. State 3 is a preparation state, in which captured images are linked to time information and recorded. State 4 is a detection state, in which parking space detection is performed speculatively. State 5 is a calculation state, in which, after turning 30 degrees or more and stopping, parking path calculation is performed speculatively and the occupant is asked whether he or she intends to park.

[0026] State 6 is a query state, where the parking route is displayed and the vehicle waits for a parking instruction operation. State 7 is a self-driving state, where the vehicle performs automatic parking according to the calculated route. State 8 is an end state, where the vehicle is notified that automatic parking is complete.

[0027] The image acquisition unit 130 acquires a camera image and generates a surrounding image showing the surroundings of the vehicle 1. The surrounding image may be an overhead image in which the camera image is projected onto the road surface, may be an image in another format, or may be the camera image as it is. In this embodiment, the surrounding image is assumed to be an overhead image.

[0028] The parking space detection unit 140 sets a parking space detection area in the peripheral image for detecting a parking space, and extracts a white line from the peripheral image in the parking space detection area. The parking space detection area may be the entire peripheral image, or may be an area in the peripheral image where a parking space is expected. By limiting the parking space detection area to an area where a parking space is expected, the processing time required for detection can be shortened. In addition, by limiting the size of the parking space detection area to a size where one parking space can be accommodated but two parking spaces cannot be accommodated, the processing when two parking spaces are detected simultaneously can be omitted. In general, detection is performed for the purpose of obtaining information about a specific object, and if a specific object is present as a result of the detection, it is determined that "detection has been performed" and information about the detected object is output. The result of the detection is divided into a case where it is detected and a case where it is not detected, and if it is not detected, the purpose of the detection is not achieved. If it is detected, the purpose of the detection is achieved, so the detection may be rephrased as a successful detection.

[0029] The parking space detection unit 140 further generates pairs of the extracted white lines, and searches for pairs of white lines that meet the conditions of a parking space. For example, when the parking space detection unit 140 detects a pair of white lines from the peripheral image, if the following three conditions are met: the two white lines are parallel on the road surface, the two white lines are longer than the vehicle length, and the distance between the two white lines is greater than the vehicle width, the parking space detection unit 140 determines that the area between the two white lines is an area where the vehicle 1 can be parked (i.e., a parking space). When a parking space is detected, the white lines that constitute the parking space can be said to be parking space lines. The parking space detection unit 140 outputs the end points of the two parking space lines, that is, the coordinates of the four corners of the parking space. These coordinates are relative coordinates based on the vehicle itself.

[0030] The route calculation unit 150 sets the target parking position so that the center of the vehicle body is located in the center of the coordinates of the four corners of the parking space output by the parking space detection unit 140, and sets the target parking angle (the direction of the vehicle body when parked at the target parking position) so that the vehicle body is parallel to the long side of the parking space, and calculates the parking route to the target parking position. A proposed publicly known method may be applied to the parking route calculation.

[0031] The driving control unit 160 controls the steering angle and the vehicle speed so that the vehicle 1 drives along the calculated parking path. More specifically, the driving control unit 160 outputs the steering angle and the vehicle speed instruction values ​​according to the target values ​​of the steering angle and the vehicle speed to the vehicle control device 30. The driving control unit 160 calculates the actual values ​​of the steering angle and the vehicle speed based on the data output by the vehicle control device 30. For example, the actual value of the vehicle speed is calculated from the data of the rotation speed of the wheels and the data of the outer circumference of the wheels. The actual value of the steering angle may be data obtained from the steering device, or may be calculated from the inner wheel difference calculated from the difference in the rotation speed of the wheels. When there is a difference between the target value and the actual value, the driving control unit 160 corrects the instruction values ​​of the steering angle and the vehicle speed so that the actual values ​​of the steering angle and the vehicle speed become the same as the target values ​​of the steering angle and the vehicle speed.

[0032] The driving control unit 160 tracks the position and orientation of the vehicle 1 while it is traveling, based on the calculated steering angle and actual measured values ​​of the vehicle speed. That is, the driving control unit 160 calculates the traveling trajectory of the vehicle 1 during automatic traveling or manual traveling. The traveling trajectory may be calculated in the form of movement information in which the travel distance, the travel direction, and the angle of the vehicle body are added to the time for each unit time. The form of the movement information is not limited to this, and may be, for example, a form using coordinate information.

[0033] During automatic driving, the driving control unit 160 compares the parking route calculated by the route calculation unit 150 with the driving trajectory represented by the movement information, and corrects the steering angle so that the driving trajectory follows the parking route. For example, when the vehicle 1 reverses while turning with a steering angle, if the driving trajectory passes outside the parking route as viewed from the center point of the turn, the driving control unit 160 increases the steering angle to reduce the turning radius. This causes the driving trajectory of the vehicle 1 to be pulled back toward the parking route.

[0034] The storage unit 170 stores the operation information acquired by the operation reception unit 110, the movement information calculated by the driving control unit 160, the surrounding image generated by the image acquisition unit 130, the parking frame line and the parking frame position information detected by the parking frame detection unit 140, etc. in a history information area in a chronological order, linked to time. Since the vehicle 1 moves in response to the operation of the occupant, it is not necessary to store either the movement information or the operation information in the storage unit 170. However, in this embodiment, both the movement information and the operation information are stored in the storage unit 170.

[0035] For example, the storage unit 170 may add information every unit time and fix the size of the storage area (memory block size) allocated for each addition. In this case, the history information area is divided into memory blocks and the start address of the information increases monotonically every unit time, so that the address storing the information is linked to the time. Alternatively, the storage unit 170 may not fix the memory size, but may separately provide a start address table that links the start address to the time, and update the start address table every time information is added.

[0036] By specifying the time using any method, it is possible to access the latest information at that time. Also, by accumulating the movement information up to that time, it is possible to specify the relative positional relationship of the position at that time to the current position. Conversely, it is also possible to add the positional information in chronological order as history information, and specify the relative positional relationship to the current position from the difference between the positional information at that time and the current position. In other words, it is possible to calculate the positional information from the history information.

[0037] Also, for large-sized data such as peripheral images, only the starting address needs to be stored in the history information area.

[0038] Since it is sufficient for history information to be able to be referenced going back a predetermined time, old history information that is no longer necessary may be erased from storage unit 170. For example, a history information area may be set in storage unit 170 as a fixed area sandwiched between a first address and a last address, and when the address for storing information reaches the last address, it may be made a loop memory that continues storing information in the first address. In this way, past history information going back one loop is overwritten and erased, eliminating the need for erasure processing.

[0039] Storing in the history information area does not need to be performed all the time, but may be performed in states 2 to 7 above.

[0040] Furthermore, the storage unit 170 stores the parking route calculated by the route calculation unit 150, and enables the driving control unit 160 to refer to the parking route.

[0041] In addition, since the parking space detection takes time to process, the parking space information, which is the detection result, may be obtained at a time significantly delayed from the time of the camera image capture. Even in this case, the parking space information is stored in the storage unit 170 in association with the capture time. For example, the storage unit 170 adds information every unit time, and the memory block is assigned fixedly regardless of the amount of data written in one add-on by fixing the capacity (memory block size) of the memory area assigned to one add-on. When the storage unit 170 adds information, the head address of the surrounding image is obtained, but the result of detecting the parking space in the surrounding image (parking space information) is not obtained at that time. In this case, the parking assistance device 100 writes only the obtained information into the memory block without waiting for the parking space information, and when the parking space information is obtained, writes the parking space information into the memory block in which the head address of the surrounding image that was the object of the detection is written.

[0042] For example, if the vehicle 1 turns in state 3 and speculatively executes parking space detection when it proceeds to state 4, whether parking space information has been obtained or not when it proceeds to state 5 changes depending on the relationship between the time from when the vehicle 1 turns to when it stops and the time required for parking space detection. For example, if the parking space information is not written in the corresponding memory block when it proceeds to state 5, the process of starting the route calculation may be a waiting loop that polls the address where the parking space information is written, or a processor (e.g., IMP) that performs the parking space detection may be set to generate an interrupt when the parking space detection is completed. In this way, the route calculation can be started immediately when the parking space detection is completed. In this way, the parking space information and the like are stored in the storage unit 170 in association with the shooting time, so that the process of automatic parking can be efficiently controlled.

[0043] The display image output unit 180 generates a display image based on a camera image or a surrounding image. The display image may be an overhead image that overlooks the surroundings of the vehicle 1 from above. In other words, the display image output unit 180 may output the overhead image as the display image.

[0044] The display image output unit 180 displays a message or a figure by superimposing it on the display image in response to a request from the state management unit 120. For example, when starting automatic parking, the display image output unit 180 may display a semi-transparent rectangle indicating the position of the parking space detected by the parking space detection unit 140 by superimposing it on the overhead image, and may display the route from the vehicle to the target parking position calculated by the route calculation unit 150 with a dotted line.

[0045] The message may not only be displayed on the image, but may also be read aloud. Alternatively, the message may be output only as a voice. Specifically, the state management unit 120 issues a command to specify one of the preset texts and output it, and in response, the display image output unit 180 generates an image of a character string corresponding to the specified text, superimposes it on the display image, and outputs it to the HMI device 20. At the same time, the display image output unit 180 outputs the voice data that was saved together with the text to the HMI device 20 together with the display image. This allows the occupant to receive the message even if he or she is not looking at the HMI device 20.

[0046] Next, the states managed by the state management unit 120 will be described in detail. As described above, the state management unit 120 accepts the operation of the occupant and manages the parking assistance state. The operation of the occupant is not limited to the operation of the parking assistance device 100, but includes the operation for driving. For example, when the occupant starts the vehicle 1 by turning on the IG (Ignition)-ON, the parking assistance device 100 also starts up during the start-up sequence of the vehicle 1, and the state progresses from state 0 (initial state) to state 1 (pause state).

[0047] In state 1 (resting state), only the state management unit 120 and the driving control unit 160 operate. The state management unit 120 acquires vehicle speed information via the driving control unit 160, and when the vehicle speed drops below a threshold (e.g., 10 km / h), inquires of the navigation device 40 whether the vehicle 1 is located on a road. If the vehicle 1 is off the road (e.g., on a store premises), the state management unit 120 advances the state to 2. State 1 may be called a road driving scene. Since a state corresponds to a scene, the state management unit 120 may be called an example of a scene determination unit described later. In addition, the state management unit 120 may be called an example of a function restriction unit because it partially or fully restricts functions such as a parking space detection unit depending on the state (scene).

[0048] In state 2 (monitoring state), part of the parking space detection unit 140 also operates. In other words, in state 1 (road driving scene), execution of all functions of the parking space detection unit 140 is restricted. The parking space detection unit 140 periodically (for example, at one-second intervals) executes white line detection. The white line detection is executed by releasing some of the functions of the parking space detection unit 140, and does not execute other functions such as evaluation of whether a white line is a parking space line. The parking space detection unit 140 detects a white line that is at an angle within a predetermined range (60 degrees to 120 degrees) with respect to the axis of symmetry of the vehicle 1 (a line passing through the center of the front end of the vehicle and the center of the rear end of the vehicle 1) from an image captured around the vehicle 1. Since the white line detection is performed for the purpose of determining whether the vehicle 1 is in a parking lot, the interval or length of the white line is not evaluated. Therefore, a limited range in which only the part in front of the parking space line is captured in the surrounding image may be set as the detection range. If the parking space detection unit 140 detects multiple white lines that meet the conditions, it determines that the vehicle 1 is in a parking lot, and the state management unit 120 advances the state.

[0049] In the example shown in FIG. 1, state 2 is a state in which the vehicle 1 is moving straight through the portion on the + side in the Y direction from the parking frames S1, S2, and S3 and before the white line is detected.

[0050] In state 3 (preparation state), the peripheral images are linked to time information and stored in the storage unit 170. Of the images used for white line detection, only images in which white lines (white lines approximately perpendicular to the axis of symmetry of the vehicle 1) are detected may be stored as peripheral images, or images captured periodically (for example, every second) may be stored as peripheral images without being selected.

[0051] The state management unit 120 stores in the memory unit 170 the time point at which a steering force equal to or greater than a threshold value (for example, 1 kg) is applied to the steering wheel from a straight-ahead state with a steering angle of approximately 0 as the steering time point, and advances the state.

[0052] In the example shown in FIG. 1, state 3 is the state in which the vehicle 1 is moving straight ahead in the +Y direction from the parking spaces S1, S2, and S3, from when the white line is detected until when the vehicle 1 turns at position P1.

[0053] In state 4 (detection state), the parking space detection unit 140 detects the parking space from the image captured just before the steering wheel is turned. Alternatively, the parking space detection unit 140 may detect the parking space from the image captured at the steering wheel is turned.

[0054] At this point, it is not yet determined whether or not to perform automatic parking, so this parking space detection can be said to be a speculative parking space detection. If there is a parked vehicle in the parking space, a part of the parking space line is blocked by the parked vehicle, so the length of the detected white line does not meet the condition, and the parking space is not detected. Therefore, when a parking space is detected, it can be said that an empty parking space is detected.

[0055] The state management unit 120 advances the state when the vehicle body turns 30 degrees to 60 degrees from the steering point and the vehicle 1 stops.

[0056] In the example shown in FIG. 1, state 4 is the state in which the vehicle 1 is steered at position P1 and stops at position P2.

[0057] In state 5 (calculation state), the route calculation unit 150 integrates movement information from the time when the image in which the parking space is detected is captured to the time when the vehicle 1 stops, and calculates the vehicle position (coordinate offset) at the time of capturing the image based on the current vehicle position. After the parking space detection is completed, the route calculation unit 150 adds the coordinate offset to the position information of the parking space to calculate the position information of the parking space based on the current vehicle position. The route calculation unit 150 calculates a parking route for parking in the parking space from the current vehicle position.

[0058] At this point, it is not yet decided whether to perform automatic parking, so it may be said that the route calculation is speculative. Once the parking route is calculated, the display image output unit 180 outputs an image showing the parking space and the parking route, and outputs a message to inquire the driver about his / her intention to park. After outputting the image and message, the state management unit 120 advances the state.

[0059] In the example shown in FIG. 1, state 5 is the state from when the vehicle 1 stops at position P2 and when the parking assistance device 100 calculates the parking path until it outputs the above message.

[0060] In state 6 (inquiry state), the operation reception unit 110 receives a parking instruction operation. If a parking instruction operation is performed, the state management unit 120 advances the state. In the example shown in FIG. 1, state 6 is the state from when the message is output until the driver performs a parking instruction operation.

[0061] In state 7 (self-driving state), the driving control unit 160 performs automatic parking according to the calculated route. When automatic parking is completed, the state management unit 120 advances the state. In the example shown in FIG. 1, state 7 is a state in which the vehicle 1 performs automatic parking from position P2 to position P3.

[0062] In State 8 (end state), the parking assistance device 100 notifies the end of automatic parking. After the notification is completed, the state management unit 120 returns the state to State 2. In the example shown in FIG. 1, State 8 is the state from when the vehicle 1 stops at position P3 until the above notification is made.

[0063] When the vehicle 1 travels to a vacant parking space where the vehicle 1 is to be parked, the occupant turns the steering wheel when he or she visually confirms that the parking space is available. Therefore, the parking space detection unit 140 may set the parking space detection area according to the position of the occupant when the vehicle 1 turns the steering wheel. The position of the occupant may be specified as the position of the occupant's head.

[0064] The occupant must also visually confirm the direction of travel of the vehicle 1, and therefore visually confirms the parking space located diagonally forward. However, there is a time lag between seeing the vacant parking space and turning the steering wheel, and the vehicle 1 may be moving to the side of the vacant parking space at the time of turning the steering wheel. Therefore, it is preferable for the parking space detection unit 140 to set the parking space detection area diagonally forward from the side of the occupant.

[0065] 6 shows parking frames S1, S2, and S3 similar to those shown in FIG. 1, and shows an example in which a vehicle 1 traveling from the + side to the - side in the X direction is trying to park in the parking frame S2 in the - side of the parking frames S1, S2, and S3 in the Y direction. In this example, when the vehicle 1 turns at a position P4 in front of the parking frame line L3 in order to park in the parking frame S2, the area from the side of the occupant (the + side in the Y direction) to the diagonally forward (the - side in the X direction and the + side in the Y direction) is set as the parking frame detection area D1.

[0066] Since the occupant turns the steering wheel for reverse parking, the parking space detection area is located on the opposite side of the vehicle 1 across the line along which the vehicle 1 has traveled. Therefore, the parking space detection area is preferably set on the opposite side of the direction in which the vehicle 1 has turned across the line along which the vehicle 1 has traveled, and includes the side of the occupant or the diagonal front of the occupant based on the position of the occupant when the vehicle 1 turns the steering wheel. The line along which the vehicle 1 has traveled can be rephrased as a line extending a straight line portion of the route the vehicle 1 has traveled. When turning the steering wheel, as shown in FIG. 6, the camera 2A (side camera) provided on the side of the vehicle 1 is often located on the side of the vacant parking space, so the parking space detection unit 140 may set the side of the side camera as the parking detection area.

[0067] In addition, when an occupant is positioned in front of a vacant parking space, the vehicle 1 is steered and turns approximately 45 degrees before stopping, and in many cases, the camera 2 (rear camera) installed at the rear of the vehicle 1 captures the entire vacant parking space, as shown in Figure 7.

[0068] FIG. 7, like FIG. 6, shows an example in which the vehicle 1 is trying to park in the parking space S2. After the vehicle 1 turns to park in the parking space S2, it moves forward and turns to stop. When the vehicle 1 stops, the camera 2 installed at the rear of the vehicle 1 can detect the four corners of the parking space S2.

[0069] Therefore, for example, the parking space detection unit 140 may not perform speculative parking space detection in state 4, and may first perform parking space detection from the image of the rear camera in state 5. If no parking space is detected, the parking space detection unit 140 may use the image at the time of turning the steering wheel to detect the parking space.

[0070] In this way, the effect of concealing the processing time is smaller than when parking space detection is started speculatively in state 4. However, since parking space detection is started speculatively as soon as vehicle 1 is stopped without waiting for an instruction for automatic parking, the time required for parking is shorter than with the conventional method.

[0071] For example, if the vehicle 1 turns after an occupant passes in front of a vacant parking space, or if the vehicle 1 turns with a small steering angle and a large turning radius, the entire vehicle body may pass in the direction of the entrance of the vacant parking space. In this case, if there is a parked vehicle in a parking space adjacent to the vacant parking space, the vacant parking space (long side) may be blocked by the parked vehicle, and the rear camera may not capture the entire vacant parking space (for example, see FIG. 8). If speculative parking space detection is not performed in State 4, first, parking space detection is performed in the image of the rear camera in State 5, and after the parking space detection fails, parking space detection is performed in the image at the time of turning.

[0072] FIG. 8 shows an example in which the vehicle 1 is trying to park in the parking space S2, as in FIG. 6. When the vehicle 1 turns to park in the parking space S2 and then turns while moving forward, the parking vehicle 1A in the parking space S1 on the negative side of the X direction from the parking space S2 blocks the parking space line L2, so the camera 2 installed at the rear of the vehicle 1 cannot capture the whole parking space S2.

[0073] In this case, the parking space detection is performed twice, so the time required for parking will be longer than with the conventional method. However, if the driver drives properly, the situation will not occur, so the probability of the situation occurring is low. Therefore, the expected time required for parking will be smaller than with the conventional method.

[0074] Also, for example, speculative parking space detection may be executed in state 3 instead of starting in state 4 after steering. In this way, it is expected that parking space detection will be completed when the vehicle 1 turns and stops and enters state 5. In this case, since only route calculation needs to be performed in state 5, it is possible to present the parking position and parking route at an earlier timing than when parking space detection is performed in state 4.

[0075] At that time, the parking space detection may be performed using an image from a camera 2 (front camera) provided at the front of the vehicle 1. The front camera approaches the entrance of the parking space before the occupant, so the parking space can be detected at an earlier timing.

[0076] For example, FIG. 9 shows an example in which the front camera 2 of the vehicle 1 moving straight from the + side of the X direction to the - side of the X direction from the parking space S3 captures the entire parking space S1 located at the most - side (forward) in the X direction among the parking spaces S1, S2, and S3.

[0077] However, since the parking frame lines are drawn periodically, it is possible to deal with this issue by predicting the appropriate position for parking frame detection. Since the front camera is positioned lower than the side cameras, it is often possible to capture the parking frame lines without being blocked by parked vehicles, even if the parking frame lines are drawn in front of the line extended from the long side.

[0078] When speculative parking space detection is performed in state 3, it may happen that a plurality of parking spaces are detected at the time of state 5, such as parking space S1 and parking space S2 in FIG. 10. In such a case, the parking space information (for example, parking space S1) whose photographing time is immediately before the position P5 at the time of turning is selected from the history information (not the latest parking space detection result obtained at the time of turning). Since the purpose is to reduce the time required for parking, it is more suitable to the purpose not to allow the occupant to select.

[0079] If the time required for the vehicle 1 to move forward by one parking space is shorter than the time required for parking space detection, the parking space detection in the area of ​​the parking space S1 is not completed at the time of generating the peripheral image of the parking space S1. In such a case, the parking space detection unit 140 may store the peripheral image of the parking space S1 in the storage unit 170, skip the parking space detection, and perform the parking space detection when the information of the parking space S1 becomes necessary. If the parking space detection in the area of ​​the parking space S2 is terminated and the parking space detection in the area of ​​the parking space S1 is started, the parking space detection of the parking space S2 that was terminated will be re-executed when the information of the parking space S2 becomes necessary. Therefore, the expected value of the time required for parking is larger when the parking space detection is terminated. Alternatively, the parking assistance device 100 may request the occupant to decelerate the vehicle 1 so that both the parking space S1 and the parking space S2 can be detected so that the parking space detection is not required to be skipped.

[0080] In addition, the standard interval between parking frame lines (long sides of parking frame lines) is 2.5 m. When the vehicle 1 travels in front of a parking frame at 2.5 m per second, the vehicle 1 moves forward by one frame per second. 2.5 m per second is 9 km per hour, which is a little faster than the 8 km per hour posted in many parking lots. When the vehicle 1 travels while searching for a vacant parking frame, it travels at a speed slower than 8 km per hour. Therefore, if the vehicle 1 travels while storing surrounding images in the storage unit 170 at intervals of one frame per second, there is no risk of missing a vacant parking frame.

[0081] For example, when a parking space is detected from an image captured by the front camera, it is preferable to capture the image with the front camera positioned slightly in front of the extended parking space line, and therefore, rather than making the time interval of the image capture constant, it is preferable to make the interval of the image capture constant (for example, 2.5 m interval). By repeating the parking space line detection from state 2 onwards, the distance between the parking space lines can be known, so that the position of the next parking space line can be predicted, and if the image is captured when the vehicle 1 advances to that position, a front camera image captured at a position suitable for parking space detection can be obtained. When traveling at a speed of 8 km per hour, the time interval for storing the front camera image is 1 second or more. In addition, since the speed is not necessarily constant and the time interval for the image capture is irregular, it is preferable to store the image capture time as well.

[0082] For example, when the movement information is stored in a memory block at one-second intervals, it is advisable to store the start address of the surrounding image in the latest memory block, and also store fractions of less than one second of the shooting time. This allows the distance traveled by the vehicle 1 between the time the movement information was stored and the time the image was shot to be calculated by an interpolation method. This allows the distance from the surrounding image to the detected parking space to be correctly calculated later.

[0083] Also, the storage interval of the movement information from the time of turning can be shortened (for example, 0.1 second intervals). The more detailed the movement information is stored, the more accurately the movement of the vehicle 1 from the turning point to the stopping position can be estimated. The storage interval can be rough because the movement information before turning is not directly related to the estimation accuracy of the parking space position. If the time when automatic parking starts and the time when the image is taken are different, the amount of movement between them needs to be compensated for, but this can be compensated for by the above method.

[0084] Next, an operation example of the parking assistance device 100 according to the first embodiment will be described. Figures 11, 12, and 13 are flowcharts showing an operation example of parking assistance control in the parking assistance device 100 according to the first embodiment. The following flowchart starts when the power of the vehicle 1 is turned on.

[0085] 11, the parking assistance device 100 (state management unit 120) sets the parking assistance state to State 0 (step S101). In step S101, the state management unit 120 initializes the internal state of the parking assistance device 100.

[0086] Next, the state management unit 120 sets the parking assistance state to State 1 (step S102). In step S102, the state management unit 120 acquires vehicle speed information from the driving control unit 160.

[0087] Next, the state management unit 120 determines whether the vehicle speed is less than a speed threshold (for example, 10 km / h) (step S103). If the vehicle speed is equal to or greater than the speed threshold (step S103, YES), the process returns to step S102.

[0088] On the other hand, if the vehicle speed is less than the speed threshold (step S103, NO), the parking assistance device 100 (state management unit 120) determines whether the vehicle 1 is on a road or not (step S104). The determination as to whether the vehicle 1 is on a road or not is made based on information from the navigation device 40. If the result of the determination is that the vehicle 1 is on a road (step S104, YES), the process returns to step S102.

[0089] On the other hand, if the vehicle 1 is not on the road (step S104, NO), the state management unit 120 sets the parking assistance state to State 2 (step S105). Note that, even if information from the navigation device 40 cannot be obtained, the state may proceed to State 2. This may be said to skip Step S104. In State 2 (step S105), the parking assistance device 100 (parking space detection unit 140) performs white line detection at a low frequency (at one-second intervals).

[0090] In State 2, the parking assistance device 100 (state management unit 120) determines whether or not the conditions for determining a parking scene are met (step S106). The conditions for determining a parking scene are that there are multiple white lines that are approximately perpendicular to the axis of symmetry of the vehicle 1, and that these white lines are parallel to each other. If the determination result shows that the conditions for determining a parking scene are not met (step S106, NO), the process returns to step S105.

[0091] On the other hand, if the parking scene determination condition is met (step S106, YES), the state management unit 120 sets the parking assistance state to State 3 (step S107). In step S107, the parking assistance device 100 (storage unit 170) stores the peripheral image in association with time information.

[0092] Next, the state management unit 120 acquires steering angle data from the driving control unit 160 and determines whether the absolute value of the steering angle is greater than an angle threshold value (e.g., 5 degrees) (step S108). If the result of the determination is that the absolute value of the steering angle is greater than the angle threshold value (step S108, YES), the process of step S108 is repeated.

[0093] On the other hand, if the absolute value of the steering angle is less than the angle threshold value (step S108, NO), as shown in Fig. 12, the state management unit 120 acquires data on the steering force applied to the steering wheel from the driving control unit 160, and determines whether the absolute value of the steering force is greater than a threshold value (e.g., 1 kg) (step S109). That is, when turning, that is, when the steering angle is large, it is not determined that steering has occurred, but it is determined that steering has occurred when the vehicle is traveling straight, that is, when the steering angle becomes small and then a steering force is applied. The conditions for determining straight traveling may also include traveling forward a predetermined distance (e.g., 1 m) with a small steering angle.

[0094] If the result of the determination is that the absolute value of the steering force is less than the threshold value (step S109, NO), the process returns to step S108 (see also FIG. 11). On the other hand, if the absolute value of the steering force is greater than the threshold value (step S109, YES), the state management unit 120 sets the parking assistance state to State 4 (step S110).

[0095] In step S110, the parking assistance device 100 (storage unit 170) stores the steering time, and the parking space detection unit 140 starts parking space detection using the image captured just before the steering time. The parking assistance device 100 (route calculation unit 150) also determines the straight-ahead direction based on history information before the steering time. Specifically, at the time of steering, the surrounding image generated from the image captured just before the steering remains in the memory, so the storage unit 170 prohibits overwriting of the area containing the surrounding image, and writes the head address and the shooting time in the memory block. Then, the state management unit 120 passes the head address to the parking space detection unit 140 and instructs it to detect a parking space. The state management unit 120 also passes the start address to the route calculation unit 150, and the route calculation unit 150 reads the traveling direction (yaw angle) of the vehicle 1 from 3 seconds before the steering time to the steering time from the history information (memory block up to 3 seconds ago) and determines the average value as the straight heading direction. Note that the above processing and numerical values ​​are merely examples.

[0096] Next, the parking assistance device 100 (route calculation unit 150) calculates a turning angle based on the straight heading direction (step S111), and the state management unit 120 determines whether the turning angle is greater than a first angle (e.g., 60 degrees) (step S112). The turning angle is calculated, for example, by subtracting the straight heading direction from the yaw angle and finding the absolute value.

[0097] As a result of the determination, if the turning angle is greater than the first angle (step S112, YES), the parking assistance device 100 stops the parking space detection and cancels the turning time (step S113). After that, the process returns to step S108 (see also FIG. 11). In step S113, if the turning angle is greater than the first angle, the parking assistance device 100 (state management unit 120) determines that the vehicle 1 has made a right or left turn, and cancels the process associated with the turning.

[0098] On the other hand, if the turning angle is equal to or smaller than the first angle (step S112, NO), the parking assistance device 100 judges whether the vehicle speed is 0 or not (step S114). If the result of the judgment is that the vehicle speed is not 0 (step S114, NO), the process returns to step S111. That is, the turning angle monitoring process is in a loop state until the vehicle 1 stops.

[0099] On the other hand, if the vehicle speed is 0 (step S114, YES), the parking assistance device 100 (state management unit 120) determines whether the turning angle is greater than a second angle (e.g., 30 degrees) (step S115). If the result of the determination is that the turning angle is equal to or less than the second angle (step S115, NO), the process returns to step S111. In other words, if the turning angle is small, the turning angle monitoring process is in a loop state.

[0100] On the other hand, if the turning angle is greater than the second angle (step S115, YES), the state management unit 120 sets the parking assistance state to State 5 (step S116). This can also be said to proceed to State 5 (step S116) when the vehicle 1 turns 45 degrees±15 degrees from the straight-ahead state and stops. In step S116, the parking assistance device 100 specifies the amount of movement of the vehicle 1 since the image capture time. For example, the route calculation unit 150 accumulates the amount of movement since the image capture time of the image that is the target of the parking space detection recorded in the history information, and calculates the current position (including relative coordinates and yaw angle) based on the position of the vehicle 1 at the image capture time.

[0101] In state 5, the parking assistance device 100 (state management unit 120) judges whether the parking space detection is completed (step S117). If the result of the judgment is that the parking space detection is not completed (step S117, NO), the process of step S117 is repeated. On the other hand, if the parking space detection is completed (step S117, YES), the parking assistance device 100 judges whether the parking space detection is successful (step S118).

[0102] If the result of the judgment is that the parking space detection is not successful (step S118, NO), the parking assistance device 100 (display image output unit 180) outputs a message that the parking space is not detected (step S119), and this control ends (see also FIG. 13). If the parking space detection is not successful, it may be rephrased as the parking space detection unit 140 does not detect the parking space, and if the parking space detection is successful, it may be rephrased as the parking space detection unit 140 detects the parking space.

[0103] On the other hand, if the parking space detection is successful (step S118, YES), as shown in FIG. 13, the parking assistance device 100 calculates the position of the parking space (step S120). In step S116, the current position is calculated based on the position of the vehicle 1 at the time of image capture, and the result of the parking space detection indicates the position of the parking space based on the position of the vehicle 1 at the time of image capture, so by adding up the two, the position of the parking space based on the current position can be calculated. This calculation may be performed by the route calculation unit 150. Next, the route calculation unit 150 executes parking route calculation based on the position of the parking space based on the current position (step S121).

[0104] Next, the parking assistance device 100 (state management unit 120) judges whether the parking route calculation is completed or not (step S122). If the result of the judgment is that the parking route calculation is not completed (step S122, NO), the process of step S122 is repeated. On the other hand, if the parking route calculation is completed (step S122, YES), the parking assistance device 100 judges whether the parking route calculation is successful or not (step S123).

[0105] If the result of the determination is that the parking path calculation has not been successful (step S123, NO), the parking assistance device 100 (display image output unit 180) outputs a message to the effect that the parking path cannot be set (step S124), and this control ends.

[0106] On the other hand, if the parking path calculation is successful (step S123, YES), the state management unit 120 sets the parking assistance state to State 6 (step S125). In step S125, the display image output unit 180 outputs a message asking the occupant whether or not to select automatic parking, and the state management unit 120 sets the timer to 0. The timer is for measuring the waiting time for the occupant's response to the above message, i.e., the inquiry about automatic parking.

[0107] In State 6, the state management unit 120 determines whether the timer is less than 3 seconds (step S126). If the timer is 3 seconds or more (step S126, NO), the parking assistance device 100 outputs a message to the effect that automatic parking is canceled (step S127), and this control ends. In other words, if the occupant does not respond within 3 seconds, it is assumed that the occupant did not select automatic parking.

[0108] On the other hand, if the timer is less than 3 seconds (step S126, YES), the parking assistance device 100 judges whether or not a parking instruction operation has been performed (step S128). The parking instruction operation includes at least one of the following, or a combination of two or more of: changing the gear position, releasing the steering wheel, releasing the brake, turning on the hazard lamps, and operating a button corresponding to a parking instruction. For example, if the gear is in R and the steering wheel and the brake are released, the parking assistance device 100 judges that a parking instruction operation has been performed.

[0109] As a result of the determination, if a parking instruction operation has not been performed (step S128, NO), the process returns to step S126. On the other hand, if a parking instruction operation has been performed (step S128, YES), the state management unit 120 sets the parking assistance state to state 7 (step S129). In step S129, the parking assistance device 100 (driving control unit 160) controls the vehicle 1 to start automatic driving. This can also be rephrased as, if the occupant performs a parking instruction operation within 3 seconds, it is assumed that the occupant has selected automatic parking, and automatic parking is started.

[0110] Next, the parking assistance device 100 judges whether the vehicle position is a parking position or not (step S130). If the result of the judgment is that the vehicle position is not a parking position (step S130, NO), the process of step S130 is repeated. This may be said to continue automatic driving until the vehicle 1 reaches the parking position.

[0111] On the other hand, if the vehicle position is a parking position (step S130, YES), the parking assistance device 100 outputs a message to the effect that automatic parking has ended (step S131). After that, the process returns to step S105.

[0112] In the above steps S104 to S124, if the vehicle speed becomes greater than the speed threshold, the process may immediately transition to step S102. In addition, in the above steps S105 to S124, if the position of the vehicle 1 is on a road, the process may immediately transition to step S102. In other words, even if the vehicle 1 is traveling at a low speed outside the road, a white line is detected, and storage of the movement information of the vehicle 1 is started, when the vehicle 1 appears on the road or the vehicle speed increases, the parking assistance device 100 may determine that the scene is not a parking scene and return to step S102 for determining whether or not the scene is a parking scene.

[0113] Furthermore, in the determination process from step S125 onwards, the occupant may give an instruction not to perform automatic parking. For example, when waiting for a parking instruction operation in steps 126 and 128, an instruction not to perform automatic parking may be received by the HMI device 20. If an instruction is given by the occupant, the instruction takes priority, so the determination may not be made based on the position or speed of the vehicle 1. Furthermore, in any step, if IG-OFF is set, this control ends.

[0114] Also, the message asking the occupant whether to select automatic parking may be output at a timing earlier than step S125. This may be rephrased as speculatively inquiring of the occupant without waiting for the completion of route calculation. For example, the above message may be output at the point in time when the vehicle 1 turns 45 degrees ±15 degrees and stops in step S116. In this way, route calculation is performed in parallel with the occupant performing the parking instruction operation, so that the overall time required for parking can be shortened.

[0115] Also, considering that there are multiple parking instruction operations and that each of the multiple parking instruction operations has a specific order, the step of determining the parking instruction operation may be divided into multiple steps. For example, when the gear position is in R after step S116, a message may be output asking the occupant whether or not to select automatic parking, and when the steering wheel and brake are subsequently released, acceptance of the parking instruction operation may be completed and automatic driving may be started.

[0116] In this way, when the timing of inquiring of the occupant is advanced, parking assistance device 100 may lengthen the waiting time for the inquiry in step S126 (the time until it is assumed that the occupant has not selected automatic parking) in consideration of the time required for the occupant to operate and the time for the calculation process performed in parallel. For example, when making an inquiry when the gear position is in R, parking assistance device 100 may increase the waiting time by 1 second from 3 seconds in step S126 to 4 seconds, and when making an inquiry at the time of step S116 (when the vehicle is stopped), the waiting time may be increased by 3 seconds from 3 seconds to 6 seconds.

[0117] According to the present embodiment configured as above, the state management unit 120 detects a parking instruction from an occupant based on the history of operation information or movement information of the vehicle 1 stored in the storage unit 170. The state management unit 120 corresponds to the "instruction detection unit" of the present disclosure.

[0118] Specifically, the state management unit 120 determines that a parking instruction has been detected when the history of the operation information or movement information of the vehicle 1 indicates that the vehicle 1 has stopped after steering and turning from a straight line and the occupant has performed a predetermined parking instruction operation. When the state management unit 120 detects a parking instruction, the driving control unit 160 outputs a command to the vehicle control device 30 to automatically park the vehicle 1 based on the parking space information. The driving control unit 160 corresponds to the "vehicle control unit" of the present disclosure.

[0119] As a result, when the vehicle 1 turns and stops, automatic parking can be started in response to a parking instruction. The route for automatic parking generally overlaps with the area that the vehicle 1 has passed, so there is no need to request safety confirmation again. In addition, since the location where the vehicle will be parked can be determined from the position and direction of the turn, there is no need to instruct the parking location.

[0120] As described above, in this embodiment, since no confirmation or instruction is required when starting automatic parking, automatic parking can be started in a short time.

[0121] In addition, in this embodiment, the vehicle 1 is triggered to turn from a straight-ahead state to perform the parking space detection speculatively, so that the parking space detection can be performed by utilizing the time from turning to stopping the vehicle. As a result, automatic parking can be started in a shorter time than the configuration in which the parking space detection is performed after stopping the vehicle. Therefore, in this embodiment, automatic parking can be performed more smoothly.

[0122] Furthermore, the state management unit 120 determines whether a scene is a parking scene in which the vehicle 1 is parked, and when the scene is a parking scene, the state of the parking assistance is set to state 3, the state of the parking assistance in a scene where it is determined whether it is a parking scene is set to state 2, and the state of the parking assistance in a scene where it is not determined whether it is a parking scene (a scene in which the vehicle is traveling on a road) is set to state 1. In other words, when the scene is not a parking scene, the state management unit 120 sets the state of the parking assistance to state 2, which restricts the functions of the parking assistance device 100 more than state 3 and after, and when it is not necessary to determine whether it is a parking scene, the state management unit 120 sets the state of the parking assistance to state 1, which further restricts the functions of the parking assistance device 100. Thus, the state management unit 120 corresponds to the "scene determination unit" and "function restriction unit" of the present disclosure.

[0123] That is, when the vehicle is traveling on a road and it is clearly not a parking scene (State 1), the state management unit 120 does not allow the parking space detection unit 140 to function at all. Also, in a scene where it is determined whether or not the scene is a parking scene (State 2), the state management unit 120 only allows the parking space detection unit 140 to detect white lines at a low frequency, and does not allow the parking space detection unit 140 to detect parking spaces or the storage of history by the storage unit 170 to perform these operations. In other words, the function restriction performed by the state management unit 120 restricts the parking space detection by the parking space detection unit 140 and the storage of history by the storage unit 170 depending on the scene.

[0124] As a result, the parking space detection and the history storage are not performed all the time, so that it is possible to reduce the operation load of the parking assistance device 100. Note that, the above-mentioned parking space detection restriction and the history storage restriction may be performed in either case.

[0125] In addition, when the vehicle 1 is off the road and the vehicle speed is equal to or lower than the specified value (the above-mentioned speed threshold), the state management unit 120 sets the parking assistance state to State 2, and otherwise sets the parking assistance state to State 1. In other words, when the vehicle 1 is off the road and the vehicle speed is equal to or lower than the specified value (the above-mentioned speed threshold), the state management unit 120 does not restrict the execution of the detection of the white line by the parking space detection unit 140.

[0126] This allows unrestricted detection of white lines when vehicle 1 is traveling at a relatively slow speed off the road, i.e., in a location that can be assumed to be a parking lot, thereby preventing unnecessary operational load from occurring in determining that it is a parking scene.

[0127] In addition, the status management unit 120 determines that the scene is a parking scene when the vehicle 1 is located outside the road, the vehicle speed is below a specified value, and multiple white lines are detected that are at approximately right angles to the direction of travel of the vehicle 1.

[0128] In this way, when determining a parking scene, the white lines indicating the passage are distinguished from the parking frame lines, so that it is possible to avoid erroneously determining a scene without a parking frame as a parking scene. The state management unit 120 may determine that a parking scene is occurring when either the vehicle 1 is located off the road, the vehicle speed is equal to or lower than a specified value, or a plurality of white lines that form an angle of approximately right angles with the traveling direction of the vehicle 1 are detected. For example, the vehicle 1 being located off the road may be excluded from the determination conditions.

[0129] In addition, the parking space detection unit 140 may be located on the opposite side of the direction in which the vehicle 1 turns, across the line along which the vehicle 1 travels straight, and may be set to include the side or diagonally forward of the occupant based on the position of the occupant when the vehicle 1 turns.

[0130] In this way, the parking space detection area is limited to an area where a parking space is expected, thereby shortening the processing time required for detection. Also, by limiting the size of the parking space detection area to a size that can accommodate one parking space but not two parking spaces, the processing when two parking spaces are detected at the same time can be omitted.

[0131] In addition, the parking space detection unit 140 may select a peripheral image from the peripheral images that includes an area that is on the opposite side of the direction in which the vehicle 1 turns, across the line along which the vehicle 1 travels straight, and that is located to the side or diagonally forward of the occupant based on the position of the occupant when the vehicle 1 turns, and perform parking space detection based on the selected peripheral image.

[0132] This also limits the parking space detection area to an area where a parking space is expected, thereby shortening the processing time required for detection.

[0133] In addition, the route calculation unit 150 calculates a parking route based on information of a parking space selected from the surrounding image, that is, information of a parking space detected by the parking space detection unit 140.

[0134] This allows route calculation to be performed on a targeted basis, thereby shortening processing time compared to a configuration in which parking routes are calculated for all parking spaces.

[0135] In addition, when the parking space detection unit 140 detects a parking space at the stopping position, the driving control unit 160 performs automatic parking toward the detected parking space, and when a parking space is not detected at the stopping position, the parking space detection unit 140 may detect a parking space based on a surrounding image taken when turning or before turning.

[0136] That is, the parking assistance device 100 may first detect a parking space from an image of the rear camera at the stopping position, and if a parking space is not detected, detect a parking space using an image at the time of turning.

[0137] In this way, the parking space detection is not started speculatively before the vehicle 1 stops, so that the operation load for the parking space detection is not generated unnecessarily. In addition, since the probability that the parking space cannot be detected from the image of the rear camera at the stopping position is low, the expected value of the time required for the parking space detection is not significantly larger than the time required for one parking space detection, and there is an effect of time reduction by detecting the parking instruction based on the operation information or movement information history of the vehicle 1. As a result, the time required for parking can be shortened compared to the conventional method.

[0138] In addition, the hazard lamps may be turned on during automatic parking. That is, the driving control unit 160 may output a command to turn on the hazard lamps when the vehicle 1 turns from a straight-ahead state, stops at a position where it has turned by a predetermined angle or more, and performs automatic driving.

[0139] This makes it easier for people around to recognize that the vehicle is in autonomous driving mode.

[0140] (Modification of the first embodiment) A modified example of the first embodiment of the present disclosure will be described below. In the first embodiment, a scene determination is performed, and when the vehicle 1 goes off the road, white line detection is started according to the result of the scene determination, and when a periodically drawn white line is detected, parking space detection is started according to the result of the scene determination.

[0141] Specifically, the state management unit 120 functions as a scene determination unit and a function restriction unit, and determines whether or not the scene is a parking scene, and restricts the parking space detection and the storage of the movement history by the storage unit based on the determination result, but this determination may be left to the occupant. For example, a start button for starting the parking assistance device 100 may be provided, and the parking space detection unit 140 may start the parking space detection when the start button is pressed. In other words, the operation of the occupant may be substituted for the scene determination.

[0142] In the first embodiment, the surrounding image is stored while the vehicle 1 is traveling, and the parking space detection unit 140 detects the parking space when the vehicle 1 is turned. However, the parking space detection may be performed at all times, not only at a specific timing. The storage unit 170 may store the information of the detected parking space, and the route calculation unit 150 may perform route calculation based on the information of the parking space stored in the storage unit 170.

[0143] In the first embodiment, an example is shown in which a parking frame that matches the history is selected based on the position of the occupant at the time of transition from straight driving to turning. However, a parking frame that matches the history may be selected based on the position of the vehicle or the occupant at the time of transition from straight driving to turning. For example, when an occupant is looking at an image of the camera 2 (front camera) provided at the front of the vehicle 1, the front camera approaches the entrance of the parking frame before the occupant, so it is possible to know that the parking frame is available at an earlier timing, and if the parking frame is detected by the front camera image, it may be notified that the parking frame has been detected before the occupant can see it. In other words, regardless of the position of the occupant, it may be determined that there is an available parking frame and transition to turning, so a parking frame that matches the history may be selected based on the position of the vehicle. The entrance of the parking frame is a name given to the area on the aisle in front of the parking frame for convenience, and may be called the area adjacent to the short side of the entrance of the parking frame, or may be called the area extending the parking frame toward the aisle.

[0144] (Second embodiment) The second embodiment of the present disclosure will be described below. In the first embodiment, the operation or route until the occupant stops the vehicle in the parking lot is stored as a history, and the parking instruction is regarded as including an instruction of a target parking frame where the vehicle is to be parked. An example was shown in which a parking frame detected by the parking frame detection unit is selected as a target parking frame, which is located on the opposite side of the vehicle's stop position across the path of the vehicle when traveling straight ahead and matches the history. In other words, the parking position of the subsequent automatic parking is determined by driving to the stop position. When a parking frame is detected, information on the positional relationship of the vehicle 1 with respect to the parking frame at that time is obtained, and by adding information on the vehicle speed and path, the transition of the positional relationship when parking is intended can be predicted. Therefore, at the time when the positional relationship between the vehicle 1 and the parking frame, or the positional relationship between the occupant and the parking frame, an appropriate notification based on the information on the parking frame detected by the parking frame detection unit is performed, thereby enabling the driving to the stop position to be assisted. In the second embodiment, driving assistance to the stop position is disclosed.

[0145] In addition, in the first embodiment, the parking space detection area is limited to a size that does not accommodate two parking spaces, but the parking space detection area may be set so that multiple parking spaces can be detected at the same time. When multiple adjacent parking spaces are detected, driving assistance may be performed to a stopping position so that the driver can park in the parking space that the driver intends to park in.

[0146] When a plurality of consecutive parking frames are detected, such as the parking frame S1 and the parking frame S2 in Fig. 10, the parking frame intended by the occupant may not match the target parking frame (the parking frame for which the target parking position is set). The parking assistance device 100 determines the target parking frame based on the position of the vehicle or the occupant at the time when the vehicle transitions from moving straight to turning when the vehicle is turned. That is, there is a range of positions where the target parking frame becomes the parking frame S1 and a range of positions where the target parking frame becomes the parking frame S2, so when the occupant is located near the boundary line that separates the two ranges, the parking frame intended by the occupant and the target parking frame are likely to mismatch.

[0147] For example, when the driver sees the parking space S1 diagonally ahead and turns the vehicle 1 to park the vehicle in the parking space S1, the point where the driver starts to turn the steering wheel is P5. If the parking assistant 100 selects the parking space S2 as the target parking space based on the fact that the point P5 where the vehicle 1 turns is located in front of the parking space S2 (frontage), the target parking space S2 will not match the parking space S1 that the driver intends.

[0148] In the second embodiment, the parking assistance device 100 provides assistance by notifying the driver, thereby guiding the driver to turn the steering wheel while avoiding timing at which erroneous estimation is likely to occur. The second embodiment will be described below.

[0149] In the example shown in Fig. 14, a parking lot is shown in which three parking frames S4, S5, and S6 are arranged in order from the negative side of the X direction. The parking frame S4 ​​is a parking frame sandwiched between the parking frame lines L5 and L6, the parking frame S5 is a parking frame sandwiched between the parking frame lines L6 and L7, and the parking frame S6 is a parking frame sandwiched between the parking frame lines L7 and L8. In the example shown in Fig. 14, the vacant parking frames are S5 and S6, and the vehicle 1 is proceeding from the negative side to the positive side of the X direction along the passage facing the three parking frames S4, S5, and S6.

[0150] FIG. 14 illustrates positions Q1 to Q5 of the occupant (driver) at each time when the vehicle 1 passes in front of a parking space. Note that the position of the occupant and the vehicle 1 is shown in the position Q1, but the vehicle 1 is omitted and only the occupant is shown in each of the positions Q2 to Q5. Q1 is the position of the occupant at the time when the detection of the parking space S5 is notified, for example. It is preferable to notify the occupant in advance that there is a countdown and that the driver should turn the steering wheel during the countdown, such as "Turn the steering wheel during the countdown" when the parking assistance device 100 starts detecting the parking space. In this way, the occupant can turn the steering wheel at a timing according to his / her parking intention and the notification.

[0151] For example, while the occupant's position is in the range from Q1 to Q2, the parking assistant 100 announces "Do you want to park in the parking space in front?" Then, since the parking space S6 is farther from the occupant than the parking space S5, and the parking space S5 is closer, the occupant can determine that the parking assistant 100 is asking whether to park the vehicle 1 in the parking space S5. Next, while the occupant's position moves from Q2 to Q3, the parking assistant 100 counts down "3, 2, 1". Since the occupant has heard "Turn the steering wheel during the countdown" in advance, if the occupant wants to park in the parking space S5, he or she can turn the steering wheel accordingly. Then, the target parking position is set to the parking space S5. If the occupant does not want to park in the parking space S5, he or she can simply not turn the steering wheel.

[0152] When the occupant's position reaches Q3, the parking assistant 100 announces "Do you want to enter the next space?". The occupant knows that the next space is parking space S6 from the positional relationship between the parking space and the vehicle 1. Then, while the occupant's position moves from Q4 to Q5, the parking assistant 100 counts down "3, 2, 1". If the occupant turns the steering wheel during this time, the target parking position is set to parking space S6.

[0153] In this way, by receiving a notification at a timing according to the relative position with respect to the parking space and turning the steering wheel in response to the notification, the target parking position can be reliably set to the parking space where the occupant intends to park.

[0154] Next, an operation example of the parking assistance device 100 according to the second embodiment will be described. FIG. 15 and FIG. 16 are flowcharts showing an operation example of parking assistance control in the parking assistance device 100 according to the second embodiment. The flow of FIG. 16 starts when a button for starting the parking assistance device 100 is pressed. It is assumed that at the start, the vehicle 1 is traveling at low speed through the passage of the parking lot, and its course is approximately perpendicular to the detected white lines. The flowcharts of FIG. 15 and FIG. 16 shown below show the part related to notification, and omit the part not related to notification. Therefore, the processing shown in FIG. 11 to FIG. 13 in the first embodiment may be executed in parallel with the processing of the following flowcharts.

[0155] When the button for starting the parking assistance device 100 is pressed, the parking assistance device 100 starts speed control (step S201) as shown in Fig. 15. This speed control is a control for suppressing the speed of the vehicle 1 in order to increase the time for the occupant to judge the parking space. For example, when the vehicle speed is controlled to 3 km / h, the vehicle 1 passes in front of a parking space spaced 2.5 m apart in 3 seconds.

[0156] The parking assistance device 100 further starts parking space detection (step S202). The parking space detection is performed based on the image of the front camera. The speed control and the parking space detection are not limited to the above steps, but are continuously performed in the process after step S202.

[0157] After step S202, the parking assistance device 100 judges whether or not a parking space is detected (step S203). If the result of the judgment is that a parking space is not detected (step S203, NO), the process of step S203 is repeated.

[0158] On the other hand, when a parking space is detected (step S203, YES), the parking assistance device 100 calculates a time margin based on the distance to the parking space and the vehicle speed (step S204). The time margin is, for example, the time until the vehicle 1 reaches the guidance end point. The guidance end point is a position corresponding to Q3 or Q5 in FIG. 14, and is a position where the countdown by the parking assistance device 100 ends.

[0159] After step S204, the parking assistance device 100 judges whether the time margin is 5 seconds or less (step S205). If the result of the judgment is that the time margin exceeds 5 seconds (step S205, NO), the process returns to step S204.

[0160] If the time margin is 5 seconds or less (step S205, YES), the parking assistance device 100 starts outputting the message A (step S206). That is, it waits until the time margin is 5 seconds, and then outputs the message A. The message A may be a message including a countdown, such as "Shall I enter this parking space? 3, 2, 1." In this way, the parking assistance device 100 adjusts the time so that the vehicle 1 will reach the guidance end point when the output of the message A ends.

[0161] Next, the parking assistance device 100 judges whether or not the absolute value of the steering angle is greater than 5 degrees (step S207). Note that the processes of steps S207 and S208 are executed while the message A is being output in step S206. In other words, it is judged whether or not the steering wheel is turned during the countdown.

[0162] As a result of the determination, if the absolute value of the steering angle is greater than 5 degrees (step S207, YES), the process proceeds to step S218. On the other hand, if the absolute value of the steering angle is less than 5 degrees (step S207, NO), the parking assistance device 100 determines whether the time margin is 0 seconds or less (step S208). This time margin is the same as the number notified as message A. In other words, it is determined whether the steering wheel is turned before the countdown is notified as zero. As a result of the determination, if the time margin is greater than 0 seconds (step S208, NO), the process returns to step S207. On the other hand, if the time margin is less than 0 seconds (step S208, YES), that is, if zero has been notified, the parking assistance device 100 sets the timer to 0 seconds and starts timing (step S209). The timer counts the elapsed time after starting timing.

[0163] After step S209, the parking assistance device 100 outputs a beep sound such as "beep" (step S210). The beep sound is a sound to inform the driver that the parking frame detected in step S203 is not the target parking frame because no steering operation was detected during the guidance period, and the beep sound continues only while the process of step S210 is being executed.

[0164] As shown in FIG. 16, after step S210, the parking assistance device 100 judges whether or not a parking space has been detected (step S211). The parking space to be judged in step S211 is not the first parking space detected in step S203, but the next parking space. After the parking space is detected in step S203, the parking space detection is continued. If the next parking space is detected, the parking assistance device 100 immediately judges YES in step S211. If the parking space is not detected as a result of the judgment (step S211, NO), the parking assistance device 100 judges whether or not the timer is less than 1 second (step S212).

[0165] As a result of the determination, if the timer is greater than 1 second (step S212, NO), the process returns to step S203. Since step S203 is a step for detecting the first parking space, if the next parking space is not detected within 1 second, it means that it is determined that the situation is not one in which parking spaces are to be detected continuously. In other words, if the next parking space is not detected within 1 second, the next detected parking space is treated as the first parking space. On the other hand, if the timer is less than 1 second (step S212, YES), the process returns to step S210. Since step S210 is a beep sound step, a beep sound is output for an upper limit of 1 second.

[0166] Returning to the determination in step S211, if the next parking frame is detected (step S211, YES), the parking assistance device 100 calculates a time margin (step S213). The time margin is, for example, a time until the vehicle 1 reaches the guidance end point.

[0167] After step S213, the parking assistance device 100 judges whether the time margin is four seconds or less (step S214). If the result of the judgment is that the time margin exceeds four seconds (step S214, NO), the process returns to step S213.

[0168] On the other hand, if the time margin is four seconds or less (step S214, YES), the parking assistance device 100 starts outputting message B (step S215). That is, it waits until the time margin is four seconds before outputting message B. The time threshold being shorter than step S205 corresponds to message B being shorter than message A, since message B is for guidance to the second parking space. Message B may be, for example, a countdown message such as "Shall I enter next? 3, 2, 1." Furthermore, the parking assistance device 100 adjusts the time so that the vehicle 1 reaches the guidance end point when the output of message B ends.

[0169] Next, the parking assistance device 100 judges whether or not the absolute value of the steering angle is greater than 5 degrees (step S216). Note that the processes of steps S216 and S217 are executed while the message B is being output. In other words, it is judged whether or not the steering wheel is turned during the countdown.

[0170] If the result of the determination is that the absolute value of the steering angle is 5 degrees or less (step S216, NO), parking assistance device 100 determines whether the time margin is 0 seconds or less (step S217). This time margin is the same as the number notified as message A. In other words, it is determined whether the steering wheel is turned before the countdown is notified as zero. If the result of the determination is that the time margin is greater than 0 seconds (step S217, NO), the process returns to step S216.

[0171] On the other hand, if the time margin is 0 seconds or less (step S217, YES), the process returns to step S209. In other words, if the time margin is zero, a beep sound such as "beep" is generated to notify that the detected parking frame is not the target parking frame.

[0172] Returning to the judgment of step S216, if the absolute value of the steering angle is greater than 5 degrees (step S216, YES), the process transitions to step S218. The same applies to the case of YES in step S207. In step S218, the parking assistance device 100 starts outputting a message C (step S218). The message C is a notification that the target parking frame has been determined, and may be, for example, a message such as "Automatic parking will be performed in this frame. Please park with your back to the frame." After step S218, this control ends.

[0173] According to the second embodiment configured as described above, even if there are adjacent vacant parking spaces, an action to specify a parking space can be accepted so that the space in which the vehicle 1 will be parked is not unclear.

[0174] The flowcharts shown in Figures 15 and 16 are an example of control for instructing the driver to turn the steering wheel at the timing when the target parking frame can be specified, and an example of control for notification when parking in the specified parking frame is shown below. The process of the flowchart shown in Figure 17 is an example of control for notification when parking, and this may be performed after step S218 in Figure 16.

[0175] 17 starts with the premise that the occupant has turned the steering wheel and is moving forward toward the reverse start position (turning position). In order to detect that the reverse start position has been reached, the parking assistance device 100 determines whether the vehicle speed is 0 (step S219). If the result of the determination is that the vehicle speed is not 0 (step S219, NO), the process of step S219 is repeated.

[0176] On the other hand, if the vehicle speed is 0 (step S219, YES), the vehicle has reached the reverse start position, so the parking assistance device 100 superimposes a frame image on the peripheral image of the vehicle 1, calculates the parking path, and outputs a message D (step S220). The message D is a confirmation request for the target parking frame, and may be, for example, a message such as "If this parking frame is OK, put the gear in R and release the brake."

[0177] Next, the parking assistance device 100 judges whether or not the gear is in R and the brake is released (step S221). If the result of the judgment is that the gear is in R and the brake is not released (step S221, NO), the process of step S221 is repeated. That is, the parking assistance device 100 waits for the driver to confirm the target parking frame.

[0178] On the other hand, if the gear is in R and the release of the brake is detected (step S221, YES), the parking assistance device 100 outputs a message E notifying the start of automatic parking (step S222). The message E may be, for example, a message such as "Automatic parking will start. Please be careful around the vehicle."

[0179] After step S222, the parking assistance device 100 automatically drives the vehicle 1 along the parking path (step S223).

[0180] After step S223, the parking assistance device 100 determines whether or not the vehicle 1 has reached the end point (step S224). If the result of the determination is that the vehicle 1 has not reached the end point (step S224, NO), the process returns to step S223.

[0181] On the other hand, if the vehicle 1 reaches the end point (step S224, YES), the parking assistance device 100 automatically stops the vehicle 1 and outputs a message F notifying the completion of automatic parking (step S225). The message F may be, for example, a message such as "Automatic parking has ended. Please put the gear into park."

[0182] After step S225, the parking assistance device 100 judges whether or not it has detected that the gear has been put in P (step S226). If the result of the judgment is that it has not been detected that the gear has been put in P (step S226, NO), the process of step S226 is repeated. On the other hand, if it has been detected that the gear has been put in P (step S226, YES), this control ends.

[0183] In order to avoid the flowchart becoming complicated, illustrations and explanations of exception processing such as when the occupant does not follow the instructions of the message are omitted. This exception processing can be implemented by supplementing it appropriately. For example, there may be a case where the occupant turns the steering wheel when a parking space is detected, but does not intend to park in the detected parking space, and only turns into the passage. As an exception processing in such a case, in step S219, it may be determined that the steering operation is not an operation for indicating a parking space, on the condition that the vehicle speed does not become 0 even after a predetermined time has elapsed, and the processing may be transitioned to step S203.

[0184] In this way, when there are multiple parking frames detected, such as the parking frame S1 and the parking frame S2 in Fig. 10, the support by notification can prevent the setting of the target parking frame from being erroneous. However, when there is only one detected parking frame, as in Fig. 6, the setting of the target parking frame is not erroneous, so the support by notification is ineffective, and the occupant may feel bothered.

[0185] Therefore, when only one parking space is detected, it is better to make the number of notifications smaller than when multiple parking spaces are detected. For example, when multiple parking spaces are detected, the inquiry of parking intention and the time allowance are notified, such as "Do you want to enter this parking space? 3, 2, 1", but when only one parking space is detected, only the inquiry of parking intention, such as "Do you want to enter this parking space?" may be notified. Also, when only one parking space is detected, nothing may be notified.

[0186] (Third embodiment) The third embodiment of the present disclosure will be described below. If the parking frame intended for parking is reliably set as the target parking frame, the driver can save the trouble of checking the target parking frame and correcting the target parking frame. Therefore, the parking assistance device 100 may allow the driver to steer the vehicle so as to identify the parking frame intended for parking, and set the target parking frame based on the observation and analysis of the steering behavior.

[0187] 18 to 23 are diagrams for explaining examples of steering actions of the vehicle 1 in the third embodiment.

[0188] In the example shown in FIG. 18, etc., a parking lot in which four parking frames S7, S8, S9, and S10 are arranged in order from the negative side in the X direction is shown. The parking frame S7 is a parking frame sandwiched between the parking frame lines L9 and L10, the parking frame S8 is a parking frame sandwiched between the parking frame lines L10 and L11, the parking frame S9 is a parking frame sandwiched between the parking frame lines L11 and L12, and the parking frame S10 is a parking frame sandwiched between the parking frame lines L12 and L13. In the example shown in FIG. 18, etc., the vacant parking frames are S8 and S9, and the vehicle 1 proceeds along the passage facing the four parking frames S7, S8, S9, and S10 from the positive side to the negative side in the X direction, and a steering action is performed to identify the parking frame in which the vehicle 1 intends to park.

[0189] The parking assistance device 100 detects a turn in the opposite direction to the turning direction at the time when the vehicle stopped, which is indicated by the history, and selects a parking frame that matches the history based on either the position of the vehicle when the turn in the opposite direction starts, or the position of the vehicle when the turn in the opposite direction ends, or the direction of the vehicle body. Turning in the opposite direction is steering to swing the front end of the vehicle body toward the opposite side of the stopping position, that is, toward the parking frame, and may be called reverse steering. For example, as shown in FIG. 18, when the occupant steers the vehicle body so that the corner of the front end of the vehicle body approaches the parking frame to which the vehicle is intended to park, the parking frame to which the corner of the vehicle body approaches closest may be determined and the target parking frame may be set. In the example of FIG. 18, the parking frame S9 corresponding to the position when the vehicle 1 moves the furthest in the + side of the Y direction is set as the target parking frame.

[0190] In addition, the parking assistance device 100 may specify the parking frame indicated by the line W extending forward from the right side of the vehicle body, that is, the side facing the parking frame, when the occupant turns the steering wheel toward the parking position, as shown in Fig. 19, for example, and set the target parking frame. In the example of Fig. 19, the parking frame S8 overlapping with the line W is set as the target parking frame.

[0191] In addition, the parking assistance device 100 may specify the moving direction M of the right front corner just before the vehicle 1 turns left, and set the parking frame in the moving direction M (the direction of the arrow M) as the target parking frame, as shown in Fig. 20. In the example of Fig. 20, the parking frame S9 overlapping with the moving direction M is set as the target parking frame.

[0192] Also, as shown in FIG. 21, for example, the parking assistance device 100 may specify the direction E (the direction of the arrow E) of the right front corner as seen by the occupant just before the vehicle 1 turns left, and set the parking frame in that direction E as the target parking frame. In the example of FIG. 21, the parking frame S8 overlapping with the direction E is set as the target parking frame. That is, the steering to specify the intended parking frame may be any of steering to the opposite side of the intended parking frame at the entrance of the intended parking frame, steering to temporarily turn to the side of the intended parking frame at the entrance of the intended parking frame, steering to direct the vehicle body to the intended parking frame, and steering to bring the vehicle body closer to the intended parking frame. When specifying a parking frame based on the orientation (direction) of the vehicle body, it may be any of the direction pointed by the side of the vehicle opposite to the parking position, the moving direction of the corner of the tip of the vehicle body opposite to the parking position, or the direction in which the occupant looks at the corner of the tip of the vehicle opposite to the parking position.

[0193] When identifying a parking space based on the direction, the parking assistance device 100 may determine the area from the entrance of the parking space on the aisle side to the center of the parking space as the determination area representing the parking space. This determination area is the range where the occupant directs his / her gaze when selecting a parking space, and when the occupant operates the vehicle 1 toward the determination area, the parking assistance device 100 may set the center of each determination area, evaluate the distance between the line extending in the direction and each center, and set the parking space that is the smallest distance from the line extending in the direction among the centers of the determination areas as the target parking space.

[0194] In addition, the steering to identify the parking space where the occupant intends to park may be steering based on a reverse steering operation as shown in Fig. 22 and Fig. 23. A reverse steering operation is, for example, a steering action of turning slightly to the right before turning significantly to the left, and expanding the trajectory to the right. Reverse steering may be habitually performed as a habit, and there are many people who perform reverse steering unconsciously. Therefore, even if the occupant performs a reverse steering operation when turning the vehicle 1 for parking, there is no risk that the behavior will be seen as unusual by those around.

[0195] The position where the reverse steering operation is performed may be specified as the position where the reverse steering operation is started. For example, as shown in FIG. 22, if the position of the front end of the vehicle body at the time when the reverse steering operation is started is the closest to the center line C1 of the parking frame S9 among the multiple parking frames, the parking assistant 100 sets the target parking frame to the parking frame S9. Also, as shown in FIG. 23, if the position of the front end of the vehicle body at the time when the reverse steering operation is started is the closest to the center line C2 of the parking frame S8 among the multiple parking frames, the parking assistant 100 sets the target parking frame to the parking frame S8. In other words, it is only necessary to select a parking frame that matches the history based on either the position of the vehicle when the reverse turning starts, or the position of the vehicle when the reverse turning ends, or the direction of the vehicle body.

[0196] In addition, when multiple parking spaces are detected, such as the parking space S1 and the parking space S2 in FIG. 10, steering is effective to specify the parking space that is intended to be parked. However, when only one parking space is detected, as in FIG. 6, setting error of target parking space does not occur, so special steering is not required.

[0197] Therefore, the parking assistance device 100 may request steering to specify the parking space in which the vehicle is intended to park only when multiple parking spaces are detected, and may not notify the driver of steering or may only notify the driver of a simple steering when only one parking space is detected.

[0198] For example, when there are a plurality of vacant parking spaces on the right side of the vehicle 1, the parking assistance device 100 may announce a predetermined message. The predetermined message may be, for example, any one of "Turn the steering wheel in front of the parking space where you want to park", "Lightly turn the front of the vehicle body toward the parking space where you want to park", "Bring the corner of the vehicle body closer to the parking space where you want to park", and "Before turning the front of the vehicle body to the left, point the front of the vehicle body toward the parking space where you want to park".

[0199] Following the predetermined message, the parking assistance device 100 may announce, for example, "Automatic parking will start if you park with your back to the parking frame." Also, when only one parking frame is detected, the parking assistance device 100 may simply announce, "Automatic parking will start if you park with your back to the parking frame." without requesting steering to specify the parking frame in which the driver wants to park.

[0200] In addition, the parking assistance device 100 does not need to repeat the announcement for each parking frame even when there are multiple vacant parking frames. For example, if there is no steering to specify the intended parking frame in front of the parking frame S1, the target parking frame candidates are narrowed down to one, so there is no need to repeat the announcement for the parking frame S2. For example, when the parking assistance device 100 detects the second parking frame, it may announce only once, "Please lightly swing the tip of the vehicle toward the parking frame you want to park in." And, if there is no corresponding action in front of the first parking frame S1, the parking assistance device 100 can determine the intention of parking even if there is no corresponding action in front of the next parking frame S2, so the parking frame S2 may be set as the target parking frame.

[0201] In addition, the above-mentioned embodiments are merely examples of the embodiment of the present disclosure, and the technical scope of the present disclosure should not be interpreted as being limited by them. In other words, the present disclosure can be embodied in various forms without departing from the gist or main features thereof. [Industrial Applicability]

[0202] The device disclosed herein is useful as a parking assistance device and a parking assistance method that enable smooth automatic parking in a short amount of time. [Explanation of symbols]

[0203] 1 vehicle 2 Camera 10 Operating device 20 HMI device 30 Vehicle control device 40 Navigation devices 100 Parking assistance device 101 CPU 102 ROM 103 RAM 104 I / O 105 IMP 110 Operation reception section 120 Status Management Unit 130 Image acquisition unit 140 Parking space detection unit 150 Route Calculation Unit 160 Driving control unit 170 Storage section 180 Display image output unit

Claims

1. A parking space detection unit that detects a parking space based on a surrounding image showing the surroundings of the vehicle; A storage unit that stores a history of either an operation by a passenger of the vehicle or a movement of the vehicle; an instruction detection unit that detects a parking instruction from the occupant based on the history; a vehicle control unit that performs automatic parking of the vehicle after the instruction detection unit detects the parking instruction; Equipped with The instruction detection unit determines that the parking instruction has been detected when the history indicates that the vehicle has stopped after steering and turning from a straight line and the occupant has performed a predetermined parking instruction operation, The predetermined parking instruction operation includes at least one of releasing the steering wheel, releasing the brake, turning on the hazard lamp, and operating a button corresponding to a parking instruction. Parking assistance device.

2. A parking space detection unit that detects a parking space based on a surrounding image showing the surroundings of a vehicle; A storage unit that stores a history of either an operation by a passenger of the vehicle or a movement of the vehicle; an instruction detection unit that detects a parking instruction from the occupant based on the history; a vehicle control unit that performs automatic parking of the vehicle after the instruction detection unit detects the parking instruction; Equipped with The instruction detection unit determines that the parking instruction has been detected when the history indicates that the vehicle has stopped after steering and turning from a straight line and the occupant has performed a predetermined parking instruction operation, a scene determination unit that determines whether or not a scene is a parking scene in which the vehicle is parked; a function limiting unit that limits a function of the parking assistance device when the scene is not a parking scene; Further equipped with The function restriction performed by the function restriction unit is When the scene is not a parking scene, the parking space detection unit is restricted from detecting a parking space; If the scene is not a parking scene, limiting the storage of the history by the storage unit; At least one of Parking assistance device.

3. A parking space detection unit that detects a parking space based on a surrounding image showing the surroundings of the vehicle; A storage unit that stores a history of either an operation by a passenger of the vehicle or a movement of the vehicle; an instruction detection unit that detects a parking instruction from the occupant based on the history; a vehicle control unit that performs automatic parking of the vehicle after the instruction detection unit detects the parking instruction; Equipped with The instruction detection unit determines that the parking instruction has been detected when the history indicates that the vehicle has stopped after steering and turning from a straight line and the occupant has performed a predetermined parking instruction operation, A route calculation unit that calculates a route for the automatic parking is further provided. The parking instruction includes an instruction for a target parking space in which the vehicle is to be parked. The instruction detection unit selects, as a target parking frame, a parking frame that is located on the opposite side of the vehicle stop position across the path of the vehicle when traveling straight ahead and that matches the history, among the parking frames detected by the parking frame detection unit; The route calculation unit calculates the route based on information of the target parking frame. Parking assistance device.

4. The instruction detection unit selects a parking frame that matches the history based on a position of the vehicle or an occupant at a time when the vehicle transitions from moving straight to turning, as indicated by the history.

4. The parking assistance device according to claim 3.

5. The instruction detection unit is Detecting a turn in a direction opposite to a turning direction at the time when the vehicle stopped, which is indicated by the history; the position of the vehicle when the reverse turn began; or selecting a parking frame that matches the history based on either the position of the vehicle or the orientation of the vehicle body when the reverse turn is completed; 4. The parking assistance device according to claim 3.

6. The storage unit stores either the surrounding image captured while the vehicle is traveling or information about a parking space detected while the vehicle is traveling, The instruction detection unit selects the target parking frame from the information of the parking frame detected by the parking frame detection unit or the information of the parking frame detected while the vehicle is traveling and stored in the storage unit based on the surrounding image captured while the vehicle is traveling and stored in the storage unit.

4. The parking assistance device according to claim 3.

7. The parking space detection unit is A parking space detection is performed based on the surrounding image captured when the vehicle is stopped. When a parking frame is detected based on the peripheral image captured when the vehicle is stopped, the target parking frame is selected from information on the parking frame based on the peripheral image captured when the vehicle is stopped.

7. A parking assistance device according to claim 6.

8. The position of the occupant is the position of the head of the occupant.

6. A parking assistance device according to claim 4 or 5.

9. The orientation of the vehicle body is The direction in which the side surface of the vehicle body facing the opposite direction is pointing, or The direction of travel of the tip corner of the vehicle body on the opposite side, or The direction in which the occupant looks at the tip corner of the vehicle body on the opposite side, Either 6. A parking assistance device according to claim 5.

10. When it is determined that the parking instruction has been detected, the vehicle control unit outputs a command to turn on a hazard lamp.

2. The parking assistance device according to claim 1.

11. A notification unit that notifies an occupant of the vehicle is further provided. The notification includes a notification based on information of the parking space detected by the parking space detection unit.

2. The parking assistance device according to claim 1.

12. The notification includes an inquiry about an intention to park in the parking space, and is notified when the parking space and the vehicle body are in a predetermined positional relationship.

12. A parking aid according to claim 11.

13. The notification includes a notification of a timing to steer the vehicle.

12. A parking aid according to claim 11.

14. The notification includes a notification requesting steering to specify a parking frame in which parking is intended.

12. A parking aid according to claim 11.

15. The steering to designate the intended parking frame is Steering to the opposite side of the intended parking space at the entrance of the intended parking space, Steering that temporarily turns toward the intended parking space at the entrance of the intended parking space; Steering to point the vehicle towards the intended parking space, Steering to bring the vehicle closer to the intended parking space, Contains any of the following:

15. A parking aid according to claim 14.

16. When the parking frame detection unit detects one parking frame, the notification unit reduces the notification compared to when the parking frame detection unit detects a plurality of parking frames. A parking assistance device according to any one of claims 11 to 15.

17. The function restriction unit does not restrict the execution of detection of a white line by the parking space detection unit when the vehicle is located outside a road and the vehicle speed is equal to or lower than a specified value.

3. The parking assistance device according to claim 2.

18. The scene determination unit The vehicle is off the road and the vehicle speed is equal to or lower than a specified value; and a plurality of white lines are detected, the angles of which with respect to the traveling direction of the vehicle are within a predetermined range; If either or both of the above conditions are met, the scene is determined to be a parking scene.

3. The parking assistance device according to claim 2.

Citation Information

Patent Citations

  • Parking support device

    JP2019073191A

  • Image display device and driving assist system

    JP2019104336A

  • Parking support apparatus

    JP2019182038A

  • Parking assist device, parking assist method, and parking assist program

    JP2022069161A

  • Parking assist device

    WO2014073193A1