Parking assistance device

Abstract

To provide a parking support device that enables a vehicle to be automatically parked smoothly in a short time.SOLUTION: A parking support device includes a parking frame sensing part that senses parking frames on the basis of a periphery image showing a periphery of a vehicle, a memorizing part that memorizes a history of either of operation by an occupant in the vehicle and movement of the vehicle and an instruction detecting part that detects a parking instruction by the occupant, on the basis of the history, in which when the history shows that the vehicle has run straight, turned by being steered and then stopped and when the occupant performs predetermined parking instruction operation, determines that the instruction detecting part has detected the parking instruction. The parking instruction includes an instruction for a target parking frame on which the vehicle is made to be parked. The instruction detecting part selects, as the target parking frame, a parking frame positioned at the opposite side across a course on which the vehicle runs straight of a position on which the vehicle is stopped and matching with the history, of the parking frames detected by the parking frame sensing part.SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] The present disclosure relates to a parking assistance device. [Background technology]

[0002] Conventionally, there is known a parking assistance device that can automatically park a vehicle in a parking space by turning the steering wheel at a predetermined position around the parking space and driving backward from the position where the steering wheel is turned at a predetermined angle. For example, Patent Document 1 discloses a configuration in which the parking assistance device is activated after the vehicle is parked next to the parking space, and the parking position of the vehicle in the parking space is uniquely determined. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-69161 Summary of the Invention [Problem to be solved by the invention]

[0004] However, in the conventional technology, after the parking assistance system is activated, it is necessary to detect a parking space and confirm the safety of the area the vehicle will pass through when parking, which causes the vehicle to stay in the passage for a long time, which results in a longer time required for parking overall, and may cause the following vehicle to honk its horn while the parking assistance system is being operated.

[0005] An object of the present disclosure is to provide a parking assistance device that can perform automatic parking smoothly in a short amount of time. [Means for solving the problem]

[0006] The parking assistance device according to the present disclosure comprises: a parking space detection unit that detects a parking space based on a surrounding image showing the surroundings of the vehicle; a storage unit that stores a history of either an operation by a vehicle occupant or a movement of the vehicle; an instruction detection unit that detects a parking instruction from the occupant based on the history; Equipped with the instruction detection unit determines that the parking instruction has been detected when the history indicates that the vehicle has stopped after steering from a straight ahead direction and turning, and the occupant has performed a predetermined parking instruction operation, the parking instruction includes an instruction for a target parking space in which the vehicle is to be parked; The instruction detection unit selects, as a target parking space, a parking space that is located on the opposite side of the vehicle's stopping position across the path of the vehicle when traveling straight ahead from the parking space detected by the parking space detection unit and that matches the history. [Effects of the Invention]

[0007] According to the present disclosure, automatic parking can be performed smoothly in a short time. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram for explaining automatic parking by a vehicle to which a parking assistance device according to a first embodiment of the present disclosure is applied. [Figure 2] 1 is a diagram showing a vehicle to which a parking assistance device according to an embodiment of the present invention can be applied; [Figure 3] FIG. 1 is a diagram showing a network configuration of a system to which a parking assistance device is applied. [Figure 4] FIG. 2 is a block diagram showing hardware that implements the functions of the parking assistance device. [Figure 5] FIG. 2 is a block diagram showing a parking assistance device. [Figure 6] FIG. 10 is a diagram for explaining setting of a parking space detection area. [Figure 7] FIG. 10 is a diagram illustrating an example of detecting a parking space using a rear camera. [Figure 8] FIG. 10 is a diagram showing an example in which the camera cannot capture the entire parking space. [Figure 9] FIG. 10 is a diagram showing an example in which the entire parking space is captured by the front camera. [Figure 10]FIG. 10 is a diagram illustrating an example in which a plurality of parking spaces are detected. [Figure 11] 4 is a flowchart showing an example of the operation of parking assist control in the parking assist device according to the first embodiment. [Figure 12] 4 is a flowchart showing an example of the operation of parking assist control in the parking assist device according to the first embodiment. [Figure 13] 4 is a flowchart showing an example of the operation of parking assist control in the parking assist device according to the first embodiment. [Figure 14] FIG. 10 is a diagram for explaining an example of parking a vehicle in the second embodiment. [Figure 15] 10 is a flowchart showing an example of the operation of parking assist control in the parking assist device according to the second embodiment. [Figure 16] 10 is a flowchart showing an example of the operation of parking assist control in the parking assist device according to the second embodiment. [Figure 17] 10 is a flowchart showing an example of the operation of parking assist control in the parking assist device according to the second embodiment. [Figure 18] FIG. 10 is a diagram for explaining an example of a steering behavior of a vehicle in the third embodiment. [Figure 19] FIG. 10 is a diagram for explaining an example of a steering behavior of a vehicle in the third embodiment. [Figure 20] FIG. 10 is a diagram for explaining an example of a steering behavior of a vehicle in the third embodiment. [Figure 21] FIG. 10 is a diagram for explaining an example of a steering behavior of a vehicle in the third embodiment. [Figure 22] FIG. 10 is a diagram for explaining an example of a steering behavior of a vehicle in the third embodiment. [Figure 23] FIG. 10 is a diagram for explaining an example of a steering behavior of a vehicle in the third embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0009] (First embodiment) A first embodiment of the present disclosure will be described in detail below with reference to the drawings. FIG. 1 is a diagram illustrating automatic parking by a vehicle 1 to which a parking assistance device 100 according to the first embodiment of the present disclosure is applied. FIG. 2 is a diagram illustrating a vehicle 1 to which the parking assistance device 100 according to the present embodiment can be applied. Note that a Cartesian coordinate system (X, Y) is used in the description of FIG. 1 and other figures. The same Cartesian coordinate system (X, Y) is also used in the figures described below. For example, the X direction indicates the left-right direction relative to the parking position (the position of the vehicle 1 within the parking space), and the Y direction indicates the front-rear direction relative to the parking position. Furthermore, the position of the vehicle 1 is generally represented by the position of a representative point of the vehicle 1. The representative point of the vehicle 1 is the midpoint of the contact points of the two rear wheels. Furthermore, the trajectory of the movement of the vehicle 1 is represented by the trajectory of the representative point of the vehicle 1.

[0010] As shown in Fig. 1, a vehicle 1 is equipped with a camera 2 (see Fig. 2, etc.) that monitors the periphery of the vehicle, and is configured to be able to perform automatic parking in a parking lot where one or more parking spaces are arranged by a parking assistance device 100 shown in Fig. 3. A parking space is an area sandwiched between two approximately parallel parking frame lines. The parking frame lines are arranged at an interval greater than the width of the vehicle 1.

[0011] In the example shown in FIG. 1, a parking lot is shown in which three parking spaces S1, S2, and S3 are arranged in order from the negative side in the X direction. The parking space S1 is a parking space sandwiched between the parking frame lines L1 and L2, the parking space S2 is a parking space sandwiched between the parking frame lines L2 and L3, and the parking space S3 is a parking space sandwiched between the parking frame lines L3 and L4. The left side is the negative side in the X direction and the right side is the positive side in the X direction, with a line parallel to the parking frame lines L2 and L3 and equidistant from both the parking frame lines L2 and L3 as the boundary. The boundary line between the positive and negative sides in the Y direction is the side closest to the vehicle 1 among the short sides of the parking space.

[0012] Here, an example of the case where the vehicle 1 automatically parks in the parking stall S2 will be described. First, the vehicle 1 moves straight in the portion on the positive side of the Y direction from the negative side to the positive side in the X direction of the three parking stalls S1, S2, and S3. The vehicle 1 in the straight-moving state turns at a position P1 in front of the parking stall line L2 located on the negative side in the X direction among the parking stall lines L2 and L3 of the parking stall S2, turns while moving forward to the positive side in the Y direction, and stops at a position P2. The position P2 is a position on the positive side of the parking stall line L3 in the X direction and on the positive side in the Y direction, and is a position where the vehicle 1 can park between the parking stall lines L2 and L3 by turning while retreating from the position P2. The vehicle 1 travels to the position P2 by manual operation of the occupant.

[0013] Then, when the occupant performs a parking instruction operation such as putting the gear into R (reverse), the vehicle 1 performs automatic parking. The vehicle 1 turns while backing up from position P2, enters parking stall S2, and stops at position P3 within parking stall S2 while backing up. During automatic parking, the hazard lights may be turned on. Furthermore, since which parking stall the vehicle will be parked in can be determined from the steering position and steering direction, the occupant does not need to specify a parking position during automatic parking. In the following explanation, it is assumed that the vehicle 1 performs the above-mentioned behavior during automatic parking.

[0014] As shown in FIG. 2, cameras 2 are mounted on four locations on the front, rear, left, and right sides of the body of vehicle 1. Each camera 2 is equipped with a fisheye lens and has a horizontal field of view of 180 degrees or more (see dashed lines). Each camera 2 is mounted at a depression angle to capture the road surface, so when the range of the road surface captured is converted into a horizontal field of view, a single camera 2 captures a range of about 240 degrees of the road surface. For example, the front and rear wheels and the sides of the vehicle are captured in an image captured by side cameras 2A mounted on the left and right sides of the vehicle.

[0015] As shown in FIG. 3 , the vehicle 1 includes four cameras 2, an operation device 10, an HMI (human-machine interface) device 20, a vehicle control device 30, and a parking assistance device 100. The operation device 10 is manually operated by the driver (passenger) and includes physical switches on a panel at the driver's seat, software switches displayed on a touch panel, and devices for driving operations such as a steering wheel, pedals, and gears. The HMI device 20 is used as an HMI for the passenger to input operations to the parking assistance device 100, such as a touch panel attached to a navigation device 40 provided in the vehicle 1. The touch panel may be included in the operation device 10, and various switches grounded to the driver's seat may also be included in the operation device 10, so the operation device 10 and the HMI device 20 may be overlapping.

[0016] The vehicle control device 30 includes a CPU (Central Processing Unit), ROM (Read Only Memory), RAM (Random Access Memory), and input / output circuits, none of which are shown. When the occupant operates the operation device 10, the vehicle control device 30 accepts the operation. In normal driving mode, the vehicle control device 30 controls the steering angle and vehicle speed by driving a motor (not shown) in accordance with operation information, and simultaneously outputs the operation information and vehicle information such as the steering angle and vehicle speed to a LAN (Local Area Network: in-vehicle LAN, etc.). In parking assistance mode, the vehicle control device 30 receives commands for speed and steering angle from the parking assistance device 100 via the LAN, and controls the speed and steering angle in accordance with the commands.

[0017] The parking assistance device 100 acquires operation information of the operation device 10 via a LAN because the vehicle control device 30 monitors the operation of the operation device 10. The occupant can operate the parking assistance device 100 via the HMI device 20. The parking assistance device 100 may receive position information of the vehicle 1 output by the navigation device 40 via a LAN or may acquire the position information directly from the navigation device 40. The camera 2 constantly outputs captured images to the parking assistance device 100. Even when not providing parking assistance, the parking assistance device 100 generates and outputs display images showing the vehicle's surroundings from the captured images. The touch panel also functions as a display that outputs the display images generated by the parking assistance device 100, so the parking assistance device 100 and the HMI device 20 are directly connected. The parking assistance device 100 may receive information on operations performed on the touch panel directly or via a LAN. The parking assistance device 100 can also receive position information of the vehicle 1 output by the navigation device 40 via a LAN.

[0018] The functions of the parking assistance device 100 may also be implemented in the hardware shown in FIG. 4. The parking assistance device 100 includes a CPU 101, a ROM 102, a RAM 103, an I / O (input / output interface) 104, and an IMP (image processor) 105. The parking assistance device 100 may be a computer in which each element is connected by a bus. Multiple elements may be housed on a single chip, or one element may be composed of multiple chips. The bus may not be a single bus, but may be a combination of multiple types of buses. For example, the CPU 101, ROM 102, RAM 103, and IMP 105 housed on a single chip may be connected by a parallel bus, and the I / O 104, which is composed of multiple chips, may be connected to the chip housing the CPU 101, etc., by a serial bus.

[0019] The CPU 101 controls the entire parking assistance device 100. The ROM 102 is an electrically rewritable memory that stores programs executed by the CPU 101 and also functions as a non-volatile data storage area, retaining data and the like even when the parking assistance device 100 is turned off. The RAM 103 is used for temporary storage as a work area for the CPU 101. For example, data that only needs to be stored temporarily, such as the latest surrounding image, is stored in the RAM 103. The RAM 103 has a capacity that can store multiple surrounding images. The IMP 105 is a processor with enhanced processing performance specialized for image processing, and executes the processes of the image acquisition unit 130, the parking space detection unit 140, and the display image output unit 180 shown in FIG. 5.

[0020] The above-mentioned camera 2 constantly outputs the captured camera images to the parking assistance device 100. Even when not providing parking assistance, the parking assistance device 100 generates and outputs a display image showing the periphery of the vehicle 1 from the camera images. As shown in FIG. 5 , the parking assistance device 100 has an operation reception unit 110, a state management unit 120, an image acquisition unit 130, a parking space detection unit 140, a route calculation unit 150, a travel control unit 160, a storage unit 170, and a display image output unit 180.

[0021] The operation receiving unit 110 receives operation information indicating operations by the driver, including driving operations.

[0022] The state management unit 120 manages the parking assistance state (state) according to the operation information and the state of the vehicle. There are a plurality of parking assistance states, for example, state 0 to state 8.

[0023] State 0 is the initial state, and is the state when the parking assistance function is not activated. State 1 is the idle state, and is the state when vehicle speed information and position information are acquired, and when the vehicle speed drops below a threshold, it is determined whether the vehicle is on a road. A road is a road on a map, such as a public road that is not a parking lot with parking spaces.

[0024] State 2 is a monitoring state, in which white line detection is performed at low frequency. State 3 is a preparation state, in which captured images are recorded in association with time information. State 4 is a detection state, in which parking space detection is performed speculatively. State 5 is a calculation state, in which, once the vehicle has turned 30 degrees or more and stopped, parking path calculation is performed speculatively and the occupant's intention to park is queried.

[0025] State 6 is the inquiry state, where the parking route is displayed and the system is waiting for a parking instruction operation. State 7 is the self-driving state, where automatic parking is performed according to the calculated route. State 8 is the end state, where the system is notified that automatic parking has finished.

[0026] The image acquisition unit 130 acquires camera images and generates a surrounding image showing the surroundings of the vehicle 1. The surrounding image may be an overhead image in which the camera image is projected onto the road surface, an image in another format, or the camera image itself. In this embodiment, the surrounding image is assumed to be an overhead image.

[0027] The parking space detection unit 140 sets a parking space detection area in the peripheral image for detecting parking spaces and extracts white lines from the peripheral image within the parking space detection area. The parking space detection area may be the entire peripheral image, or it may be an area in the peripheral image where a parking space is expected. By limiting the parking space detection area to the area where a parking space is expected, the processing time required for detection can be shortened. Furthermore, by limiting the size of the parking space detection area to a size that can accommodate one parking space but not two, processing can be omitted when two parking spaces are detected simultaneously. Generally, detection is performed with the purpose of obtaining information about a specific object, and if a specific object is detected as a result of the detection, it is determined that the object has been "detected" and information about the detected object is output. Detection results are divided into detection and non-detection. If the object is not detected, the purpose of the detection is not achieved. If the object is detected, the purpose of the detection is achieved, and the detection can be rephrased as successful detection.

[0028] The parking space detection unit 140 further generates pairs of extracted white lines and searches for pairs of white lines that meet the conditions for a parking space. For example, when the parking space detection unit 140 detects a pair of white lines from the peripheral image, if the following three conditions are met: the two white lines are parallel on the road surface, the two white lines are longer than the vehicle length, and the distance between the two white lines is greater than the vehicle width, the parking space detection unit 140 determines that the area between the two white lines is an area where the vehicle 1 can be parked (i.e., a parking space). When a parking space is detected, the white lines that make up the parking space can be said to be parking space lines. The parking space detection unit 140 outputs the endpoints of the two parking space lines, that is, the coordinates of the four corners of the parking space. These coordinates are relative coordinates based on the host vehicle.

[0029] The route calculation unit 150 sets the target parking position so that the center of the vehicle body is located in the center of the coordinates of the four corners of the parking space area output by the parking space detection unit 140, and also sets the target parking angle (the direction of the vehicle body when parked at the target parking position) so that it is parallel to the long side of the parking space area, and calculates the parking route to the target parking position. A proposed publicly known method may be applied for parking route calculation.

[0030] The driving control unit 160 controls the steering angle and vehicle speed so that the vehicle 1 drives along the calculated parking path. More specifically, the driving control unit 160 outputs command values ​​for the steering angle and vehicle speed to the vehicle control device 30 according to the target values ​​for the steering angle and vehicle speed. The driving control unit 160 calculates actual measured values ​​for the steering angle and vehicle speed based on the data output by the vehicle control device 30. For example, the actual measured value of the vehicle speed is calculated from data on the rotation speed of the wheels and data on the outer circumference of the wheels. The actual measured value of the steering angle may be data obtained from the steering device, or may be calculated from an inner wheel difference calculated from the difference in the rotation speed of the wheels. When there is a difference between the target values ​​and the actual measured values, the driving control unit 160 corrects the command values ​​for the steering angle and vehicle speed so that the actual measured values ​​for the steering angle and vehicle speed become the same as the target values ​​for the steering angle and vehicle speed.

[0031] The driving control unit 160 tracks the position and orientation of the vehicle 1 while it is traveling, based on the calculated steering angle and actual measurement values ​​of the vehicle speed. In other words, the driving control unit 160 calculates the traveling trajectory of the vehicle 1 during automatic traveling or manual traveling. The traveling trajectory may be calculated in the form of movement information in which the travel distance, the travel direction, and the angle of the vehicle body are added to the time for each unit time. The form of the movement information is not limited to this, and may be, for example, a form using coordinate information.

[0032] During automatic driving, the driving control unit 160 compares the parking route calculated by the route calculation unit 150 with the driving trajectory represented by the movement information, and corrects the steering angle so that the driving trajectory follows the parking route. For example, when the vehicle 1 reverses while turning with a steering angle, if the driving trajectory passes outside the parking route as viewed from the center point of the turn, the driving control unit 160 increases the steering angle to reduce the turning radius. This causes the driving trajectory of the vehicle 1 to be pulled back toward the parking route.

[0033] The storage unit 170 stores the operation information acquired by the operation reception unit 110, the movement information calculated by the driving control unit 160, the surrounding image generated by the image acquisition unit 130, the position information of the parking frame lines and the parking frame detected by the parking frame detection unit 140, etc. in a history information area in chronological order, linking them to time. Since the vehicle 1 moves in response to the operation of the occupant, it is not necessary for either the movement information or the operation information to be stored in the storage unit 170. However, in this embodiment, both the movement information and the operation information are stored in the storage unit 170.

[0034] For example, the storage unit 170 may add information every unit time and fix the size of the storage area allocated for each addition (memory block size). In this case, the history information area is divided into memory blocks and the starting address of the information increases monotonically every unit time, so that the address where the information is stored is linked to the time. Alternatively, the storage unit 170 may not fix the memory size, but may provide a separate starting address table that links the starting address to the time, and update the starting address table every time information is added.

[0035] By specifying the time using any method, it is possible to access the latest information at that time. Also, by accumulating movement information up to that time, it is possible to specify the relative positional relationship of the position at that time to the current position. Conversely, it is also possible to add location information in chronological order as history information, and specify the relative positional relationship to the current position from the difference between the location information at that time and the current position. In other words, it is possible to calculate location information from history information.

[0036] Furthermore, for large-sized data such as peripheral images, only the first address needs to be stored in the history information area.

[0037] Since it is sufficient for history information to be referenced going back a predetermined time, old history information that is no longer needed may be erased from the storage unit 170. For example, the history information area may be set in the storage unit 170 as a fixed area sandwiched between a first address and a last address, and when the address for storing information reaches the last address, it may be configured as a loop memory that continues storing information at the first address. In this way, past history information going back one loop is overwritten and erased, eliminating the need for erasure processing.

[0038] Storing in the history information area does not need to be performed all the time, but may be performed in states 2 to 7 above.

[0039] Furthermore, the storage unit 170 stores the parking route calculated by the route calculation unit 150, and enables the driving control unit 160 to refer to the parking route.

[0040] Furthermore, since parking space detection takes time to process, parking space information, which is the detection result, may be obtained at a time significantly later than the time the camera image was captured. Even in this case, the parking space information is stored in the memory unit 170 in association with the capture time. For example, information is added to the memory unit 170 at each unit time, and the capacity of the memory area allocated for each addition (memory block size) is fixed, so that the memory block is allocated at a fixed size regardless of the amount of data written in each addition. When information is added to the memory unit 170, if the starting address of the peripheral image has been obtained but the result of detecting a parking space in the peripheral image (parking space information) has not yet been obtained, the parking assistance device 100 writes only the obtained information into the memory block without waiting for the parking space information. When the parking space information is obtained, the parking space information is written into the memory block in which the starting address of the peripheral image that was the target of the detection is written.

[0041] For example, if a steering operation is performed in state 3 and speculative parking space detection is performed when the system proceeds to state 4, whether parking space information has been obtained or not when the system proceeds to state 5 depends on the relationship between the time from when the vehicle 1 turns to when it stops and the time required for parking space detection. For example, if parking space information is not written in the corresponding memory block when the system proceeds to state 5, the process of starting route calculation may involve a wait loop that polls the address where the parking space information is written, or the processor (e.g., IMP) that performs parking space detection may be configured to generate an interrupt when parking space detection is completed. In this way, route calculation can be started immediately when parking space detection is completed. In this way, by storing parking space information, etc. in the storage unit 170 in association with the image capture time, the automatic parking process can be efficiently controlled.

[0042] The display image output unit 180 generates a display image based on a camera image or a peripheral image. The display image may be a bird's-eye view image that overlooks the periphery of the vehicle 1 from above. In other words, the display image output unit 180 may output a bird's-eye view image as the display image.

[0043] The display image output unit 180 displays a message or a graphic superimposed on the display image in response to a request from the state management unit 120. For example, when starting automatic parking, the display image output unit 180 may superimpose a semi-transparent rectangle indicating the position of the parking space detected by the parking space detection unit 140 on the overhead image, and may display the route from the vehicle to the target parking position calculated by the route calculation unit 150 with a dotted line.

[0044] The message may not only be displayed on the image, but may also be read aloud. Alternatively, the message may be output only as audio. Specifically, the state management unit 120 issues a command to designate one of the preset texts and output it. In response, the display image output unit 180 generates an image of a character string corresponding to the designated text, superimposes it on the display image, and outputs it to the HMI device 20. At the same time, the display image output unit 180 outputs the audio data that was saved together with the text to the HMI device 20 together with the display image. This allows the occupant to receive the message even if they are not looking at the HMI device 20.

[0045] Next, the states managed by the state management unit 120 will be described in detail. As described above, the state management unit 120 accepts operations by the occupant and manages the parking assistance state. The operations by the occupant are not limited to operations on the parking assistance device 100, but also include operations for driving. For example, when the occupant starts the vehicle 1 by turning on the IG (Ignition) switch, the parking assistance device 100 also starts up during the startup sequence of the vehicle 1, and the state progresses from State 0 (initial state) to State 1 (pause state).

[0046] In State 1 (resting state), only the state management unit 120 and the driving control unit 160 operate. The state management unit 120 acquires vehicle speed information via the driving control unit 160, and when the vehicle speed drops below a threshold value (e.g., 10 km / h), queries the navigation device 40 to see if the vehicle 1 is located on a road. If the vehicle 1 is off the road (e.g., on a store premises), the state management unit 120 advances the state to 2. State 1 may be referred to as a road driving scene. Since a state corresponds to a scene, the state management unit 120 may be referred to as an example of a scene determination unit, which will be described later. Furthermore, the state management unit 120 may be referred to as an example of a function restriction unit, since it partially or fully restricts functions such as a parking space detection unit depending on the state (scene).

[0047] In State 2 (monitoring state), the parking space detection unit 140 also partially operates. In other words, in State 1 (road driving scene), execution of all functions of the parking space detection unit 140 is restricted. The parking space detection unit 140 periodically (for example, every one second) performs white line detection. White line detection is performed by releasing some of the functions of the parking space detection unit 140 and not performing other functions, such as evaluating whether a white line is a parking space line. The parking space detection unit 140 detects white lines that are at an angle within a predetermined range (60 degrees to 120 degrees) with respect to the axis of symmetry of the vehicle 1 (a line passing through the center of the front end of the vehicle and the center of the rear end of the vehicle 1) from an image captured around the vehicle 1. Since white line detection is performed to determine whether the vehicle 1 is in a parking lot, the spacing or length of the white lines is not evaluated. Therefore, the detection range may be a limited range in the peripheral image that captures only the portion in front of the parking space line. If the parking space detection unit 140 detects multiple white lines that meet the conditions, it determines that the vehicle 1 is in a parking lot, and the state management unit 120 advances the state.

[0048] In the example shown in FIG. 1, state 2 is a state in which the vehicle 1 is moving straight ahead in the +Y direction from the parking stalls S1, S2, and S3 and before detecting the white line.

[0049] In State 3 (preparation state), the peripheral images are linked to time information and stored in the storage unit 170. Of the images used for white line detection, only images in which white lines (white lines approximately perpendicular to the axis of symmetry of the vehicle 1) are detected may be stored as peripheral images, or images captured periodically (for example, every second) may be stored as peripheral images without being selected.

[0050] The state management unit 120 stores in the storage unit 170 the time when a steering force equal to or greater than a threshold value (for example, 1 kg) is applied to the steering wheel from a straight-ahead state with a steering angle of approximately 0 as the steering time, and advances the state.

[0051] In the example shown in FIG. 1, the vehicle 1 is traveling straight ahead in the +Y direction from the parking spaces S1, S2, and S3, and the state from when the white line is detected until the vehicle 1 turns at position P1 is state 3.

[0052] In state 4 (detection state), the parking space detection unit 140 detects the parking space from the image that is taken just before the steering time. Alternatively, the parking space detection unit 140 may detect the parking space from the image that is taken just before the steering time.

[0053] At this point, it has not been determined whether or not to perform automatic parking, so this parking space detection can be said to be speculative. If there is a parked vehicle in the parking space, part of the parking frame line will be blocked by the parked vehicle, so the length of the detected white line will not satisfy the condition, and the parking space will not be detected. Therefore, when a parking space is detected, it can be said that an empty parking space is detected.

[0054] The state management unit 120 advances the state when the vehicle body turns 30 to 60 degrees from the steering point and the vehicle 1 stops.

[0055] In the example shown in FIG. 1, State 4 is the state from when the vehicle 1 turns at position P1 until it stops at position P2.

[0056] In State 5 (calculation state), the route calculation unit 150 accumulates movement information from the time when the image in which the parking space is detected was captured to the time when the vehicle 1 stopped, and calculates the vehicle position (coordinate offset) at the time when the image was captured, based on the current vehicle position. After completing the parking space detection, the route calculation unit 150 adds the coordinate offset to the position information of the parking space, and calculates the position information of the parking space based on the current vehicle position. The route calculation unit 150 calculates a parking route for parking in the parking space from the current vehicle position.

[0057] At this point, it has not been determined whether or not to perform automatic parking, so it can be said that the route calculation is speculative. Once the parking route is calculated, the display image output unit 180 outputs an image showing the parking space and the parking route, and outputs a message asking the occupant whether they wish to park. After outputting the image and message, the state management unit 120 advances the state.

[0058] In the example shown in FIG. 1, State 5 is the state from when the vehicle 1 stops at position P2 and the parking assistance device 100 calculates the parking path until it outputs the above message.

[0059] In State 6 (inquiry state), the operation reception unit 110 receives a parking instruction operation. If a parking instruction operation is received, the state management unit 120 advances the state. In the example shown in FIG. 1, State 6 is the state from when the message is output until the occupant performs a parking instruction operation.

[0060] In State 7 (self-driving state), the driving control unit 160 performs automatic parking according to the calculated route. After automatic parking is completed, the state management unit 120 advances the state. In the example shown in FIG. 1, State 7 is the state in which the vehicle 1 performs automatic parking from position P2 to position P3.

[0061] In State 8 (end state), the parking assistance device 100 notifies the user that automatic parking has ended. After the notification is completed, the state management unit 120 returns the state to State 2. In the example shown in FIG. 1, State 8 is the state from when the vehicle 1 stops at position P3 until the notification is made.

[0062] When the vehicle 1 drives to a vacant parking space where the vehicle 1 is to park, the occupant turns the steering wheel when they visually recognize that the parking space is available. Therefore, the parking space detection unit 140 may set a parking space detection area according to the position of the occupant when they turn the steering wheel. The position of the occupant may be determined as the position of the occupant's head.

[0063] The occupant must also visually confirm the traveling direction of the vehicle 1, so he / she visually confirms the parking space located diagonally ahead. However, there is a time lag between seeing the vacant parking space and turning the steering wheel, and the vehicle 1 may be moving to the side of the vacant parking space at the time of turning the steering wheel. Therefore, it is preferable that the parking space detection unit 140 sets the parking space detection area diagonally ahead from the side of the occupant.

[0064] 6 shows the same parking frames S1, S2, S3 as shown in FIG. 1, and shows an example in which the vehicle 1 traveling from the + side of the X direction to the - side of the parking frames S1, S2, S3 in the Y direction is about to park in the parking frame S2. In this example, when the vehicle 1 turns at a position P4 before the parking frame line L3 in order to park in the parking frame S2, the area from the side of the occupant (the + side of the Y direction) to the diagonally forward (the - side of the X direction and the + side of the Y direction) is set as the parking frame detection area D1.

[0065] Since the occupant turns the steering wheel to reverse into parking space, the parking space detection area is located on the opposite side of the vehicle 1 across the line along which the vehicle 1 has traveled straight. Therefore, the parking space detection area is preferably set on the opposite side of the direction in which the vehicle 1 has turned across the line along which the vehicle 1 has traveled straight, and includes the side or diagonally forward of the occupant based on the position of the occupant when the vehicle 1 has turned. The line along which the vehicle 1 has traveled straight can also be described as a line extending a straight portion of the route the vehicle 1 has traveled. When turning the steering wheel, as shown in FIG. 6, the camera 2A (side camera) provided on the side of the vehicle 1 is often positioned to the side of an empty parking space, so the parking space detection unit 140 may set the side of the side camera as the parking detection area.

[0066] Furthermore, if the vehicle 1 is turned around when an occupant is positioned in front of a vacant parking space, and the vehicle 1 turns approximately 45 degrees before stopping, camera 2 (rear camera) installed at the rear of vehicle 1 often captures the entire vacant parking space, as shown in Figure 7.

[0067] FIG. 7 shows an example in which the vehicle 1 is trying to park in the parking space S2, similar to FIG. 6. After the vehicle 1 turns to park in the parking space S2, the camera 2 installed at the rear of the vehicle 1 can detect the four corners of the parking space S2 when the vehicle 1 turns forward and stops.

[0068] Therefore, for example, the parking space detection unit 140 may not perform speculative parking space detection in state 4, and may first perform parking space detection from the image of the rear camera in state 5, and if no parking space is detected, may perform parking space detection using the image at the time of turning.

[0069] In this way, the effect of concealing the processing time is smaller than when parking space detection is started speculatively in state 4, but since parking space detection is started speculatively as soon as vehicle 1 stops without waiting for an instruction for automatic parking, the time required for parking is shorter than with the conventional method.

[0070] For example, if a driver passes in front of a vacant parking space and then the vehicle 1 is turned, or if the steering angle is small and the vehicle 1 turns with a large turning radius, the entire vehicle body may pass in the direction of the entrance of the vacant parking space. In this case, if there is a parked vehicle in a parking space adjacent to the vacant parking space, the vacant parking space (long side) may be blocked by the parked vehicle, and the entire vacant parking space may not be captured by the rear camera (see, for example, Figure 8). If speculative parking space detection is not performed in State 4, parking space detection is first performed in State 5 using the image from the rear camera, and after the parking space detection fails, parking space detection is performed using the image at the time of turning the steering wheel.

[0071] In Fig. 8, as in Fig. 6, the example is shown that the vehicle 1 is going to park in the parking space S2, and after the vehicle 1 turns to park in the parking space S2, when the vehicle 1 turns forward, the parking space S1 of the parking space S1 on the negative side of the X direction from the parking space S2 blocks the parking space line L2, so the camera 2 that is provided at the rear of the vehicle 1 cannot capture the whole parking space S2.

[0072] In this case, the parking space detection is performed twice, so the time required for parking will be longer than with the conventional method. However, if the driver drives properly, this situation will not occur, so it can be said that the probability of this situation occurring is low. Therefore, the expected time required for parking will be smaller than with the conventional method.

[0073] Also, for example, speculative parking space detection may be executed in State 3 instead of starting in State 4 after steering. In this case, it is expected that parking space detection will be completed when the vehicle 1 turns and stops and enters State 5. In this case, only route calculation needs to be performed in State 5, so that the parking position and parking route can be presented at an earlier timing than when parking space detection is performed in State 4.

[0074] In this case, parking space detection may be performed using an image from a camera 2 (front camera) provided at the front of the vehicle 1. The front camera approaches the entrance of the parking space before the occupant, so it can detect the parking space at an earlier timing.

[0075] For example, FIG. 9 shows an example in which the front camera 2 of the vehicle 1 moving straight from the parking space S3 in the +X direction to the -X direction captures the entire parking space S1, which is located on the most -X side (forward) of the parking spaces S1, S2, and S3.

[0076] However, since parking frames are drawn periodically, this can be addressed by predicting the optimum position for parking frame detection.Since the front camera is positioned lower than the side cameras, it can often capture the parking frame line without being blocked by parked vehicles, even if it is just before the line extending the long side.

[0077] When speculative parking space detection is performed in State 3, it may happen that a plurality of parking spaces are detected, such as the parking space S1 and the parking space S2 in FIG. 10, when the state shifts to State 5. In such a case, the information of the parking space (for example, the parking space S1) whose photographing time is immediately before the position P5 at the time of turning is selected from the history information (not the latest parking space detection result obtained at the time of turning). Since the purpose is to shorten the time required for parking, it is more effective to not allow the driver to make a selection.

[0078] If the time required for the vehicle 1 to move one parking space distance is shorter than the time required for parking space detection, the parking space detection in the area of ​​the parking space S1 has not been completed at the time of generating the peripheral image of the parking space S1. In such a case, the parking space detection unit 140 may store the peripheral image of the parking space S1 in the storage unit 170, skip the parking space detection, and perform the parking space detection when the information of the parking space S1 becomes necessary. If the parking space detection in the area of ​​the parking space S2 is terminated and the parking space detection in the area of ​​the parking space S1 is started, the terminated parking space detection for the parking space S2 will be re-executed when the information of the parking space S2 becomes necessary. Therefore, the expected value of the time required for parking is greater when the parking space detection is terminated. Alternatively, to eliminate the need to skip the parking space detection, the parking assistance device 100 may request the occupant to decelerate the vehicle 1 so that both the parking space S1 and the parking space S2 can be detected.

[0079] Furthermore, the standard interval between parking space lines (the long sides of the parking space lines) is 2.5 m. When vehicle 1 travels in front of a parking space at 2.5 m per second, vehicle 1 moves forward by one space per second. 2.5 m per second is 9 km / h, which is slightly faster than the 8 km / h speed posted in many parking lots. When vehicle 1 is traveling while searching for an available parking space, it is traveled at a speed even slower than 8 km / h. Therefore, if vehicle 1 is traveled while storing surrounding images in storage unit 170 at intervals of one frame per second, for example, there is no risk of missing an available parking space.

[0080] For example, when a parking space is detected from an image captured by the front camera, it is best to capture the image with the front camera positioned slightly in front of the extension of the parking space lines. Therefore, rather than maintaining a constant time interval between captures, it is best to maintain a constant interval between captures (for example, 2.5 m). By repeating parking space line detection from State 2 onwards, the distance between parking space lines can be determined. Therefore, by predicting the location of the next parking space line and capturing an image when vehicle 1 reaches that location, a front camera image captured at a position suitable for parking space detection can be obtained. When traveling at a speed of 8 km / h, the time interval for storing front camera images is 1 second or more. Furthermore, since the speed is not necessarily constant and the time interval between captures is irregular, it is best to also store the time of capture.

[0081] For example, if movement information is stored in a memory block at one-second intervals, it is advisable to store the starting address of the peripheral image in the latest memory block, as well as the fraction of a second from the time of photographing. This allows the distance traveled by vehicle 1 between the time the movement information was stored and the time of photographing to be calculated using an interpolation method. This allows the distance to the detected parking space from the peripheral image to be correctly calculated later.

[0082] Furthermore, the storage interval for movement information from the time of steering can be shortened (for example, 0.1 second intervals). Storing movement information in detail allows for more accurate estimation of the movement of the vehicle 1 from the steering point to the parking position. The storage interval can be longer because the movement information before steering is not directly related to the estimation accuracy of the parking space position. If the time when automatic parking starts and the time when the image is taken are different, the amount of movement between them needs to be compensated for, but this can be compensated for using the method described above.

[0083] Next, an operation example of the parking assistance device 100 according to the first embodiment will be described. Figures 11, 12, and 13 are flowcharts showing an operation example of parking assistance control in the parking assistance device 100 according to the first embodiment. The following flowchart starts when the power of the vehicle 1 is turned on.

[0084] 11, the parking assistance device 100 (state management unit 120) sets the parking assistance state to State 0 (step S101). In step S101, the state management unit 120 initializes the internal state of the parking assistance device 100.

[0085] Next, the state management unit 120 sets the parking assistance state to State 1 (step S102). In step S102, the state management unit 120 acquires vehicle speed information from the driving control unit 160.

[0086] Next, the state management unit 120 determines whether the vehicle speed is less than a speed threshold (for example, 10 km / h) (step S103). If the result of the determination is that the vehicle speed is equal to or greater than the speed threshold (step S103, YES), the process returns to step S102.

[0087] On the other hand, if the vehicle speed is less than the speed threshold (step S103, NO), the parking assistance device 100 (state management unit 120) determines whether the vehicle 1 is on a road (step S104). The determination as to whether the vehicle 1 is on a road is made based on information from the navigation device 40. If the determination result shows that the vehicle 1 is on a road (step S104, YES), the process returns to step S102.

[0088] On the other hand, if the vehicle 1 is not on the road (step S104, NO), the state management unit 120 sets the parking assistance state to State 2 (step S105). Note that, even if information from the navigation device 40 cannot be obtained, the parking assistance device 100 may proceed to State 2. This may also be said to skip Step S104. In State 2 (step S105), the parking assistance device 100 (parking space detection unit 140) performs white line detection at a low frequency (every 1 second).

[0089] In State 2, the parking assistance device 100 (state management unit 120) determines whether or not the conditions for determining a parking scene are met (step S106). The conditions for determining a parking scene are that there are multiple white lines that are approximately perpendicular to the axis of symmetry of the vehicle 1 and that these white lines are parallel to each other. If the determination result shows that the conditions for determining a parking scene are not met (step S106, NO), the process returns to step S105.

[0090] On the other hand, if the parking scene determination condition is met (step S106, YES), the state management unit 120 sets the parking assistance state to State 3 (step S107). In step S107, the parking assistance device 100 (storage unit 170) stores the surrounding image in association with time information.

[0091] Next, the state management unit 120 acquires steering angle data from the traveling control unit 160 and determines whether the absolute value of the steering angle is greater than an angle threshold value (for example, 5 degrees) (step S108). If the determination result shows that the absolute value of the steering angle is greater than the angle threshold value (step S108, YES), the processing of step S108 is repeated.

[0092] On the other hand, if the absolute value of the steering angle is less than the angle threshold value (step S108, NO), as shown in Fig. 12, state management unit 120 acquires data on the steering force applied to the steering wheel from cruise control unit 160 and determines whether the absolute value of the steering force is greater than a threshold value (e.g., 1 kg) (step S109). In other words, it is not determined that steering has occurred during a turn, that is, when the steering angle is large, but it is determined that steering has occurred when the vehicle is traveling straight, that is, when a steering force is applied after the steering angle has become small. Furthermore, the conditions for determining straight traveling may include traveling forward a predetermined distance (e.g., 1 m) with a small steering angle.

[0093] If the result of the determination is that the absolute value of the steering force is less than the threshold value (step S109, NO), the process returns to step S108 (see also FIG. 11). On the other hand, if the absolute value of the steering force is greater than the threshold value (step S109, YES), the state management unit 120 sets the parking assistance state to State 4 (step S110).

[0094] In step S110, the parking assistance device 100 (storage unit 170) stores the steering time, and the parking space detection unit 140 starts parking space detection using the image captured just before the steering time as the target. In addition, the parking assistance device 100 (route calculation unit 150) determines the straight-ahead direction based on history information before the steering time. Specifically, at the time of steering, the peripheral image generated from the image captured just before the steering remains in the memory, so the storage unit 170 prohibits overwriting of the area containing the peripheral image, and writes the starting address and the capturing time into the memory block. Then, the state management unit 120 passes the starting address to the parking space detection unit 140 and instructs it to detect a parking space. The state management unit 120 also passes the start address to the route calculation unit 150, and the route calculation unit 150 reads the traveling direction (yaw angle) of the vehicle 1 from 3 seconds before the steering time to the steering time from the history information (memory block up to 3 seconds ago) and determines the average value as the straight-ahead heading. Note that the above processing and numerical values ​​are merely examples.

[0095] Next, the parking assistance device 100 (route calculation unit 150) calculates a turning angle based on the straight-ahead direction (step S111), and the state management unit 120 determines whether the turning angle is greater than a first angle (e.g., 60 degrees) (step S112). The turning angle is calculated, for example, by subtracting the straight-ahead direction from the yaw angle and finding the absolute value.

[0096] As a result of the determination, if the turning angle is greater than the first angle (step S112, YES), the parking assistance device 100 stops detecting the parking space and cancels the turning time (step S113). After that, the process returns to step S108 (see also FIG. 11). In step S113, if the turning angle is greater than the first angle, the parking assistance device 100 (state management unit 120) determines that the vehicle 1 has made a right or left turn and cancels the process associated with the turning.

[0097] On the other hand, if the turning angle is equal to or less than the first angle (step S112, NO), the parking assistance device 100 determines whether the vehicle speed is 0 (step S114). If the result of the determination is that the vehicle speed is not 0 (step S114, NO), the process returns to step S111. That is, the turning angle monitoring process is in a loop state until the vehicle 1 comes to a stop.

[0098] On the other hand, if the vehicle speed is 0 (step S114, YES), the parking assistance device 100 (state management unit 120) determines whether the turning angle is greater than a second angle (e.g., 30 degrees) (step S115). If the determination result shows that the turning angle is equal to or less than the second angle (step S115, NO), the process returns to step S111. In other words, if the turning angle is small, the turning angle monitoring process enters a loop state.

[0099] On the other hand, if the turning angle is greater than the second angle (step S115, YES), the state management unit 120 sets the parking assistance state to State 5 (step S116). This can also be expressed as the state progressing to State 5 (step S116) when the vehicle 1 turns 45 degrees±15 degrees from a straight-ahead state and stops. In step S116, the parking assistance device 100 determines the amount of movement of the vehicle 1 since the image capture time. For example, the route calculation unit 150 adds up the amount of movement since the image capture time of the image that is the target of parking space detection, which is recorded in the history information, and calculates the current position (including relative coordinates and yaw angle) based on the position of the vehicle 1 at the image capture time.

[0100] In State 5, the parking assistance device 100 (state management unit 120) determines whether the parking space detection is completed (step S117). If the determination result shows that the parking space detection is not completed (step S117, NO), the process of step S117 is repeated. On the other hand, if the parking space detection is completed (step S117, YES), the parking assistance device 100 determines whether the parking space detection is successful (step S118).

[0101] As a result of the determination, if the parking space detection is not successful (step S118, NO), the parking assistance device 100 (display image output unit 180) outputs a message that the parking space is not detected (step S119), and this control ends (see also FIG. 13). If the parking space detection is not successful, it may be rephrased as a case where the parking space detection unit 140 does not detect a parking space, and if the parking space detection is successful, it may be rephrased as a case where the parking space detection unit 140 detects a parking space.

[0102] On the other hand, if the parking space detection is successful (step S118, YES), as shown in FIG. 13, the parking assistance device 100 calculates the position of the parking space (step S120). In step S116, the current position is calculated based on the position of the vehicle 1 at the time the image was captured, and the result of the parking space detection indicates the position of the parking space based on the position of the vehicle 1 at the time the image was captured. Therefore, by adding up both the current position and the result, the position of the parking space based on the current position can be calculated. This calculation may be performed by the route calculation unit 150. Subsequently, the route calculation unit 150 performs parking route calculation based on the position of the parking space based on the current position (step S121).

[0103] Next, the parking assistance device 100 (state management unit 120) determines whether the parking path calculation is completed (step S122). If the determination result shows that the parking path calculation is not completed (step S122, NO), the process of step S122 is repeated. On the other hand, if the parking path calculation is completed (step S122, YES), the parking assistance device 100 determines whether the parking path calculation is successful (step S123).

[0104] If the result of the determination is that the parking path calculation has not been successful (step S123, NO), the parking assistance device 100 (display image output unit 180) outputs a message to the effect that the parking path cannot be set (step S124), and this control ends.

[0105] On the other hand, if the parking path calculation is successful (step S123, YES), the state management unit 120 sets the parking assistance state to State 6 (step S125). In step S125, the display image output unit 180 outputs a message asking the occupant whether or not to select automatic parking, and the state management unit 120 sets the timer to 0. The timer is used to measure the waiting time for the occupant's response to the above message, i.e., the inquiry about automatic parking.

[0106] In State 6, the state management unit 120 determines whether the timer is less than three seconds (step S126). If the timer is three seconds or more (step S126, NO), the parking assistance device 100 outputs a message indicating that automatic parking is being canceled (step S127), and this control ends. This can also be rephrased as saying that if the occupant does not respond within three seconds, it is assumed that the occupant did not select automatic parking.

[0107] On the other hand, if the timer is less than 3 seconds (step S126, YES), the parking assistance device 100 determines whether or not a parking instruction operation has been performed (step S128). The parking instruction operation includes at least one of the following, or a combination of two or more of: changing the gear position, releasing the steering wheel, releasing the brake, turning on the hazard lights, and operating a button corresponding to a parking instruction. For example, if the gear is in R and the steering wheel and brake are released, the parking assistance device 100 determines that a parking instruction operation has been performed.

[0108] As a result of the determination, if a parking instruction operation has not been performed (step S128, NO), the process returns to step S126. On the other hand, if a parking instruction operation has been performed (step S128, YES), the state management unit 120 sets the parking assistance state to State 7 (step S129). In step S129, the parking assistance device 100 (driving control unit 160) controls the vehicle 1 to start automatic driving. This can also be rephrased as saying that if the occupant performs a parking instruction operation within three seconds, it is assumed that the occupant has selected automatic parking, and automatic parking begins.

[0109] Next, the parking assistance device 100 determines whether the vehicle position is a parking position (step S130). If the determination result shows that the vehicle position is not a parking position (step S130, NO), the processing of step S130 is repeated. This can also be said to continue automatic driving until the vehicle 1 reaches a parking position.

[0110] On the other hand, if the vehicle position is a parking position (step S130, YES), the parking assistance device 100 outputs a message to the effect that automatic parking has ended (step S131). After that, the process returns to step S105.

[0111] Note that, in the above steps S104 to S124, if the vehicle speed becomes greater than the speed threshold, the process may immediately transition to the process of step S102. Also, in the above steps S105 to S124, if the position of vehicle 1 becomes on a road, the process may immediately transition to the process of step S102. In other words, even if the vehicle 1 is traveling at a low speed off a road and a white line is detected, and storage of movement information of vehicle 1 has started, when vehicle 1 appears on the road or the vehicle speed increases, parking assistance device 100 may determine that the scene is not a parking scene and return to step S102 for determining whether or not it is a parking scene.

[0112] Furthermore, in the determination process from step S125 onwards, the occupant may issue an instruction not to perform automatic parking. For example, when waiting for a parking instruction operation in steps 126 and 128, the HMI device 20 may accept an instruction not to perform automatic parking. If an instruction is given by the occupant, the instruction takes priority, so the determination may not be based on the position or speed of the vehicle 1. Furthermore, if the IG is turned off in any step, this control ends.

[0113] Furthermore, the message asking the occupant whether to select automatic parking may be output at a timing earlier than step S125. This may be rephrased as speculatively inquiring of the occupant without waiting for the completion of route calculation. For example, in step S116, the above message may be output when vehicle 1 turns 45 degrees ±15 degrees and stops. In this way, route calculation is performed in parallel with the occupant's operation to issue a parking instruction, thereby shortening the overall time required for parking.

[0114] Furthermore, taking into consideration that there are multiple parking instruction operations and that each of the multiple parking instruction operations has a specific order in which they are performed, the step of determining whether or not the parking instruction operation has been performed may be divided into multiple steps. For example, when the gear position is shifted to R after step S116, a message may be output asking the occupant whether or not to select automatic parking, and when the steering wheel and brake are subsequently released, acceptance of the parking instruction operation may be completed and automatic driving may begin.

[0115] In this way, when the timing of querying the occupant is advanced, parking assistance device 100 may lengthen the waiting time for the query in step S126 (the time until it is determined that the occupant has not selected automatic parking) taking into consideration the time required for the occupant to perform the operation and the time for parallel calculation processing. For example, when parking assistance device 100 performs the query when the gear position is in R, the waiting time may be increased by one second from three seconds in step S126 to four seconds, and when the query is performed at the time of step S116 (when the vehicle is stopped), the waiting time may be increased by three seconds from three seconds to six seconds.

[0116] According to the present embodiment configured as described above, the state management unit 120 detects a parking instruction from the occupant based on the history of operation information or movement information of the vehicle 1 stored in the storage unit 170. The state management unit 120 corresponds to the "instruction detection unit" of the present disclosure.

[0117] Specifically, the state management unit 120 determines that a parking instruction has been detected when the history of operation information or movement information of the vehicle 1 indicates that the vehicle 1 has stopped after steering from a straight-ahead position and making a turn, and the occupant has performed a predetermined parking instruction operation. When the state management unit 120 detects a parking instruction, the driving control unit 160 outputs a command to the vehicle control device 30 to automatically park the vehicle 1 based on the parking space information. The driving control unit 160 corresponds to the "vehicle control unit" of the present disclosure.

[0118] As a result, when vehicle 1 turns and stops, automatic parking can be initiated in response to a parking instruction. The route for automatic parking generally overlaps with the area that vehicle 1 has passed through, so there is no need to request safety confirmation again. In addition, since the location to park can be determined from the position and direction of the turn, there is no need to instruct the parking location.

[0119] In this way, in this embodiment, no confirmation or instruction is required when starting automatic parking, so automatic parking can be started in a short time.

[0120] In addition, in this embodiment, the vehicle 1 performs speculative parking space detection using the steering of the vehicle 1 from a straight traveling state as a trigger, so that the parking space detection can be performed by utilizing the time from steering to stopping. As a result, automatic parking can be started in a shorter time than a configuration in which parking space detection is performed after stopping. Therefore, in this embodiment, automatic parking can be performed more smoothly.

[0121] Furthermore, the state management unit 120 determines whether the scene is a parking scene in which the vehicle 1 is parking, and when it is a parking scene, the state of the parking assistance is set to State 3, the state of the parking assistance in a scene in which it is determined whether it is a parking scene is set to State 2, and the state of the parking assistance in a scene in which it is not determined whether it is a parking scene (a scene in which the vehicle is traveling on a road) is set to State 1. In other words, when it is not a parking scene, the state management unit 120 sets the state of the parking assistance to State 2, which restricts the functions of the parking assistance device 100 more than State 3 and thereafter, and when it is not necessary to determine whether it is a parking scene, the state management unit 120 sets the state of the parking assistance to State 1, which further restricts the functions of the parking assistance device 100. Thus, the state management unit 120 corresponds to the "scene determination unit" and "function restriction unit" of the present disclosure.

[0122] That is, when the vehicle is traveling on a road and it is clearly not a parking scene (State 1), the state management unit 120 does not allow the parking space detection unit 140 to function at all. Also, in a scene where it is determined whether or not it is a parking scene (State 2), the state management unit 120 only allows the parking space detection unit 140 to detect white lines at a low frequency, and does not allow the parking space detection unit 140 to detect parking spaces or the storage of history by the storage unit 170 to do so. In other words, the function restriction performed by the state management unit 120 restricts the parking space detection by the parking space detection unit 140 and the storage of history by the storage unit 170 depending on the scene.

[0123] As a result, parking space detection and history storage are not performed all the time, which reduces the operational load of the parking assistance device 100. Note that the above-mentioned restrictions on parking space detection and history storage may be performed in either order.

[0124] Furthermore, when the vehicle 1 is located off the road and the vehicle speed is equal to or less than the specified value (the above-mentioned speed threshold), the state management unit 120 sets the parking assistance state to State 2, and otherwise sets the parking assistance state to State 1. In other words, when the vehicle 1 is located off the road and the vehicle speed is equal to or less than the specified value (the above-mentioned speed threshold), the state management unit 120 does not restrict the execution of white line detection by the parking space detection unit 140.

[0125] This allows detection of white lines to be performed without restriction when vehicle 1 is traveling at a relatively slow speed off the road, i.e., in a location that can be assumed to be a parking lot, thereby preventing unnecessary operational load from occurring in determining that it is a parking scene.

[0126] In addition, the state management unit 120 determines that the scene is a parking scene if the vehicle 1 is located outside the road, the vehicle speed is below a specified value, and multiple white lines are detected that are at approximately right angles to the direction of travel of the vehicle 1.

[0127] In this way, when determining a parking scene, the white lines indicating the passage are distinguished from parking frame lines, so that it is possible to avoid erroneously determining a scene without a parking frame as a parking scene. Note that the state management unit 120 may determine that a parking scene exists when either the vehicle 1 is located off the road, the vehicle speed is equal to or less than a specified value, or multiple white lines that form an angle of approximately right angles with the traveling direction of the vehicle 1 are detected. For example, the vehicle 1 being located off the road may be excluded from the determination conditions.

[0128] In addition, the parking space detection unit 140 may be located on the opposite side of the direction in which the vehicle 1 turns across the line along which the vehicle 1 travels straight, and the parking space detection area may be set to include the side or diagonally forward of the occupant based on the position of the occupant when the vehicle 1 turns.

[0129] In this way, the parking space detection area is limited to the area where the parking space is expected, thereby shortening the processing time required for detection. Also, by limiting the size of the parking space detection area to a size that can accommodate one parking space but not two parking spaces, it is possible to omit processing when two parking spaces are detected at the same time.

[0130] In addition, the parking space detection unit 140 may select a peripheral image from the peripheral images that is located on the opposite side of the direction in which the vehicle 1 turned, across the line along which the vehicle 1 traveled straight, and that includes an area located to the side or diagonally forward of the occupant based on the position of the occupant when the vehicle 1 turned, and perform parking space detection based on the selected peripheral image.

[0131] This also limits the parking space detection area to an area where a parking space is expected, thereby shortening the processing time required for detection.

[0132] In addition, the route calculation unit 150 calculates a parking route based on information about a parking space selected from the surrounding image, that is, information about a parking space detected by the parking space detection unit 140.

[0133] This allows route calculation to be performed with a narrowed target, thereby shortening the processing time compared to a configuration in which parking routes are calculated for all parking spaces.

[0134] In addition, when the parking space detection unit 140 detects a parking space at the stopping position, the driving control unit 160 performs automatic parking toward the detected parking space, and when a parking space is not detected at the stopping position, the parking space detection unit 140 may detect a parking space based on a surrounding image taken when turning or before turning.

[0135] That is, the parking assistance device 100 may first detect a parking space from the image of the rear camera at the stopping position, and if the parking space is not detected, may detect a parking space using the image at the time of turning.

[0136] In this way, the parking space detection is not started speculatively before the vehicle 1 stops, so that the operational load for the parking space detection is not generated unnecessarily. In addition, since the probability that the parking space cannot be detected from the image of the rear camera at the stopped position is low, the expected value of the time required for the parking space detection is not significantly longer than the time required for one parking space detection, and there is an effect of time reduction by detecting the parking instruction based on the operation information or movement information history of the vehicle 1. As a result, the time required for parking can be shortened compared to the conventional method.

[0137] Furthermore, the hazard lights may be turned on during automatic parking. That is, the driving control unit 160 may output a command to turn on the hazard lights when the vehicle 1 turns from a straight-ahead state, stops at a position where it has turned by a predetermined angle or more, and performs automatic driving.

[0138] This makes it easier for those around you to recognize that the vehicle is in autonomous driving mode.

[0139] (Modification of the first embodiment) Hereinafter, a modified example of the first embodiment of the present disclosure will be described. In the first embodiment, scene determination is performed, and when the vehicle 1 moves off the road, white line detection is started according to the result of the scene determination, and when periodically drawn white lines are detected, parking space detection is started according to the result of the scene determination.

[0140] Specifically, the state management unit 120 functions as a scene determination unit and a function restriction unit, and determines whether or not a parking scene is occurring, and restricts parking space detection and storage of the movement history by the storage unit based on the determination result. However, this determination may be left to the occupant. For example, a start button for starting the parking assistance device 100 may be provided, and the parking space detection unit 140 may start parking space detection when the start button is pressed. In other words, the occupant's operation may be substituted for the scene determination.

[0141] In the first embodiment, an example is shown in which a peripheral image is stored while the vehicle 1 is traveling, and the parking space detection unit 140 detects a parking space when the vehicle 1 is turned, but the parking space detection may be performed at all times instead of only at specific timings. In addition, the storage unit 170 may store information on the detected parking space, and the route calculation unit 150 may perform route calculation based on the information on the parking space stored in the storage unit 170.

[0142] In the first embodiment, a parking space matching the history is selected based on the position of the occupant when the vehicle transitions from straight driving to turning. However, a parking space matching the history may be selected based on the position of the vehicle or the occupant when the vehicle transitions from straight driving to turning. For example, when the occupant is looking at an image from camera 2 (front camera) installed at the front of vehicle 1, the front camera approaches the entrance of the parking space before the occupant, so the occupant can determine that a parking space is available at an earlier timing. If the parking space is detected in the front camera image, the occupant may be notified that the parking space has been detected before he or she can see it. In other words, regardless of the occupant's position, the vehicle may determine that there is an available parking space and transition to turning. Therefore, a parking space matching the history may be selected based on the vehicle's position. The entrance of the parking space is a convenient name given to the area on the aisle in front of the parking space. It may be referred to as the area adjacent to the short side of the entrance of the parking space, or it may be referred to as the area extending from the parking space toward the aisle.

[0143] (Second embodiment) A second embodiment of the present disclosure will be described below. In the first embodiment, the operation or route taken by the occupant to park the vehicle in a parking lot is stored as a history, and the parking instruction is considered to include an instruction for a target parking space where the vehicle will be parked. Among the parking spaces detected by the parking space detection unit, a parking space that is located on the opposite side of the vehicle's stopping position across the vehicle's straight-ahead path and matches the history is selected as the target parking space. In other words, the parking position for subsequent automatic parking is determined by driving to the stopping position. When a parking space is detected, information on the positional relationship of the vehicle 1 with respect to the parking space at that time is obtained. Adding information on the vehicle speed and path makes it possible to predict the transition of the positional relationship when parking is intended. Therefore, when the positional relationship between the vehicle 1 and the parking space or the positional relationship between the occupant and the parking space reaches a specific positional relationship, an appropriate notification based on the parking space information detected by the parking space detection unit is provided, thereby enabling assistance in driving to the stopping position. In the second embodiment, driving assistance to the stopping position is disclosed.

[0144] In addition, in the first embodiment, an example is shown in which the parking space detection area is limited to a size that does not accommodate two parking spaces, but the parking space detection area may be set so that multiple parking spaces can be detected simultaneously. And, when multiple consecutive parking spaces are detected, driving assistance may be performed to the stopping position so that the occupant can park in the parking space that he / she intends to park in.

[0145] When a plurality of consecutive parking frames are detected, such as the parking frame S1 and the parking frame S2 in Fig. 10, the parking frame intended by the occupant may not match the target parking frame (the parking frame for which the target parking position is set). The parking assistance device 100 determines the target parking frame based on the position of the vehicle or the occupant at the time when the vehicle transitions from straight driving to turning when steering. That is, there is a range of positions where the target parking frame becomes the parking frame S1 and a range of positions where the target parking frame becomes the parking frame S2. Therefore, when the occupant is located near the boundary line separating the two ranges, the parking frame intended by the occupant and the target parking frame may not match.

[0146] For example, suppose that the occupant sees the parking space S1 diagonally ahead and turns the vehicle 1 with the intention of parking in the parking space S1, and starts turning the steering wheel at a point P5. Here, if the parking assistance device 100 selects the parking space S2 as the target parking space based on the fact that the point P5 where the vehicle 1 turns is located in front of (the entrance of) the parking space S2, the target parking space S2 will not match the parking space S1 intended by the occupant.

[0147] In the second embodiment, the parking assistance device 100 provides assistance by notifying the driver, thereby guiding the driver to turn the steering wheel while avoiding timing when erroneous estimation is likely to occur. The second embodiment will be described below.

[0148] In the example shown in Figure 14, a parking lot is shown in which three parking spaces S4, S5, and S6 are arranged in order from the negative side of the X direction. The parking space S4 is a parking space sandwiched between the parking space lines L5 and L6, the parking space S5 is a parking space sandwiched between the parking space lines L6 and L7, and the parking space S6 is a parking space sandwiched between the parking space lines L7 and L8. In the example shown in Figure 14, the vacant parking spaces are S5 and S6, and the vehicle 1 is proceeding from the negative side to the positive side of the X direction along the passage facing the three parking spaces S4, S5, and S6.

[0149] FIG. 14 illustrates Q1 to Q5, which are the positions of the occupant (driver) at each time point when the vehicle 1 passes in front of a parking space. Note that the position of Q1 indicates the positions of the occupant and vehicle 1, but the positions of Q2 to Q5 omit vehicle 1 and only show the occupant. Q1 is, for example, the position of the occupant at the time when the detection of the parking space S5 is notified. It is advisable to notify the occupant in advance that there will be a countdown and that the occupant should turn the steering wheel during the countdown, for example, when the parking assistance device 100 starts parking space detection. In this way, the occupant can turn the steering wheel at a timing that suits their parking intention and the notification.

[0150] For example, while the occupant's position is in the range from Q1 to Q2, the parking assistance device 100 announces, "Do you want to park in the nearest parking space?" Then, since parking space S6 is farther from the occupant than parking space S5, which is closer, the occupant can determine that he or she is being asked whether to park the vehicle 1 in parking space S5. Next, while the occupant's position moves from Q2 to Q3, the parking assistance device 100 counts down "3, 2, 1." Since the occupant has previously heard, "Please turn the steering wheel during the countdown," if the occupant wants to park in parking space S5, he or she can simply turn the steering wheel accordingly. Then, the target parking position is set to parking space S5. If the occupant does not want to park in parking space S5, he or she does not need to turn the steering wheel.

[0151] When the occupant's position reaches Q3, the parking assistance device 100 announces, "Do you want to enter the next space?" The occupant knows that the next space is parking space S6 from the positional relationship between the parking space and the vehicle 1. Then, while the occupant's position moves from Q4 to Q5, the parking assistance device 100 counts down "3, 2, 1." If the occupant turns the steering wheel during this time, the target parking position is set to parking space S6.

[0152] In this way, by receiving a notification at a timing according to the relative position with respect to the parking space and turning the steering wheel in response to the notification, the target parking position can be reliably set to the parking space where the occupant intends to park.

[0153] Next, an example of the operation of the parking assistance device 100 according to the second embodiment will be described. FIGS. 15 and 16 are flowcharts showing an example of the operation of parking assistance control in the parking assistance device 100 according to the second embodiment. The flow in FIG. 16 starts when a button for starting the parking assistance device 100 is pressed. At the start, it is assumed that the vehicle 1 is traveling at low speed through an aisle in a parking lot, with its path approximately perpendicular to the detected group of white lines. The flowcharts in FIGS. 15 and 16 shown below show parts related to notification, and omit parts not related to notification. Therefore, the processes shown in FIGS. 11 to 13 in the first embodiment may be executed in parallel with the processes in the following flowcharts.

[0154] When the button for starting the parking assistance device 100 is pressed, the parking assistance device 100 starts speed control (step S201) as shown in Fig. 15. This speed control is a control for suppressing the speed of the vehicle 1 to increase the time for the occupant to judge the parking space. For example, when the vehicle speed is controlled to 3 km / h, the vehicle 1 passes in front of a parking space spaced 2.5 m apart in 3 seconds.

[0155] The parking assistance device 100 further starts parking space detection (step S202). The parking space detection is performed based on the image of the front camera. The speed control and the parking space detection are not limited to the above steps, but are continuously performed in the processing after step S202.

[0156] After step S202, the parking assistance device 100 determines whether or not a parking space has been detected (step S203). If the determination result shows that a parking space has not been detected (step S203, NO), the processing of step S203 is repeated.

[0157] On the other hand, if a parking space is detected (step S203, YES), the parking assistance device 100 calculates a time margin based on the distance to the parking space and the vehicle speed (step S204). The time margin is, for example, the time until the vehicle 1 reaches the guidance end point. The guidance end point is a position corresponding to Q3 or Q5 in FIG. 14, and is the position where the countdown by the parking assistance device 100 ends.

[0158] After step S204, the parking assistance device 100 determines whether the time margin is 5 seconds or less (step S205). If the result of the determination is that the time margin exceeds 5 seconds (step S205, NO), the process returns to step S204.

[0159] If the remaining time is 5 seconds or less (step S205, YES), the parking assistance device 100 starts outputting message A (step S206). That is, the parking assistance device 100 waits until the remaining time is 5 seconds before outputting message A. Message A may be a message including a countdown, such as "Can you enter this parking space? 3, 2, 1." In this way, the parking assistance device 100 adjusts the time so that the vehicle 1 will reach the guidance end point when the output of message A is completed.

[0160] Next, parking assistance device 100 determines whether the absolute value of the steering angle is greater than 5 degrees (step S207). Note that the processes of steps S207 and S208 are executed while message A is being output in step S206. In other words, it is determined whether the steering wheel is turned during the countdown.

[0161] As a result of the determination, if the absolute value of the steering angle is greater than 5 degrees (step S207, YES), the process proceeds to step S218. On the other hand, if the absolute value of the steering angle is 5 degrees or less (step S207, NO), parking assistance device 100 determines whether the time margin is 0 seconds or less (step S208). This time margin is the same as the number notified as message A. That is, it is determined whether the steering wheel is turned before the countdown is notified as zero. As a result of the determination, if the time margin is greater than 0 seconds (step S208, NO), the process returns to step S207. On the other hand, if the time margin is 0 seconds or less (step S208, YES), that is, if the time margin has been notified as zero, parking assistance device 100 sets the timer to 0 seconds and starts timing (step S209). The timer counts the elapsed time since starting timing.

[0162] After step S209, the parking assistance device 100 outputs a beep sound like "beep" (step S210). The beep sound is a sound to notify that the parking frame detected in step S203 is not set as the target parking frame because no steering operation was detected during the guidance period, and the beep sound continues only while the process of step S210 is being executed.

[0163] As shown in FIG. 16, after step S210, the parking assistance device 100 determines whether or not a parking space has been detected (step S211). The parking space to be determined in step S211 is not the first parking space detected in step S203, but the next parking space. After detecting a parking space in step S203, the parking assistance device 100 continues to perform parking space detection, and if the next parking space is detected, the determination in step S211 is immediately YES. If the determination result shows that a parking space has not been detected (step S211, NO), the parking assistance device 100 determines whether or not the timer is less than 1 second (step S212).

[0164] As a result of the determination, if the timer is greater than 1 second (step S212, NO), the process returns to step S203. Since step S203 is a step for detecting the first parking space, if the next parking space is not detected within 1 second, it means that it is determined that the situation is not one in which parking spaces should be detected consecutively. In other words, if the next parking space is not detected within 1 second, the next detected parking space is treated as the first parking space. On the other hand, if the timer is less than 1 second (step S212, YES), the process returns to step S210. Since step S210 is a beep sound step, a beep sound is output for an upper limit of 1 second.

[0165] Returning to the determination of step S211, if the next parking space is detected (step S211, YES), the parking assistance device 100 calculates a time margin (step S213). The time margin is, for example, the time until the vehicle 1 reaches the guidance end point.

[0166] After step S213, the parking assistance device 100 determines whether the time margin is four seconds or less (step S214). If the result of the determination is that the time margin exceeds four seconds (step S214, NO), the process returns to step S213.

[0167] On the other hand, if the remaining time is four seconds or less (step S214, YES), the parking assistance device 100 starts outputting message B (step S215). That is, it waits until the remaining time is four seconds before outputting message B. The fact that the time threshold is shorter than that of step S205 corresponds to the fact that message B is for guidance to the second parking space and is therefore shorter than message A. Message B may be, for example, a countdown message such as "Shall I enter next? 3, 2, 1." Furthermore, the parking assistance device 100 adjusts the time so that the vehicle 1 will reach the guidance end point when the output of message B is completed.

[0168] Next, parking assistance device 100 determines whether the absolute value of the steering angle is greater than 5 degrees (step S216). Note that the processes of steps S216 and S217 are executed while message B is being output. In other words, it is determined whether the steering wheel is turned during the countdown.

[0169] If the result of the determination is that the absolute value of the steering angle is 5 degrees or less (step S216, NO), parking assistance device 100 determines whether the time margin is 0 seconds or less (step S217). This time margin is the same as the number notified as message A. In other words, it is determined whether the steering wheel is turned before the countdown is notified as zero. If the result of the determination is that the time margin is greater than 0 seconds (step S217, NO), the process returns to step S216.

[0170] On the other hand, if the time margin is 0 seconds or less (step S217, YES), the process returns to step S209. In other words, if the time margin is reported as 0, a beep sound like "boo" is emitted to notify that the detected parking frame is not to be the target parking frame.

[0171] Returning to the determination of step S216, if the absolute value of the steering angle is greater than 5 degrees (step S216, YES), the process proceeds to step S218. The same applies to the case where the result of step S207 is YES. In step S218, the parking assistance device 100 starts outputting message C (step S218). Message C is a notification that the target parking frame has been determined, and may be, for example, a message such as "Automatic parking will be performed in this frame. Please park with your back to the frame." After step S218, this control ends.

[0172] According to the second embodiment configured as described above, even if there are consecutive vacant parking spaces, it is possible to accept an operation to specify a parking space so that the space in which the vehicle 1 will park is not unclear.

[0173] The flowcharts shown in Figures 15 and 16 are an example of control to guide the driver to turn the steering wheel at the timing when the target parking space can be identified, and an example of control for notification when parking in the identified parking space is shown below. The processing of the flowchart shown in Figure 17 is an example of control for notification when parking, and this may be performed after step S218 in Figure 16.

[0174] 17 starts with the assumption that the occupant has turned the steering wheel and is moving forward toward the reverse start position (turning position). To detect that the reverse start position has been reached, the parking assistance device 100 determines whether the vehicle speed is 0 (step S219). If the determination result shows that the vehicle speed is not 0 (step S219, NO), the processing of step S219 is repeated.

[0175] On the other hand, if the vehicle speed is 0 (step S219, YES), the vehicle has reached the reverse start position, so the parking assistance device 100 displays a frame image superimposed on the image of the surroundings of the vehicle 1, calculates the parking path, and outputs a message D (step S220). The message D is a request to confirm the target parking frame, and may be, for example, a message such as "If this parking frame is OK, put the gear in R and release the brake."

[0176] Next, the parking assistance device 100 determines whether or not a state in which the gear is in R and the brake is released has been detected (step S221). If the determination result shows that a state in which the gear is in R and the brake is released has not been detected (step S221, NO), the process of step S221 is repeated. That is, the parking assistance device 100 waits for the driver to confirm the target parking frame.

[0177] On the other hand, if the gear is in R and the release of the brake is detected (step S221, YES), the parking assistance device 100 outputs a message E notifying the start of automatic parking (step S222). The message E may be, for example, a message such as "Automatic parking will start. Please be careful around the vehicle."

[0178] After step S222, the parking assistance device 100 automatically drives the vehicle 1 along the parking path (step S223).

[0179] After step S223, the parking assistance device 100 determines whether or not the vehicle 1 has reached the end point (step S224). If the result of the determination is that the vehicle 1 has not reached the end point (step S224, NO), the process returns to step S223.

[0180] On the other hand, if the vehicle 1 has reached the end point (step S224, YES), the parking assistance device 100 automatically stops the vehicle 1 and outputs a message F notifying the completion of automatic parking (step S225). The message F may be, for example, a message such as "Automatic parking has ended. Please put the gear in park."

[0181] After step S225, parking assistance device 100 determines whether or not it has detected that the gear has shifted to P (step S226). If the determination result shows that it has not detected that the gear has shifted to P (step S226, NO), the processing of step S226 is repeated. On the other hand, if it has detected that the gear has shifted to P (step S226, YES), this control ends.

[0182] In order to avoid complicating the flowchart, illustrations and descriptions of exception processing, such as when the occupant does not follow the instructions of the message, are omitted. This exception processing can be implemented by supplementing it as appropriate. For example, there may be a case where the occupant turns the steering wheel while detecting a parking space, but does not intend to park in the detected parking space, but simply turns into an aisle. As an exception processing in such a case, in step S219, if the vehicle speed does not become 0 even after a predetermined time has elapsed, it may be determined that the steering operation is not an operation for specifying a parking space, and the processing may be transitioned to step S203.

[0183] In this way, when there are multiple detected parking frames, such as the parking frame S1 and the parking frame S2 in Fig. 10, the support by notification can prevent the setting error of the target parking frame. However, when there is only one detected parking frame, as in Fig. 6, the setting error of the target parking frame does not occur, so the support by notification is ineffective, and the occupant may feel annoyed.

[0184] Therefore, when only one parking space is detected, it is preferable to make the number of notifications smaller than when multiple parking spaces are detected. For example, when multiple parking spaces are detected, a notification of the parking intention and the remaining time is made, such as "Do you want to park in this parking space? 3, 2, 1." However, when only one parking space is detected, only the notification of the parking intention, such as "Do you want to park in this parking space?" may be made. Furthermore, when only one parking space is detected, no notification may be made.

[0185] (Third embodiment) A third embodiment of the present disclosure will be described below. If the parking space intended for parking can be reliably set as the target parking space, the occupant can be saved from the trouble of checking the target parking space and correcting the target parking space. Therefore, the parking assistance device 100 may have the occupant steer the vehicle to identify the parking space intended for parking, and may set the target parking space based on the observation and analysis of the steering behavior.

[0186] 18 to 23 are diagrams for explaining examples of steering behavior of the vehicle 1 in the third embodiment.

[0187] 18 and the like, a parking lot is shown in which four parking frames S7, S8, S9, and S10 are arranged in order from the negative side in the X direction. The parking frame S7 is a parking frame sandwiched between parking frame lines L9 and L10, the parking frame S8 is a parking frame sandwiched between parking frame lines L10 and L11, the parking frame S9 is a parking frame sandwiched between parking frame lines L11 and L12, and the parking frame S10 is a parking frame sandwiched between parking frame lines L12 and L13. In the example shown in FIG. 18 and the like, the vacant parking frames are S8 and S9, and the vehicle 1 proceeds along a passage facing the four parking frames S7, S8, S9, and S10 from the positive side in the X direction to the negative side, and a steering action is performed to identify the parking frame where the vehicle 1 intends to park.

[0188] The parking assistance device 100 detects a turn in the opposite direction to the turning direction at the time the vehicle stopped, as indicated by the history, and selects a parking space that matches the history based on either the position of the vehicle when the reverse turn started, or the position of the vehicle when the reverse turn ended, or the orientation of the vehicle body. A reverse turn is a steering operation that turns the front end of the vehicle body in the opposite direction from the stopping position, i.e., toward a parking space, and may also be called a reverse steering. For example, as shown in FIG. 18, when the occupant steers the front end corner of the vehicle body toward the intended parking space, the parking space to which the corner of the vehicle body is closest may be determined and the target parking space may be set. In the example of FIG. 18, the parking space S9 corresponding to the position when the vehicle 1 moved furthest in the +Y direction is set as the target parking space.

[0189] In addition, as shown in Fig. 19, for example, when the occupant turns the steering wheel toward the parking position, the parking assistance device 100 may specify a parking frame indicated by a line W extending forward from the right side of the vehicle body, that is, the side facing the parking frame, and set the target parking frame. In the example of Fig. 19, the parking frame S8 overlapping with the line W is set as the target parking frame.

[0190] In addition, the parking assistance device 100 may, for example, identify the moving direction M of the right front corner just before the vehicle 1 turns left, and set a parking frame in the moving direction M (the direction of the arrow M) as a target parking frame, as shown in Figure 20. In the example of Figure 20, a parking frame S9 that overlaps with the moving direction M is set as a target parking frame.

[0191] 21, the parking assistance device 100 may identify the direction E (the direction of the arrow E) of the right front corner as seen from the occupant just before the vehicle 1 turns left, and set a parking space in that direction E as the target parking space. In the example of FIG. 21, a parking space S8 that overlaps with the direction E is set as the target parking space. In other words, the steering to specify the intended parking space may be any of steering to the opposite side of the intended parking space at the entrance of the intended parking space, steering to temporarily turn toward the intended parking space at the entrance of the intended parking space, steering to direct the vehicle body toward the intended parking space, and steering to bring the vehicle body closer to the intended parking space. When specifying a parking space based on the orientation (direction) of the vehicle body, any of the following may be used: the direction pointed by the side of the vehicle on the opposite side to the parking position; the traveling direction of the leading corner of the vehicle on the opposite side to the parking position; or the direction in which the occupant looks at the leading corner of the vehicle on the opposite side to the parking position.

[0192] When identifying a parking space based on the direction, the parking assistance device 100 may determine the area from the entrance on the aisle side of the parking space to the center of the parking space as a determination area representing the parking space. This determination area is the range that the occupant looks at when selecting a parking space, and when the occupant operates the vehicle 1 toward this determination area, the parking assistance device 100 may set the center of each determination area, evaluate the distance between a line extending in the direction and each center, and set the parking space that is the smallest distance from the line extending in the direction among the centers of the determination areas as the target parking space.

[0193] Furthermore, the steering that identifies the parking space where the occupant intends to park may be steering based on a reverse steering operation as shown in Figures 22 and 23. A reverse steering operation is, for example, a steering action in which, before making a large turn to the left, the occupant makes a small turn to the right, widening the trajectory to the right. Reverse steering is sometimes performed habitually, and many people perform reverse steering unconsciously. Therefore, even if the occupant performs a reverse steering operation when steering the vehicle 1 for parking, there is no risk that the behavior will be perceived as unusual by those around.

[0194] The position where the reverse steering operation is performed may be specified as the position where the reverse steering operation is started. For example, as shown in FIG. 22, if the position of the front end of the vehicle body at the time when the reverse steering operation starts is closest to the center line C1 of the parking space S9 among a plurality of parking spaces, the parking assistance device 100 sets the target parking space to the parking space S9. Also, as shown in FIG. 23, if the position of the front end of the vehicle body at the time when the reverse steering operation starts is closest to the center line C2 of the parking space S8 among a plurality of parking spaces, the parking assistance device 100 sets the target parking space to the parking space S8. In other words, it is only necessary to select a parking space that matches the history based on either the position of the vehicle when the reverse steering operation starts, or the position of the vehicle or the direction of the vehicle when the reverse steering operation ends.

[0195] In addition, when detecting a plurality of parking spaces, such as the parking space S1 and the parking space S2 in FIG. 10, steering is effective to specify the parking space that is intended to be parked; however, when detecting only one parking space, such as FIG. 6, setting error of target parking space does not occur, so special steering is not required.

[0196] Therefore, the parking assistance device 100 may request steering to specify the parking space where the vehicle is intended to park only when there are multiple detected parking spaces, and may not notify the driver about steering or may only notify the driver briefly when there is only one detected parking space.

[0197] For example, if there are multiple vacant parking spaces on the right side of the vehicle 1, the parking assistance device 100 may announce a predetermined message. The predetermined message may be, for example, any of the following: "Turn the steering wheel in front of the parking space where you want to park," "Lightly turn the front of the vehicle body toward the parking space where you want to park," "Bring the corner of the vehicle body closer to the parking space where you want to park," or "Before turning the front of the vehicle body to the left, point the front of the vehicle body toward the parking space where you want to park."

[0198] Following the predetermined message, the parking assistance device 100 may announce, for example, "Automatic parking will begin when you park with your back to the parking space." Furthermore, if only one parking space is detected, the parking assistance device 100 may simply announce, "Automatic parking will begin when you park with your back to the parking space," without requiring steering to specify the parking space where the driver wants to park.

[0199] Furthermore, even when there are multiple vacant parking spaces, the parking assistance device 100 does not need to repeatedly make an announcement for each parking space. For example, if there is no steering operation that specifies the intended parking space before the parking space S1, the number of candidates for the target parking space is narrowed down to one, so there is no need to repeatedly make an announcement for the parking space S2. For example, when the parking assistance device 100 detects the second parking space, it may announce only once, "Please gently move the front end of the vehicle toward the parking space you want to park in." Furthermore, if there is no corresponding action before the first parking space S1, the parking assistance device 100 can determine the intention to park even if there is no corresponding action before the next parking space S2, so it may set the parking space S2 as the target parking space.

[0200] Furthermore, the above-described embodiments are merely examples of specific embodiments for carrying out the present disclosure, and the technical scope of the present disclosure should not be interpreted as being limited by these embodiments. In other words, the present disclosure can be carried out in various forms without departing from its gist or main features. [Industrial Applicability]

[0201] The device disclosed herein is useful as a parking assistance device and parking assistance method that enables smooth automatic parking in a short amount of time. [Explanation of symbols]

[0202] 1 vehicle 2 Cameras 10 Operating device 20 HMI device 30 Vehicle control device 40 Navigation Devices 100 Parking assistance device 101 CPU 102 ROM 103 RAM 104 I / O 105 IMP 110 Operation reception unit 120 Status Management Unit 130 Image acquisition unit 140 Parking space detection unit 150 Route Calculation Unit 160 Travel control unit 170 Storage section 180 Display image output unit

Claims

1. a parking space detection unit that detects a parking space based on a surrounding image showing the surroundings of the vehicle; a storage unit that stores a history of either an operation by a vehicle occupant or a movement of the vehicle; an instruction detection unit that detects a parking instruction from the occupant based on the history; Equipped with the instruction detection unit determines that the parking instruction has been detected when the history indicates that the vehicle has stopped after steering from a straight ahead direction and turning, and the occupant has performed a predetermined parking instruction operation, the parking instruction includes an instruction for a target parking space in which the vehicle is to be parked; The instruction detection unit selects, as a target parking frame, a parking frame that is located on the opposite side of the stopping position of the vehicle across the path of the vehicle when traveling straight, and that matches the history, from the parking frames detected by the parking frame detection unit. Parking assistance device.

2. The instruction detection unit selects a parking frame that matches the history based on the position of the vehicle or the occupant at the time when the vehicle transitions from straight driving to turning, as indicated by the history. The parking assistance device according to claim 1.

3. The instruction detection unit Detecting a turn in a direction opposite to the turning direction at the time when the vehicle stopped, as indicated by the history; the position of the vehicle when the reverse turn began; or selecting a parking space that matches the history based on either the position of the vehicle or the orientation of the vehicle body when the reverse turn is completed; The parking assistance device according to claim 1.

4. The storage unit stores either the surrounding image captured while the vehicle is traveling or information about a parking space detected while the vehicle is traveling, The instruction detection unit selects the target parking frame from information on the parking frame detected by the parking frame detection unit or information on the parking frame detected while the vehicle is traveling and stored in the storage unit, based on the surrounding image captured while the vehicle is traveling and stored in the storage unit. The parking assistance device according to claim 1.

5. The parking space detection unit A parking space detection is performed based on the surrounding image captured when the vehicle is stopped, When a parking frame is detected based on the peripheral image taken when the vehicle is stopped, the target parking frame is selected from information on the parking frame based on the peripheral image taken when the vehicle is stopped.

5. The parking assistance device according to claim 4.

6. The position of the occupant is the position of the head of the occupant.

4. The parking assistance device according to claim 2 or 3.

7. The orientation of the vehicle body is The direction in which the side of the vehicle body facing in the opposite direction points, or The direction of travel of the corner at the tip of the vehicle body on the opposite side, or The direction in which the occupant looks at the tip corner of the vehicle body on the opposite side, Either 4. The parking assistance device according to claim 3.

8. a vehicle control unit that automatically parks the vehicle after the instruction detection unit detects the parking instruction; When the vehicle control unit determines that the parking instruction has been detected, the vehicle control unit outputs a command to turn on a hazard lamp. The parking assistance device according to claim 1.

9. a notification unit that notifies an occupant of the vehicle; The notification includes a notification based on information about the parking space detected by the parking space detection unit. The parking assistance device according to claim 1.

10. The notification includes an inquiry about an intention to park in the parking space, and is notified when the parking space and the vehicle body are in a predetermined positional relationship.

10. The parking assistance device according to claim 9.

11. The notification includes notification of timing to steer the vehicle.

10. The parking assistance device according to claim 9.

12. The notification includes a notification requesting steering to specify a parking frame where parking is intended.

10. The parking assistance device according to claim 9.

13. The steering to designate the intended parking frame is Steering to turn in the opposite direction to the intended parking space at the entrance of the intended parking space, Steering that temporarily turns toward the intended parking space at the entrance of the intended parking space, Steering to point the vehicle towards the intended parking space, Steering to bring the vehicle closer to the intended parking space, Contains any of the following:

13. The parking assistance device according to claim 12.

14. When the parking frame detection unit detects one parking frame, the notification unit reduces the number of notifications compared to when the parking frame detection unit detects a plurality of parking frames. A parking assistance device according to any one of claims 9 to 13.

15. a scene determination unit that determines whether a scene is a parking scene in which the vehicle is parked; a function limiting unit that limits a function of the parking assistance device when the scene is not a parking scene; Furthermore, The function restriction performed by the function restriction unit is When the scene is not a parking scene, the parking space detection unit is restricted from detecting a parking space; If the scene is not a parking scene, limiting the storage of the history by the storage unit; At least one of The function restriction unit does not restrict the execution of detection of a white line by the parking space detection unit when the vehicle is located off a road and the vehicle speed is equal to or less than a specified value. The parking assistance device according to claim 1.

16. a scene determination unit that determines whether a scene is a parking scene in which the vehicle is parked; a function limiting unit that limits a function of the parking assistance device when the scene is not a parking scene; Furthermore, The function restriction performed by the function restriction unit is When the scene is not a parking scene, the parking space detection unit is restricted from detecting a parking space; If the scene is not a parking scene, limiting the storage of the history by the storage unit; At least one of The scene determination unit The vehicle is located off the road and the vehicle speed is equal to or less than a specified value; and a plurality of white lines are detected, the angles of which with respect to the traveling direction of the vehicle being within a predetermined range; If either or both of the above conditions are met, the scene is determined to be a parking scene. The parking assistance device according to claim 1.

17. a vehicle control unit that automatically parks the vehicle after the instruction detection unit detects the parking instruction; the instruction detection unit determines that the parking instruction has been detected when the history indicates that the vehicle has stopped after steering from a straight ahead direction and turning, and the occupant has performed a predetermined parking instruction operation, The predetermined parking instruction operation includes at least one of changing the gear position, releasing the steering wheel, releasing the brake, turning on the hazard lamp, and operating a button corresponding to a parking instruction. The parking assistance device according to claim 1.

18. a vehicle control unit that automatically parks the vehicle after the instruction detection unit detects the parking instruction; Further provided with the instruction detection unit determines that the parking instruction has been detected when the history indicates that the vehicle has stopped after steering from a straight ahead direction and turning, and the occupant has performed a predetermined parking instruction operation, a scene determination unit that determines whether a scene is a parking scene in which the vehicle is parked; a function limiting unit that limits a function of the parking assistance device when the scene is not a parking scene; Furthermore, The function restriction performed by the function restriction unit is When the scene is not a parking scene, the parking space detection unit is restricted from detecting a parking space; If the scene is not a parking scene, limiting the storage of the history by the storage unit; At least one of The parking assistance device according to claim 1.

Images