Parking aid device and method for controlling the parking aid device

Abstract

Parking aid device that assists in parking a vehicle (1A), wherein the parking aid device is characterized in that it comprises: a route generation unit (125) that generates a route on which the vehicle travels; a selection unit (127) which, based on the route, selects an image acquisition unit, which in a case where a turning maneuver of the vehicle is included in the route, is used to detect an obstacle, from among several image acquisition units (41, 43, 45, 47) which take an image of the vehicle's surroundings; and an obstacle detection unit (129) that detects an obstacle that may be touching the vehicle from an image taken by the image acquisition unit selected by the selection unit, wherein the selection unit sets reference points at preset intervals on the route, uses a vehicle attitude angle in an initial position of the route as a reference, calculates a change in attitude angle at each of the reference points, and in a case where the reference point at which the calculated change in attitude angle is greater than a threshold (K) determines that a turning operation is present in the route.

Description

BACKGROUND OF THE INVENTIONField of the invention

[0001] The present invention relates to a parking aid device and a method for controlling a parking aid device. Description of the state of the art

[0002] In the prior art, a device that assists a driver in a driving operation is known. For example, Japanese Patent Application Laid-Open No. 2015-154337 discloses a technique in which a turning amount representing a turning degree of a subject vehicle is detected, an image pickup area of ​​an image pickup device is moved to the outside of a turning direction based on the detected turning amount, and a region likely to be overlooked by the driver is supplemented with an image captured by the image pickup device.

[0003] The subject of DE 10 2017 206 310 A1 is a driver assistance system comprising a normal-operation distance sensor device for determining normal-operation distance information between a vehicle and at least one obstacle. The driver assistance system also comprises a short-range operation distance sensor device for determining short-range operation distance information between the vehicle and the at least one obstacle, which device comprises one or more transmitters, one or more receivers, and at least one distance determination unit, and uses waves of a different type or frequency to determine the short-range operation distance information than the normal-operation distance sensor device for determining the normal-operation distance information.

[0004] DE 44 25 419 C1 concerns a short-range ultrasonic distance warning system for use as a parking aid in a motor vehicle. The system is active when reverse gear is engaged and the vehicle is traveling at low speeds and is automatically deactivated at higher speeds. If reverse gear is not engaged, only the front-end monitoring system is active at low speeds, while the system is automatically deactivated again at higher speeds.

[0005] US 2016 / 0 193 998 A1 discloses a camera system that is automatically activated when a control system detects an obstacle in the vehicle's path. This helps a driver avoid the obstacle completely. SUMMARY OF THE INVENTION

[0006] The invention relates to a parking aid device having the features of claim 1 and a method for controlling a parking aid device having the features of claim 7.

[0007] However, in a case where a plurality of image pickup units are mounted on a vehicle and captured images captured by the plurality of image pickup units are analyzed, a processing load of a device may increase.

[0008] An aspect of the present invention has been made in view of the above-mentioned circumstance, and an object is to prevent an increase in a processing load while maintaining the safety of a vehicle.

[0009] To achieve the above-mentioned object, a parking assist device from one aspect of the present invention is a parking assist device that assists in parking a vehicle, and includes: a route generation unit that generates a travel route on which the vehicle travels; a selection unit that selects, based on the travel route, an image pickup unit used for detecting an obstacle from among a plurality of image pickup units that capture an image of a surrounding of the vehicle in a case where a turning operation of the vehicle is included in the travel route; and an obstacle detection unit that detects an obstacle that may contact the vehicle from an image captured by the image pickup unit selected by the selection unit.

[0010] In the parking assist device according to the invention, the selecting unit sets reference points at preset intervals on the travel route, uses an attitude angle of the vehicle at an initial position of the travel route as a reference, calculates an amount of change in the attitude angle at each of the reference points, and determines that a turning operation is included in the travel route in a case where the reference point at which the calculated amount of change in the attitude angle is larger than a threshold value.

[0011] In the foregoing parking assist device, before starting the turning operation in a case where the vehicle is in a stopped state, the obstacle detection unit may detect an obstacle that may collide with the vehicle from an image captured by the image capturing device selected by the selecting unit.

[0012] In the foregoing parking assist device, in a case where the vehicle is performing a forward travel and the turning operation is a left turn, the selecting unit may select an image pickup unit in a front area that picks up an image of a front area of ​​the vehicle and an image pickup unit in a left area of ​​the vehicle that picks up an image of a left area of ​​the vehicle from among the plurality of image pickup units.

[0013] In the foregoing parking assist device, in a case where the vehicle is performing a forward travel and the turning operation is a right turn, the selecting unit may select a front-area image pickup unit that picks up an image of a front area of ​​the vehicle and a right-area image pickup unit that picks up an image of a right area of ​​the vehicle from among the plurality of image pickup units.

[0014] In the above parking assist device, in a case where the vehicle is reversing and the turning operation is a left turn, the selecting unit may select a rear-area image pickup unit that picks up an image of a rear portion of the vehicle and a right-area image pickup unit that picks up an image of a right portion of the vehicle from among the plurality of image pickup units.

[0015] In the above parking assist device, in a case where the vehicle is reversing and the turning operation is a right turn, the selecting unit may select a rear-area image pickup unit that picks up an image of a rear area of ​​the vehicle and a left-area image pickup unit that picks up an image of a left area of ​​the vehicle from among the plurality of image pickup units.

[0016] A method for controlling a parking assist device from one aspect of the present invention is a method for controlling a parking assist device that assists in parking a vehicle, the control method comprising: a step of generating a travel route on which the vehicle travels; a step of selecting an image pickup unit used for detecting an obstacle in a case where a turning operation of the vehicle is included in the travel route, from among a plurality of image pickup units mounted on the vehicle, based on the travel route; and a step of detecting an obstacle possibly contacting the vehicle from an image picked up by the image pickup unit selected by the selecting step.

[0017] In one aspect of the present invention, an increase in a processing load can be prevented while maintaining the safety of a vehicle. BRIEF DESCRIPTION OF THE DRAWINGS Fig. 1 is a block diagram illustrating an embodiment of a subject vehicle; Fig. 2 is a diagram illustrating an example of a travel route; Fig. 3 is a diagram illustrating an example of the travel route; Fig. 4 is a diagram illustrating an example of the travel route; Fig. 5 is a diagram illustrating an image pickup range in a case of forward movement and right turn; Fig. 6 is a diagram illustrating the image pickup range in a case of forward movement and left turn; Fig. 7 is a diagram illustrating the image pickup range in a case of backward movement and right turn; Fig. 8 is a diagram illustrating the image pickup range in a case of backward movement and left turn; Fig. 9 is a flowchart illustrating an operation of a parking assist device; and Fig. 10 is a flowchart illustrating an operation of the parking assist device. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT

[0018] An embodiment of the present invention will be described below with reference to the accompanying drawings. Fig. 1 is a block diagram illustrating a configuration of a vehicle (hereinafter referred to as a subject vehicle 1A) on which a parking assistance electronic control unit (ECU) 100 corresponding to the "parking assistance device" is mounted. The parking assistance ECU 100 is a control device that assists in parking the subject vehicle 1A in a parking region. The parking assistance ECU 100 detects a parking region for parking the subject vehicle 1A based on an image captured by a camera and a detection result from a sensor such as a sonar, and generates a travel route to the detected parking region.

[0019] The subject vehicle 1A is a vehicle driven by a driver's driving operation, and is a vehicle capable of autonomous driving through computer control without the driver's driving operation. The subject vehicle 1A is, for example, a vehicle such as a four-wheeled vehicle powered by an internal combustion engine, an electric vehicle powered by an electric motor, or a hybrid vehicle that mounts an internal combustion engine and an electric motor. The subject vehicle 1A may be a vehicle other than a four-wheeled vehicle.

[0020] The parking assistance ECU 100 is connected to a CAN bus 5. In addition to the parking assistance ECU 100, a navigation device 10, sonars 20, an attitude angle sensor 30, an image pickup unit 40, an alarm 50, and a vehicle control unit 200 are connected to the CAN bus 5. These devices are connected to the CAN bus 5 as nodes, and an in-vehicle network of the subject vehicle 1A is constructed.

[0021] The navigation device 10 includes a GPS antenna and a GPS receiving unit, and receives a GPS signal transmitted from a GPS satellite. The navigation device 10 measures distances between the subject vehicle 1A and GPS satellites and change ratios of the distances with respect to a prescribed number or more of satellites based on the received GPS signals, and thereby calculates an absolute position (latitude and longitude) of the subject vehicle 1A. The navigation device 10 outputs position information indicating the calculated absolute position of the subject vehicle 1A to the parking assistance ECU 100. The navigation device 10 calculates a course indicating a traveling direction of the subject vehicle 1A and outputs course information indicating the course of the subject vehicle 1A to the parking assistance ECU 100.

[0022] The sonar 20 is configured with, for example, an ultrasonic sonar. Several sonars 20 are mounted on the vehicle 1A in question. These several sonars 20 transmit ultrasonic waves to the surroundings of the vehicle 1A in question, receive reflected waves from an obstacle, thereby detecting the presence of the obstacle and measuring the distance to the detected obstacle. The sonar 20 outputs the detected obstacle and the distance to the obstacle to the parking assistance ECU 100.

[0023] The attitude angle sensor 30 is a sensor that detects a pitch angle, a yaw angle, and a roll angle of the subject vehicle 1A, thereby detecting the attitude angle of the subject vehicle 1A. The attitude angle is a rotation angle of the vehicle that changes according to a yaw motion of the subject vehicle 1A. The attitude angle sensor 30 outputs the detected attitude angle to the parking assistance ECU 100.

[0024] The image pickup unit 40 includes a front camera 41, a right-side camera 43, a left-side camera 45, and a rear camera 47. The front camera 41, the right-side camera 43, the left-side camera 45, and the rear camera 47 generate captured images by performing image pickup at prescribed intervals and output the generated captured images to the parking assistance ECU 100. The front camera 41, the right-side camera 43, the left-side camera 45, and the rear camera 47 correspond to the "image pickup unit" of the present invention. The front camera 41 corresponds to the "front-side image pickup unit" of the present invention, and the rear camera 47 corresponds to the "rear-side image pickup unit" of the present invention.The camera 43 on the right side corresponds to the “right area image pickup unit” of the present invention, and the camera 45 on the left side corresponds to the “left area image pickup unit” of the present invention.

[0025] Each of the front camera 41, the right-side camera 43, the left-side camera 45, and the rear camera 47 includes an image sensor such as a charge-coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS), and captures an image of surroundings of the subject vehicle 1A, including a road surface, at a prescribed frame rate. Viewing angles of the front camera 41, the right-side camera 43, the left-side camera 45, and the rear camera 47 are adjusted so that these four cameras are capable of capturing images in a 360-degree range around the subject vehicle 1A as a center point.

[0026] The front camera 41 is mounted, for example, on a tip end portion of a radiator grille or the like, a rear surface of a rearview mirror in a vehicle interior, or the like, and captures an image of a front area of ​​the subject vehicle 1A, including a road surface. The front area of ​​the subject vehicle 1A represents a direction toward a windshield as viewed from the driver seated in a driver's seat.

[0027] The right-side camera 43 is mounted, for example, on a right-hand exterior mirror and captures an image of a right-hand portion of the subject vehicle 1A, including a road surface. The left-side camera 45 is mounted, for example, on a left-hand exterior mirror and captures an image of a left-hand portion of the subject vehicle 1A, including a road surface.

[0028] The rear camera 47 is mounted, for example, on a tailgate opening lever, a rear spoiler, or the like, and captures an image of a rear portion of the subject vehicle 1A, including a road surface. The rear portion of the subject vehicle 1A represents a direction opposite to the front portion of the subject vehicle 1A and toward a rear window, as viewed from the driver seated in the driver's seat.

[0029] The number of cameras included in the image pickup unit 40 and their arrangement are examples and can be changed arbitrarily. That is, the number of cameras provided to the subject vehicle 1A can be larger or smaller, as long as it is possible to capture images of a 360-degree area around the subject vehicle 1A as the center.

[0030] The alarm 50 is a device that emits a sound to the surroundings of the subject vehicle 1A and provides warnings. The alarm 50 is, for example, a loudspeaker or a horn included in the subject vehicle 1A. The alarm 50 may be a device that emits light, such as a warning lamp or light included in the subject vehicle 1A.

[0031] The vehicle control unit 200 generates control information that controls the travel of the subject vehicle 1A based on a travel route calculated by the parking assistance ECU 100. The vehicle control unit 200 controls an engine, a steering system, a transmission, and so on based on the generated control information.

[0032] The parking assistance ECU 100 is a computer device including a memory 110 and a processor 120.

[0033] The memory 110 includes volatile memory, such as random access memory (RAM), and non-volatile memory, such as flash memory. The memory 110 stores a control program 115. The control program 115 is a program executed by the processor 120. The memory 110 temporarily stores captured images generated by the front camera 41, the right-side camera 43, the left-side camera 45, and the rear camera 47 of the image pickup unit 40.

[0034] The processor 120 is an arithmetic processing device configured with a central processing unit (CPU) or a microcontroller. The processor 120 executes the control program 115 and controls each unit of the parking assist ECU 100. The processor 120 may be configured with a single processor or may be configured with multiple processors. The processor 120 may be configured with a system-on-chip (SoC) integrated with all or part of the memory 110 or other circuitry. The processor 120 may be configured with a combination of a CPU that executes a program and a digital signal processor (DSP) that performs prescribed arithmetic processing. In addition, the processor 120 may be configured such that all functions of the processor 120 are implemented in hardware, and may be configured using a programmable device.

[0035] The parking assistance ECU 100 includes, as functional blocks, a communication control unit 121, a parking region detection unit 123, a route generation unit 125, a selection unit 127, and an obstacle detection unit 129. These functional blocks denote functions performed by executing instruction sets described in the control program 115 and performing calculation of data and control by the processor 120, for the sake of simplicity, by blocks and do not denote specific application software or hardware.

[0036] The communication control unit 121 controls data communication with the vehicle control unit 200. The parking assistance ECU 100 is connected to the vehicle control unit 200 via the CAN bus 5. The communication control unit 121 receives, for example, vehicle information, such as a vehicle speed, from the vehicle control unit 200 and sends information indicating the travel route for parking the subject vehicle 1A at a target parking position to the vehicle control unit 200.

[0037] Captured images captured by the front camera 41, the right-side camera 43, the left-side camera 45, and the rear camera 47, and detection results from the sonars 20 are input to the parking region detection unit 123. The parking region detection unit 123 detects obstacles such as another vehicle, a pole, a person, and a shopping cart, or frame lines such as parking area lines, based on the captured images and detection results from the sonars 20 that are input. The parking region detection unit 123 detects a parking region as a region where the subject vehicle 1A is capable of being parked, based on the detected obstacles, frame lines, area lines, and so on.

[0038] The route generation unit 125 generates a travel route in a parking space where the subject vehicle 1A is traveling. The travel route generated by the route generation unit 125 may be a route from a current position of the subject vehicle 1A to a parking region detected by the parking region detection unit 123. In a case where a stop position where the travel of the subject vehicle 1A is temporarily stopped is included in the travel route from the current position to the parking region, the route generation unit 125 may generate a travel route to the stop position and generate a travel route from the stop position to the parking region after the subject vehicle 1A reaches the stop position.The stop position includes a reverse movement start position at which switching from forward travel to reverse travel is performed, a turning position at which the direction of travel of the subject vehicle 1A is reversed, and so on.

[0039] The selection unit 127 selects a captured image used for obstacle detection. The selection unit 127 selects a captured image used for obstacle detection from the images captured by the front camera 41, the right-side camera 43, the left-side camera 45, and the rear camera 47.

[0040] The selection unit 127 detects an obstacle in a case where the subject vehicle 1A is traveling forward by analyzing an image captured by the front camera 41. The selection unit 127 detects an obstacle in a case where the subject vehicle 1A is traveling backward by analyzing an image captured by the rear camera 47.

[0041] In a case where a turning operation is included in the travel route generated by the route generation unit 125, the selection unit 127 selects a captured image used for obstacle detection based on the travel route. The turning operation is an operation for changing the direction of the subject vehicle 1A to change a traveling direction.

[0042] Fig. 2 is a diagram illustrating the route for parking the vehicle 1A in question at the parking region. In Fig. 2, vehicles which differ from the vehicle 1A in question are provided with a reference symbol ‘1B’ in order to distinguish them from the vehicle 1A in question.

[0043] It is assumed that the vehicle in question 1A is at the Fig. 2 and the route generation is started by the route generation unit 125. A travel route generated by the route generation unit 125 is shown in Fig. 2 as a travel route 151. The selection unit 127 sets reference points at preset intervals on the travel route 151 generated by the route generation unit 125. Fig. 2 illustrates a case where the six reference points S1, S2, S3, S4, S5, and S6 are set on the travel route 151. The reference point is a point indicating a position on the travel route 151 at which an attitude angle of the subject vehicle 1A is calculated. The selection unit 127 obtains the attitude angle of the subject vehicle 1A at each of the reference points by calculation and determines whether or not a turning operation is included in the travel on the travel route 151 based on the obtained attitude angle.

[0044] The selection unit 127 sets the reference points on the travel route 151 and then calculates the attitude angle at each of the set reference points. The selection unit 127 calculates the attitude angle at each of the other reference points based on the attitude angle at the reference point S1 as a starting position of the travel route 151. The reference point S1 is the current position of the subject vehicle 1A, and the attitude angle at the reference point S1 is calculated by the attitude angle sensor 30. The selection unit 127 calculates the attitude angle at each of the reference points S2, S3, S4, S5, and S6 based on the position coordinates of each of the reference points S1, S2, S3, S4, S5, and S6 and the attitude angle at the reference point S1. In a case where the subject vehicle 1A is caused to travel along the travel route 151, the reference point S6 is a stop position at which the subject vehicle 1A stops traveling.

[0045] Next, the selection unit 127 calculates the attitude angle difference between adjacent reference points. First, the selection unit 127 calculates the attitude angle difference at the reference point S1 as the starting position and the attitude angle at the reference point S2. The selection unit 127 compares the calculated attitude angle difference with a preset threshold. Here, the attitude angle difference between the reference point S1 and the reference point S2 generated by the selection unit 127 is referred to as (α1), and the threshold is referred to as "K." The attitude angle difference (α1) corresponds to "a change in attitude angle" at the reference point S2. The selection unit 127 determines that a turn is included in the travel on the travel route 151 when the calculated attitude angle difference (α1) is greater than the threshold "K."The selection unit 127 determines that a turning operation is included in the travel on the travel route 151 in a case where the difference (α1) in attitude angle is larger than the threshold value “K”, and then does not calculate the difference in attitude angle at each of the other reference points S3, S4, S5 and S6.

[0046] The selection unit 127 calculates the attitude angle difference between the reference point S2 and the reference point S3 when the attitude angle difference (α1) is equal to or smaller than the threshold value. Here, the attitude angle difference between the reference point S2 and the reference point S3 generated by the selection unit 127 is referred to as (α2). The selection unit 127 adds the calculated attitude angle difference (α2) to the attitude angle difference (α1) and calculates an addition value (α1 + α2). The addition value (α1 + α2) corresponds to the "attitude angle change amount" at the reference point S3.

[0047] Fig. Fig. 3 is a diagram illustrating an example of another travel route 153 generated by the route generation unit 125. For example, in a case where the subject vehicle 1A is caused to travel along the route shown in Fig. 3, causes the vehicle 1A in question to make a right turn at the reference point S2 and then a left turn at the reference point S3. Thus, the value of the attitude angle difference between the reference point S2 and the reference point S3 and the value of the attitude angle difference between the reference point S3 and the reference point S4 are provided as values ​​with opposite positive and negative signs. Thus, the addition value decreases when the attitude angle difference between the reference point S3 and the reference point S4 is added to the addition value of the attitude angle difference between the reference point S1 and the reference point S2 and the attitude angle difference between the reference points S2 and S3.

[0048] Again with reference to Fig. 2, a further description is given about a determination as to whether or not a turning operation is included in the travel on the travel route 151.

[0049] The selection unit 127 compares the calculated addition value (α1 + α2) of the attitude angle differences with the threshold value "K." In a case where the addition value (α1 + α2) of the attitude angle differences is greater than the threshold value "K," the selection unit 127 determines that a turning operation is included in the travel on the travel route 153, and then does not calculate the attitude angle difference at each of the other reference points S4, S5, and S6.

[0050] In a case where the addition value (α1 + α2) of the attitude angle differences is equal to or smaller than the threshold value "K", the selection unit 127 performs processing similar to the processing performed for the reference points S1, S2, and S3 for each of the reference points S4, S5, and S6. In other words, for the reference points S4, S5, and S6, the selection unit 127 obtains the attitude angle differences (α3, α4, α5) from first preceding reference points S3, S4, and S5, respectively, and adds the obtained attitude angle differences (α3, α4, α5) to the addition values ​​(α1 + α2), (α1 + α2 + α3), and (α1 + α2 + α3 + α4) already calculated, respectively. The addition values ​​(α1 +α2 +α3), (α1 +α2 +α3 +α4) and (α1 +α2 +α3 +α4 +α5) correspond to the “change amounts in the attitude angle” at the reference points S4, S5 and S6 respectively.

[0051] Then, the selection unit 127 compares each of the obtained addition values ​​(α1 + α2 + α3), (α1 + α2 + α3 + α4), and (α1 + α2 + α3 + α4 + α5) of the attitude angle differences with the threshold value "K." The selection unit 127 determines that a turning operation is included in the travel on the travel route 151 when any of the addition values ​​(α1 + α2 + α3), (α1 + α2 + α3 + α4), and (α1 + α2 + α3 + α4 + α5) is greater than the threshold value "K." The selection unit 127 determines that no turning operation is included in the travel on the travel route 151 in a case where (α1 + α2 + α3 + α4 + α5) is equal to or less than the threshold value “K”.

[0052] Fig. 4 is a diagram illustrating a travel route for parking the subject vehicle 1A in a parking region 130.

[0053] Fig. Fig. 4 illustrates a state in which the control by the vehicle control unit 200 causes the subject vehicle 1A to travel along the travel route 151 and causes the subject vehicle 1A to move to the position S6. When the subject vehicle 1A travels to the position S6, the parking assistance ECU 100 again generates a travel route through the route generation unit 125. It is assumed that a travel route 155 generated in Fig. 4, is generated by the route generation unit 125.

[0054] When the travel route 155 is generated, the selection unit 127 sets reference points at preset intervals on the travel route 155, similarly to a case of the travel route 151. Here, it is assumed that, as in Fig. 4 illustrates the five reference points S6, S7, S8, S9 and S10 are set.

[0055] Similar to the case of the travel route 151, the selection unit 127 obtains the attitude angle differences between adjacent reference points and obtains the addition values ​​by adding the obtained attitude angle differences. The selection unit 127 compares the obtained addition values ​​with the threshold value "K" and determines that a turning operation is included in the travel on the travel route 155 when the addition value of the attitude angle difference becomes greater than the threshold value "K."

[0056] Fig. 5 is a diagram illustrating an image pickup range in a case where the subject vehicle 1A performs a forward movement and a right turn. Fig. 6 is a diagram illustrating the image pickup range in a case where the subject vehicle 1A performs a forward movement and a left turn. Fig. 7 is a diagram illustrating the image pickup range in a case where the subject vehicle 1A performs a reverse movement and a right turn. Fig. 8 is a diagram illustrating the image pickup range in a case where the subject vehicle 1A performs a reverse movement and a left turn.

[0057] In a case where the selection unit 127 determines that a turning operation is included in a case where the subject vehicle 1A is caused to travel on the travel route 151 or the travel route 155, the selection unit 127 selects a captured image used for a determination as to whether or not an obstacle that touches or collides with the subject vehicle 1A due to the turning operation is present.

[0058] As in Fig. As illustrated in FIG. 5, in a case where the travel along the travel route generated by the route generation unit 125 is forward travel and a reference point S2 positioned in a front area is positioned on a right side of the subject vehicle 1A, the selection unit 127 determines that the turning operation is a "right turn." In this case, the selection unit 127 selects images captured by the front camera 41 and the right-side camera 43 as the captured images used for obstacle detection. The right side of the subject vehicle 1A is the right side in a case where a traveling person seated in the subject vehicle 1A faces the front area of ​​the subject vehicle 1A.

[0059] As in Fig. As illustrated in Fig. 6, in a case where the travel along the travel route generated by the route generation unit 125 is forward travel and a reference point S2 positioned in the front area is positioned on a left side of the subject vehicle 1A, the selection unit 127 determines that the turning operation is a "left turn." In this case, the selection unit 127 selects images captured by the front camera 41 and the left-side camera 45 as the captured images used for obstacle detection. The left side of the subject vehicle 1A is the left side in a case where a traveling person seated in the subject vehicle 1A faces the front area of ​​the subject vehicle 1A.

[0060] As in Fig. As illustrated in Figure 7, in a case where the travel along the travel route generated by the route generation unit 125 is a reverse travel and a reference point S2 positioned in a rear area of ​​the subject vehicle 1A is positioned on the right side of the subject vehicle 1A, the selection unit 127 determines that the turning operation is a "right turn." In this case, the selection unit 127 selects images captured by the rear camera 47 and the left-side camera 45 as the captured images used for obstacle detection.

[0061] As in Fig. As illustrated in Fig. 8, in a case where the travel along the travel route generated by the route generation unit 125 is a reverse travel and a reference point S2 positioned in the rear area of ​​the subject vehicle 1A is positioned on the left side of the subject vehicle 1A, the selection unit 127 determines that the turning operation is a "left turn." In this case, the selection unit 127 selects images captured by the rear camera 47 and the right-side camera 43 as the captured images used for obstacle detection.

[0062] The obstacle detection unit 129 detects an obstacle based on images captured by the image capture unit 40 and detection results from the sonars 20.

[0063] The obstacle detection unit 129 analyzes captured images selected by the selection unit 127 and detects an obstacle that may touch or collide with the subject vehicle 1A. In a case where the subject vehicle 1A is in a stop position, such as the one shown in Fig. 2, and images captured by the front camera 41 or the rear camera 47 and the right-side camera 43 are selected by the selection unit 127, the obstacle detection unit 129 analyzes these captured images and detects an obstacle. That is, in a stopped state immediately before the subject vehicle 1A starts a straight movement and a right turn, or in a stopped state immediately before the subject vehicle 1A starts a reverse movement and a left turn, the obstacle detection unit 129 analyzes the images captured by the front camera 41 or the rear camera 47 and the right-side camera 43 and detects an obstacle.

[0064] In a case where the subject vehicle 1A is in a stopped state and images captured by the front camera 41 or the rear camera 47 and the left-side camera 45 are selected by the selection unit 127, the obstacle detection unit 129 analyzes these captured images and detects an obstacle. That is, in a stopped state immediately before the subject vehicle 1A starts a straight movement and a left turn, or in a stopped state immediately before the subject vehicle 1A starts a reverse movement and a right turn, the obstacle detection unit 129 analyzes the images captured by the front camera 41 or the rear camera 47 and the left-side camera 45 and detects an obstacle.

[0065] The obstacle detection unit 129 detects an obstacle based on the scanning results from the sonars 20. The obstacle detection unit 129 causes the sonars 20 to emit ultrasonic waves and detects an obstacle based on the reception results of the ultrasonic waves reflected from the obstacle. The obstacle detection unit 129 measures a time from the output of the ultrasonic waves by the sonars 20 to the reception of the reflected waves of the ultrasonic waves and detects the distance to the obstacle based on the measured time. When an obstacle is detected, the obstacle detection unit 129 outputs the direction in which the detected obstacle exists and distance information indicating the distance from the subject vehicle 1A to the obstacle to the vehicle control unit 200.

[0066] Next, a description will be given of an operation flow of the parking assist ECU 100 of this embodiment with reference to FIG. Fig. 9 and Fig. 10. The operation flow begins in a case where the navigation device 10 determines that the subject vehicle 1A is entering a parking space. A description will be given of a case where the subject vehicle 1A is in a state where the subject vehicle 1A is stopped at a beginning of the operation flow.

[0067] First, the parking assistance ECU 100 acquires images captured by the image acquisition unit 40 and detection results from the sonars 20 to search for a parking route (step S1). The parking assistance ECU 100 acquires captured images by reading the captured images taken by the front camera 41, the right-side camera 43, the left-side camera 45, and the rear camera 47 at prescribed intervals from the memory 110. The parking assistance ECU 100 causes the sonars 20 to begin detecting an obstacle and acquires detection results of an obstacle from the sonars 20.

[0068] The parking assistance ECU 100 calculates a travel route based on the captured images and the detection results from the sonars 20 (step S2). The travel route calculated here is a route set based on the captured images and the detection results from the sonars 20, and is a route to a vehicle stop position where the subject vehicle 1A is stopped. The "vehicle stop position" may include, for example, a parking region for parking the subject vehicle 1A. In a case where the subject vehicle 1A is caused to change direction by turning, a position where a turn is performed corresponds to the vehicle stop position.In a case where an obstacle is detected on the parking route and a new route is calculated to avoid the obstacle, a position immediately ahead of the vehicle where the obstacle can be avoided corresponds to the vehicle stopping position.

[0069] Next, the parking assist ECU 100 sets reference points on the calculated route (step S3). The reference points are set to include the current position of the subject vehicle 1A and the stop position on the route. The reference points are set at equal intervals on the route.

[0070] Next, the parking assistance ECU 100 selects the reference points in order of their proximity to the current position of the subject vehicle 1A (step S4). The parking assistance ECU 100 calculates the difference in attitude angle between the reference point selected in step S4 and the reference point selected at a previous time (step S5). In a case where the reference point selected at the previous time does not exist, the parking assistance ECU 100 selects the reference point corresponding to the current position of the subject vehicle 1A as the reference point selected at the previous time.

[0071] Next, the parking assistance ECU 100 adds the calculated attitude angle difference (step S6). That is, the parking assistance ECU 100 adds the attitude angle difference calculated at the current time to the attitude angle difference already calculated at the previous time. Next, the parking assistance ECU 100 determines whether the difference added in step S6 is greater than the threshold value “K” or not (step S7). A processing flow in a case where the difference added in step S6 is greater than the threshold value “K” (step S7: YES) will be described with reference to the Fig. 10 illustrated flowchart.

[0072] In a case where the difference added in step S6 is equal to or less than the threshold value "K" (step S7: NO), the parking assistance ECU 100 determines whether the reference point selected in step S4 is the reference point corresponding to the vehicle stop position (step S8). In a case where the reference point selected in step S4 is not the reference point corresponding to the vehicle stop position (step S8: NO), the parking assistance ECU 100 returns to a process from step S4 and repeats the processes from steps S5 to S8.

[0073] In a case where the reference point selected in step S4 is the reference point corresponding to the vehicle stop position (step S8: YES), the parking assistance ECU 100 determines that no turning operation is included in the travel route generated in step S2. In this case, the parking assistance ECU 100 selects an image captured by the front camera 41 or the rear camera 47 as a captured image to be targeted for obstacle detection (step S9). In a case where the traveling direction of the subject vehicle 1A, which is determined based on information of the travel route calculated in step S2 or information such as a gearshift position acquired as the vehicle information, corresponds to forward movement, the parking assistance ECU 100 selects an image captured by the front camera 41 as a captured image to be targeted.In a case where the traveling direction of the subject vehicle 1A, which is determined based on information of the travel route or information such as the gearshift position, corresponds to a backward movement, the parking assist ECU 100 selects an image captured by the rear camera 47 as a captured image to be targeted.

[0074] Next, the parking assistance ECU 100 outputs the travel route information calculated in step S2 to the vehicle control unit 200. The vehicle control unit 200 generates the control information that controls the travel of the subject vehicle 1A based on the travel route calculated by the parking assistance ECU 100. The vehicle control unit 200 controls the engine, steering, transmission, and so on based on the generated control information and starts the travel of the subject vehicle 1A (step S10).

[0075] When the subject vehicle 1A starts traveling, the parking assistance ECU 100 analyzes the selected captured image captured by the front camera 41 or the rear camera 47 (step S11) and determines whether or not there is a possibility that the subject vehicle 1A will hit an obstacle (step S12). In a case where there is no possibility that the subject vehicle 1A will hit an obstacle (step S12: NO), the parking assistance ECU 100 determines whether or not the subject vehicle 1A will stop at the vehicle stop position (step S14). The parking assistance ECU 100 determines whether or not the subject vehicle 1A will stop at the vehicle stop position based on the vehicle information, such as the vehicle speed, input from the vehicle control unit 200, for example.

[0076] In a case where the subject vehicle 1A does not stop at the vehicle stop position (step S14: NO), the parking assistance ECU 100 returns to step S11 and starts analyzing the next image captured in the image capture sequence (step S11). In a case where the subject vehicle 1A stops at the vehicle stop position (step S14: YES), the parking assistance ECU 100 determines whether the vehicle stop position is the parking region or not (step S15). In a case where the vehicle stop position is not the parking region (step S15: NO), the parking assistance ECU 100 returns to step S1, acquires detection results from the sonars 20 and images captured by the image capture unit 40, and calculates the travel route to the next vehicle stop position. In a case where the vehicle stop position is the parking region (step S15: YES), the parking assist ECU 100 ends this processing flow.

[0077] In a case where the parking assistance ECU 100 determines in the determination in step S12 that there is a possibility of the subject vehicle 1A hitting an obstacle (step S12: YES), the parking assistance ECU 100 stops the travel of the subject vehicle 1A and issues an alarm sound from the alarm 50 (step S13). Subsequently, the parking assistance ECU 100 returns to step S1, acquires images captured by the image acquisition unit 40 and detection results from the sonars 20 (step S1). In a case where the parking assistance ECU 100 determines that there is no possibility of collision with an obstacle, it calculates a new travel route, and the subject vehicle 1A can travel safely based on the acquired captured images and the detection results from the sonars 20 (step S2).

[0078] Next, an operation of the parking assist ECU 100 in a case where the difference in attitude angle is larger than the threshold value “K” in step S7 will be described with reference to the Fig. 10 illustrated flowchart.

[0079] In a case where the difference in attitude angle is greater than the threshold value "K" (step S7: YES), the parking assistance ECU 100 determines whether the reference point set on the travel route is set in the front of the subject vehicle 1A or not (step S16). In a case where the reference point is set in the front of the subject vehicle 1A (step S16: YES), the parking assistance ECU 100 determines that the advancing direction of the subject vehicle 1A is the "forward direction" (step S17). In a case where the reference point is set in the rear of the subject vehicle 1A (step S16: NO), the parking assistance ECU 100 determines that the advancing direction of the subject vehicle 1A is the "reverse direction" (step S18).

[0080] In a case where the parking assistance ECU 100 determines that the traveling direction of the subject vehicle 1A is the "forward direction," the parking assistance ECU 100 determines whether the reference point is set in a left front area of ​​the subject vehicle 1A or not (step S19). In a case where the reference point is set in the left front area of ​​the subject vehicle 1A (step S19: YES), the parking assistance ECU 100 determines that a turning direction of the subject vehicle 1A is the "left direction" (step S20). In a case where the parking assist ECU 100 determines that the traveling direction of the subject vehicle 1A is the "forward direction" and the turning direction is the "left direction", the parking assist ECU 100 selects images captured by the front camera 41 and the left-side camera 45 as images to be targeted for obstacle detection (step S21).

[0081] In a case where the reference point is not set in the left front area of ​​the subject vehicle 1A (step S19: NO), the parking assistance ECU 100 determines that the turning direction of the subject vehicle 1A is the "right direction" (step S22). In a case where the parking assistance ECU 100 determines that the traveling direction of the subject vehicle 1A is the "forward direction" and the turning direction is the "right direction," the parking assistance ECU 100 selects images captured by the front camera 41 and the right-side camera 43 as images to be targeted for obstacle detection (step S23).

[0082] In a case where the parking assistance ECU 100 determines in step S18 that the traveling direction of the subject vehicle 1A is the "reverse direction," the parking assistance ECU 100 determines whether or not the reference point is set in a left rear area of ​​the subject vehicle 1A (step S24). Here, "left rear area" means a left rear area as seen from the driver sitting in the driver's seat, and corresponds to a case where, as shown in Fig. 8 illustrates that the reference point S2 is positioned on the left side of the subject vehicle 1A. In a case where the reference point is set in the left rear area of ​​the subject vehicle 1A (step S24: YES), the parking assistance ECU 100 determines that the turning direction of the subject vehicle 1A is the "left direction" (step S25). In a case where the parking assistance ECU 100 determines that the traveling direction of the subject vehicle 1A is the "reverse direction" and the turning direction is the "left direction," the parking assistance ECU 100 selects images captured by the rear camera 47 and the right-side camera 43 as images to be targeted for obstacle detection (step S26).

[0083] In a case where the reference point is not set in the left rear area of ​​the subject vehicle 1A (step S24: NO), the parking assist ECU 100 determines that the turning direction of the subject vehicle 1A is the "right direction" (step S27). In a case where, as shown in Fig. 7, the reference point S2 is set on the right side of the subject vehicle 1A, the parking assistance ECU 100 determines that the turning direction of the subject vehicle 1A is the "right direction." In a case where the parking assistance ECU 100 determines that the traveling direction of the subject vehicle 1A is the "reverse direction" and the turning direction is the "right direction," the parking assistance ECU 100 selects images captured by the rear camera 47 and the left-side camera 45 as images to be targeted for obstacle detection (step S28).

[0084] Next, the parking assistance ECU 100 analyzes the captured images selected in any one of steps S21, S23, S26, and S28 (step S29) and determines whether or not an obstacle that may touch or collide with the subject vehicle 1A exists (step S30). In a case where there is no possibility that the subject vehicle 1A touches an obstacle (step S30: NO), the parking assistance ECU 100 ends analysis of the image captured by the right-side camera 43 or the left-side camera 45 (step S31) and proceeds to a process from step S9 in the Fig. 9. Specifically, in a case where the parking assist ECU 100 determines that the traveling direction of the subject vehicle 1A is the "forward direction" in the determination at step S16, the parking assist ECU 100 selects an image captured by the front camera 41 as an image to be targeted for obstacle detection (step S9). In a case where the parking assist ECU 100 determines that the traveling direction of the subject vehicle 1A is the "reverse direction" in the determination at step S16, the parking assist ECU 100 selects an image captured by the rear camera 47 as an image to be targeted for obstacle detection (step S9).

[0085] Subsequently, the parking assistance ECU 100 outputs the information of the travel route calculated in step S2 to the vehicle control unit 200. The vehicle control unit 200 generates the control information that controls the travel of the subject vehicle 1A based on the travel route calculated by the parking assistance ECU 100. The vehicle control unit 200 controls the engine, steering, transmission, and so on based on the generated control information and starts the travel of the subject vehicle 1A (step S10).

[0086] As described above, the parking assistance ECU 100 of this embodiment is a parking assistance device that assists in parking the subject vehicle 1A. The parking assistance ECU 100 includes the route generation unit 125, the selection unit 127, and the obstacle detection unit 129.

[0087] The route generation unit 125 generates a travel route on which the subject vehicle 1A travels.

[0088] In a case where a turning operation for turning the subject vehicle 1A is included in the travel route generated by the route generation unit 125, the selection unit 127 selects the camera used for detecting an obstacle from among the front camera 41, the right-side camera 43, the left-side camera 45, and the rear camera 47 that capture images of surroundings of the subject vehicle 1A based on the travel route.

[0089] The obstacle detection unit 129 detects an obstacle that may contact the subject vehicle 1A by analyzing a captured image taken by the camera selected by the selection unit 127.

[0090] Accordingly, an obstacle that interferes with the turning process can be effectively detected because, in a case where a turning process is included in the route, the camera used to detect an obstacle is selected based on the route. This reduces the processing load when analyzing a captured image, while maintaining vehicle safety.

[0091] The selection unit 127 sets the reference points at preset intervals on the travel route, uses the attitude angle of the subject vehicle 1A at the initial position of the travel route as a reference, calculates the amount of change in the attitude angle at each of the reference points, and determines, in a case where the reference point at which the calculated amount of change in the attitude angle is greater than a threshold value K, that a turning operation is included in the travel route.

[0092] Accordingly, it can be precisely determined whether a turning operation is included in a trip on the route or not.

[0093] In a case where the subject vehicle 1A is in a stopped state, the obstacle detection unit 129 detects an obstacle that may collide with the subject vehicle 1A from an image captured by the camera selected by the selection unit 127 before starting a turning operation. Accordingly, an obstacle that may collide with the subject vehicle 1A due to a turning operation can be accurately detected before starting the turning operation. This allows the subject vehicle 1A to start moving safely.

[0094] The selection unit 127 selects, in a case where the subject vehicle 1A is traveling forward and the turning operation is a left turn, the front camera 41 which captures an image of the front area of ​​the subject vehicle 1A and the left-side camera 45 which captures an image of the left area of ​​the subject vehicle 1A.

[0095] Accordingly, in a case where the vehicle 1A in question performs a forward movement and a left turn, an obstacle that may collide can be accurately detected. Thus, in a case where the vehicle 1A in question performs a forward movement and a left turn, the processing load for analyzing a captured image can be reduced, thereby maintaining safety.

[0096] The selection unit 127 selects, in a case where the subject vehicle 1A is traveling forward and the turning operation is a right turn, the front camera 41 which captures an image of the front portion of the subject vehicle 1A and the right-side camera 43 which captures an image of a right portion of the subject vehicle 1A.

[0097] Accordingly, in a case where the vehicle 1A in question performs a forward movement and a right turn, an obstacle that may collide can be accurately detected. Thus, in a case where the vehicle 1A in question performs a forward movement and a right turn, the processing load for analyzing a captured image can be reduced, thereby maintaining safety.

[0098] The selection unit 127 selects, in a case where the subject vehicle 1A is reversing and where the turning operation is a left turn, the rear camera 47 which captures an image of the rear area of ​​the subject vehicle 1A and the right-side camera 43 which captures an image of a right area of ​​the subject vehicle 1A.

[0099] Accordingly, in a case where the vehicle 1A in question is reversing and turning left, a potential collision obstacle can be accurately detected. Thus, in a case where the vehicle 1A in question is reversing and turning left, the processing load for analyzing a captured image can be reduced, while maintaining safety.

[0100] The selection unit 127 selects, in a case where the subject vehicle 1A is reversing and where the turning operation is a right turn, the rear camera 47 which captures an image of the rear area of ​​the subject vehicle 1A and the left-side camera 45 which captures an image of the left area of ​​the subject vehicle 1A.

[0101] Accordingly, in a case where the vehicle 1A in question is reversing and making a right turn, a potential collision obstacle can be accurately detected. Thus, in a case where the vehicle 1A in question is reversing and making a right turn, the processing load for analyzing a captured image can be reduced, thereby maintaining safety.

[0102] The above-described embodiment merely illustrates one aspect of the present invention as an example, and any modifications and applications are possible without departing from the scope of the gist of the present invention.

[0103] For example, Fig. 1 is a schematic diagram illustrating a configuration of the parking assistance ECU 100 for ease of understanding the invention in application by categorizing it according to main processing contents. The configuration of the parking assistance ECU 100 can be categorized into multiple configuration elements according to processing contents. The categorization can be performed such that one configuration element executes multiple processing elements.

[0104] In the above-described embodiment, a description will be given of a case where the parking assistance device is configured with an ECU that controls the travel of a vehicle; however, the parking assistance device may be configured as a portion of functions of a navigation device having a navigation function.

[0105] In a case where a method for controlling the parking assistance device of the present invention is carried out using a computer, a program to be executed by the computer may be embodied in a form of a recording medium or a transmission medium that transmits the program. As a recording medium, a magnetic or optical recording medium, or a semiconductor storage device may be used. Specifically, portable or fixed recording media such as a floppy disk, a hard disk drive (HDD), a compact disk read-only memory (CD-ROM), a DVD, a Blu-ray(R) disk, a magneto-optical disk, a flash memory, and a map recording medium may be included. The aforementioned recording medium may be a non-volatile memory device such as a RAM, a ROM, or an HDD included in the parking assistance ECU 100.

[0106] For example, processing units in the Fig. 9 and Fig. For ease of understanding, the flowcharts illustrated in Figure 10 divide the processing by the parking assistance ECU 100 according to main processing contents, and the present invention is not limited to the types of division or names of processing units. The processing by the parking assistance ECU 100 may be divided into multiple processing units according to the processing contents. The division may be performed such that one processing unit includes multiple processing elements. List of reference symbols 1A vehicle in question 10 Navigation device 20 Sonar 30 attitude angle sensor 40 Image acquisition unit 41 front camera 43 Camera on the right side 45 Camera on the left side 47 rear camera 50 Alarm 100 Parking aid ECU 100 110 storage 115 Control program 120 processor 121 Communication control unit 123 Parking region detection unit 125 Route generation unit 127 selection unit 129 Obstacle detection unit 130 parking region 151, 153, 155 route 200 vehicle control unit

Claims

[1] Parking aid device which assists in parking a vehicle (1A), wherein the parking aid device characterized by is that it includes: a route generation unit (125) that generates a travel route on which the vehicle travels; a selection unit (127) that selects, based on the travel route, an image pickup unit used for detecting an obstacle in a case where a turning operation of the vehicle is included in the travel route, from among a plurality of image pickup units (41, 43, 45, 47) that capture an image of surroundings of the vehicle; and an obstacle detection unit (129) that detects an obstacle that may touch the vehicle from an image captured by the image capturing unit selected by the selection unit, wherein the selection unit sets reference points at preset intervals on the travel route, uses an attitude angle of the vehicle at an initial position of the travel route as a reference, calculates an amount of change in the attitude angle at each of the reference points, and in a case where the reference point at which the calculated amount of change in the attitude angle is greater than a threshold value (K), determines that a turning operation is present in the travel route. [2] The parking assist device according to claim 1, wherein, before starting the turning operation, in a case where the vehicle is in a stopped state, the obstacle detection unit detects an obstacle that may collide with the vehicle from an image picked up by the image pickup unit selected by the selection unit. [3] The parking assist device according to claim 1 or 2, wherein, in a case where the vehicle is performing a forward travel and the turning operation is a left turn, the selecting unit selects an image pickup unit (41) in a front area that picks up an image of a front area of ​​the vehicle and an image pickup unit (45) in a left area that picks up an image of a left area of ​​the vehicle from among the plurality of image pickup units. [4] The parking assist device according to claim 1 or 2, wherein, in a case where the vehicle is performing a forward travel and where the turning operation is a right turn, the selecting unit selects a front-area image pickup unit (41) that picks up an image of a front portion of the vehicle and a right-area image pickup unit (43) that picks up an image of a right portion of the vehicle from among the plurality of image pickup units. [5] The parking assist device according to claim 1 or 2, wherein, in a case where the vehicle is reversing and the turning operation is a left turn, the selecting unit selects an image pickup unit (47) in a rear area that picks up an image of a rear area of ​​the vehicle and an image pickup unit (43) in a right area that picks up an image of a right area of ​​the vehicle from among the plurality of image pickup units. [6] The parking assist device according to claim 1 or 2, wherein, in a case where the vehicle is reversing and the turning operation is a right turn, the selecting unit selects an image pickup unit (47) in a rear area that picks up an image of a rear area of ​​the vehicle and an image pickup unit (45) in a left area that picks up an image of a left area of ​​the vehicle from among the plurality of image pickup units. [7] A method for controlling a parking aid device which assists in parking a vehicle (1A), the control method characterized by is that it includes: a step of generating a route on which the vehicle travels; a step of selecting an image pickup unit used for detecting an obstacle from among a plurality of image pickup units (41, 43, 45, 47) mounted on the vehicle based on the travel route in a case where a turning operation of the vehicle is included in the travel route; and a step of detecting an obstacle that may touch the vehicle from an image captured by the image capturing unit selected by the selecting step, wherein a selecting unit sets reference points at preset intervals on the travel route, uses an attitude angle of the vehicle at an initial position of the travel route as a reference, calculates an amount of change in the attitude angle at each of the reference points, and in a case where the reference point at which the calculated amount of change in the attitude angle is greater than a threshold value (K), determines that a turning operation is present in the travel route.

Images