Parking assistance method and parking assistance device

Abstract

In the present invention, a controller (70): acquires a captured image from an imaging unit (10) that captures an image of the surroundings of a host vehicle; detects parking spaces (90) from the captured image and parked vehicles (91) parked in the parking spaces (90); determines attributes of the parked vehicles (91); causes a display unit (40) to display the parked vehicles (91) and the parking spaces (90) in the surroundings of the host vehicle; and displays, superimposed on the parked vehicles (91), an attribute image (93) indicating an attribute.

Description

Parking assistance method and parking assistance device

[0001] The present invention relates to a parking assistance method and a parking assistance device.

[0002] Conventionally, there is known a device that assists a driver in parking a vehicle in a parking space within a parking lot (see, for example, Patent Document 1). The device described in Patent Document 1 quantifies the difficulty of parking in each parking space based on multiple parking-related items. Each parking space within the parking lot and the quantified difficulty score are then displayed on a display.

[0003] Japanese Patent Application Laid-Open No. 2023-71541

[0004] The device described in Patent Document 1 allows a driver to select a parking space with a low parking difficulty level and park there, thereby reducing the risk of contact with obstacles or other vehicles. However, Patent Document 1 does not take into consideration the possibility of another vehicle contacting the driver's vehicle while the vehicle is being parked.

[0005] An object of the present invention is to provide a parking assistance method and a parking assistance device that can present suitable parking spaces based on the attributes of other parked vehicles.

[0006] In the parking assistance method disclosed herein, a computer acquires an image of the surroundings of the vehicle captured by a camera, detects parking spaces in the parking lot and parked vehicles already parked in the parking spaces from the image, and determines the attributes of each detected parked vehicle. The computer then displays the detected parking spaces and parked vehicles on a display, and displays an attribute image indicating the attributes of the parked vehicles superimposed on the display.

[0007] This allows the user to check the attributes of the parked vehicles in each parking space, thereby predicting whether or not there is a possibility that a parked vehicle will come into contact with the user's own vehicle when leaving the parking space, and to select a parking space where there is a low possibility that a parked vehicle will come into contact with the user's own vehicle.

[0008] A block diagram showing a schematic configuration of a parking assistance device according to an embodiment of the present disclosure. A diagram showing a schematic configuration of a controller and a functional configuration of a processor according to this embodiment. A diagram showing an example of a predicted exit route estimated by an area estimation unit according to this embodiment. A diagram showing an example of risk assessment information according to this embodiment. A flowchart showing a parking assistance method according to this embodiment. An example of a display image to be displayed on a display unit according to this embodiment. A diagram showing an example of a display unit on which risk causes, a predicted exit route for adjacent parked vehicles, and a work area are displayed according to this embodiment.

[0009] An embodiment of the present disclosure will now be described. FIG. 1 is a block diagram showing a schematic configuration of a parking assistance device 1 according to this embodiment. The parking assistance device 1 according to this embodiment is mounted on a vehicle and assists a user (driver) in parking a vehicle in a parking space. As shown in FIG. 1 , the parking assistance device 1 according to this embodiment includes an imaging unit 10, a surrounding detection sensor 20, a position detection sensor 30, a display unit 40, an input operation unit 50, a communication unit 60, and a parking assistance controller (hereinafter simply referred to as a controller 70). Although not shown, the vehicle equipped with the parking assistance device 1 also includes a drive mechanism that transmits the driving force of a drive source such as an engine or a motor to the wheels, a steering mechanism for steering the vehicle, a braking mechanism for braking the vehicle, and a control controller for controlling these various mechanisms.

[0010] The imaging unit 10 is a camera that captures images of the areas in front, behind, and to the sides of the vehicle. For example, in an AVM (Around View Monitor), multiple cameras are provided on the front, rear, left, and right sides of the vehicle, and images (AVM images) of a 360° range around the vehicle can be captured.

[0011] The surroundings detection sensor 20 may be of various types, such as laser radar, millimeter-wave radar, ultrasonic radar, or laser range finder, or a combination of these may be used. The surroundings detection sensor 20 transmits electromagnetic waves around the vehicle and receives the electromagnetic waves reflected by obstacles (e.g., other vehicles, pedestrians, buildings, roadside installations, etc.), thereby detecting the reflection points of the electromagnetic waves on the obstacles.

[0012] The position detection sensor 30 is a sensor that detects the current position of the vehicle. An example of the position detection sensor 30 is a receiver that receives satellite signals from a Global Navigation Satellite System (GNSS) to determine the current position.

[0013] The display unit 40 is a display that displays various information and is provided, for example, on the instrument panel of the vehicle. The input operation unit 50 inputs various command data to the controller 70 when a user performs a predetermined operation. The input operation unit 50 may be, for example, a touch panel that is integrated with the display unit 40 (display), or may alternatively be configured to include operation buttons or knobs provided on the instrument panel.

[0014] The communication unit 60 is a wireless communication device that performs wireless communication with the external device 80. The communication unit 60 communicates with a predetermined external device 80 via wireless communication and transmits and receives data based on the control of the controller 70. The communication method for wireless communication with the external device 80 is not particularly limited. For example, the communication unit 60 may communicate with the external device 80 via a network line such as an Internet line. Alternatively, when a facility server installed in a facility is used as the external device 80, or when vehicle-to-vehicle communication is performed between the vehicle and another vehicle nearby, the communication unit 60 may perform communication using short-range communication such as infrared communication or Bluetooth (registered trademark), or may perform wireless communication via a wireless LAN access point.

[0015] 2 is a diagram showing a schematic configuration of the controller 70 according to this embodiment and a functional configuration of the processor 72. The controller 70 is a computer that performs parking assistance control for the vehicle. The controller 70 includes a storage unit 71 formed of a semiconductor memory or the like, and one or more processors 72 formed of an arithmetic circuit such as a CPU (Central Processing Unit).

[0016] The storage unit 71 is a memory that stores various data and programs for providing parking assistance for a vehicle. The storage unit 71 also stores risk assessment information 711 for calculating parking risk information and exit route information 712. The risk assessment information 711 and exit route information 712 will be described later.

[0017] The processor 72 realizes various functions by reading and executing programs stored in the storage unit 71. Specifically, as shown in Fig. 2, the processor 72 functions as a surrounding information acquisition unit 721, a parking state detection unit 722, a condition extraction unit 723, a vehicle attribute determination unit 724, an area estimation unit 725, a score calculation unit 726, and a display control unit 727. Note that, here, an example is shown in which the processor 72 executes a program to realize the respective functional configurations of the surrounding information acquisition unit 721, the parking state detection unit 722, the condition extraction unit 723, the vehicle attribute determination unit 724, the area estimation unit 725, the score calculation unit 726, and the display control unit 727, but some or all of these may be realized by individual hardware configurations.

[0018] The surrounding information acquisition unit 721 functions as an image acquisition unit 721A, a point cloud acquisition unit 721B, a vehicle position calculation unit 721C, and a data synchronization unit 721D. The image acquisition unit 721A acquires captured images (including captured videos) captured by the imaging unit 10. The point cloud acquisition unit 721B acquires, as point cloud data, reflection points of electromagnetic waves from objects around the vehicle detected by the surrounding detection sensor 20. The vehicle position calculation unit 721C calculates the vehicle position (vehicle position) using dead reckoning (autonomous navigation) technology based on the positioning position measured by the position detection sensor 30 and sensing data from acceleration sensors, wheel speed sensors, etc. The data synchronization unit 721D synchronizes and associates the captured images, point cloud data, and vehicle position acquired at the same time. The synchronized data may also include the date and time when these data were acquired.

[0019] The parking state detection unit 722 detects parking spaces and parked vehicles from the captured image. The detection of parking spaces and parked vehicles is achieved by detecting image feature points in the captured image and recognizing various images within the captured image. For example, AI-based image recognition can be used to identify road lines (such as white lines) indicating parking spaces, parked vehicles, obstacles, road shapes, and the like within the captured image, and the parking state detection unit 722 detects parking spaces and parked vehicles from these. Note that in this embodiment, point cloud data and the vehicle position are synchronized with the captured image. Therefore, the parking state detection unit 722 may detect parking spaces using point cloud data and the vehicle position acquired in synchronization with the captured image in addition to the captured image. The parking state detection unit 722 may also acquire facility map data for the facility's parking lot from an external device 80 (e.g., a facility server installed in the facility or a predetermined data server that publishes information on the Internet). In this case, the parking state detection unit 722 may also use the facility map data to detect parking spaces.

[0020] The condition extraction unit 723 detects, as peripheral conditions, the conditions surrounding the parking space detected by the parking state detection unit 722. Examples of peripheral conditions include the location of the parking space, the shape of the turning space, the location of the parking space, the shape of the parking space, the presence or absence of an adjacent parking space, the presence or absence of a parked vehicle in an adjacent parked vehicle, the position of a parked vehicle parked in an adjacent parked vehicle relative to the parking space, the presence or absence of other obstacles around the parking space, the location of the obstacle, the shape of the obstacle, the presence or absence of a step, the location of the step, and the height of the step. These peripheral conditions may be detected by combining image recognition using image feature points of the captured image and object detection results using point cloud data of reflection points detected by the perimeter detection sensor 20. This makes it possible to detect obstacles, steps, and the like hidden in areas of the image where there is no difference in brightness or density.

[0021] The vehicle attribute determination unit 724 determines the attributes of the detected parked vehicle. In the present disclosure, vehicle attributes include attributes related to the vehicle driver (e.g., novice driver, elderly driver, etc.), attributes of passengers getting on and off the vehicle (e.g., wheelchair, stroller, luggage, etc.), vehicle class (e.g., vehicle overall length, vehicle width, etc.), and vehicle price (e.g., luxury car). The attributes of the parked vehicle are determined using captured images, point cloud data, and vehicle-to-vehicle communication. In determining attributes based on captured images, for example, the vehicle attribute determination unit 724 can determine whether the parked vehicle is driven by a novice driver, an elderly driver, or a wheelchair-accessible vehicle such as a welfare vehicle by detecting a beginner mark, an elderly driver mark, a wheelchair mark, etc., attached to the parked vehicle from image feature points in the captured image. Furthermore, image recognition using AI can also determine the vehicle model and class of the captured parked vehicle. By determining the vehicle model, it can also determine whether the parked vehicle is a luxury car, etc. By combining image recognition using captured images with object detection using point cloud data, it is also possible to accurately determine the class of parked vehicles.

[0022] Furthermore, the vehicle attribute determination unit 724 determines the attributes of the parked vehicle by performing vehicle-to-vehicle communication between the vehicle and the parked vehicle. In this case, even if a mark such as a beginner mark, an elderly mark, or a wheelchair mark is not displayed in the captured image, the attributes can be obtained directly from the parked vehicle. Furthermore, the vehicle attribute determination unit 724 may obtain the attributes of the parked vehicle by notifying a predetermined external device 80 using the communication unit 60, for example, via the Internet. For example, when each vehicle parks, it transmits the location of the parking space and the attributes of the vehicle to the external device 80. This allows a vehicle that enters the parking lot later to obtain the attributes of the parked vehicle that was parked earlier from the external device 80.

[0023] The area estimation unit 725 estimates the predicted exit route of adjacent parked vehicles and a work area for loading and unloading cargo such as wheelchairs. FIG. 3 is a diagram showing an example of a predicted exit route estimated by the area estimation unit 725. In FIG. 3 , the dashed line indicates a predicted exit route T1 when a general vehicle exits the parking space 90, and the solid line indicates a predicted exit route T2 when a vehicle driven by a novice driver exits the parking space 90. These predicted exit routes T1 and T2 indicate the predicted range of a route that the parked vehicle may travel when exiting. As shown in FIG. 3 , the area estimation unit 725 estimates a wider area for the predicted exit route T2 of a parked vehicle with attributes of a novice driver than for the predicted exit route T1 of a general vehicle. Note that while an example of the predicted exit route T1 of a novice driver is shown here, a predicted exit route may also be estimated for a vehicle driven by an elderly driver, for example, by predicting delays in steering, accelerator, and pedal operation. Alternatively, the predicted exit route may be estimated differently depending on attributes such as vehicle class. In this embodiment, the storage unit 71 stores exit route information 712, which stores a plurality of combinations of parked vehicle attributes and predicted exit routes. Therefore, the area estimation unit 725 reads out the predicted exit route corresponding to the determined parked vehicle attributes from the exit route information 712, and applies it to the detected parking space to estimate the predicted exit route.

[0024] Furthermore, when the attributes of a specific object, such as getting on and off a wheelchair, getting on and off a stroller, or loading and unloading luggage, are recorded as vehicle attributes, the area estimation unit 725 estimates a work area for getting on and off the object. Specifically, the area estimation unit 725 estimates an area within a predetermined distance range from the side or rear of the vehicle as the work area.

[0025] The score calculation unit 726 calculates a risk score indicating the parking difficulty level and the possibility of contact with another vehicle when the vehicle parks in a parking space. The score calculation unit 726 calculates a risk score for each vacant parking space detected by the parking state detection unit 722 based on the attributes of adjacent parked vehicles, the predicted exit paths of the adjacent parked vehicles, the surrounding conditions of the parking space (surrounding conditions), etc. The risk score is calculated using the risk assessment information 711 stored in the memory unit 71.

[0026] FIG. 4 is a diagram showing an example of risk assessment information 711 stored in the storage unit 71. The risk assessment information 711 includes, for example, a plurality of pieces of item information, a judgment value for each piece of item information, and a risk score corresponding to each judgment value. The item information is information indicating items for performing risk assessment. Examples of the item information include attributes of other parked vehicles (adjacent parked vehicles) parked in other parking spaces (adjacent parking spaces) adjacent to the parking space, the positions of adjacent parked vehicles in the adjacent parking spaces, the shape of the parking space, and the conditions surrounding the parking space. The judgment value records thresholds, ranges, judgment conditions, and the like for assessing risk for each piece of item information. The risk score is a score for determining the difficulty of parking and the possibility of contact with other vehicles when parking one's own vehicle in the parking space.

[0027] For example, for the item information "attributes of adjacent parked vehicle," a judgment value "novice driver" and a risk score "R1" are recorded. This means that the risk score is R1 when an adjacent parked vehicle parked in an adjacent parking space is driven by a novice driver. Note that FIG. 4 illustrates, as examples, judgment values ​​such as "novice driver," "elderly driver," "wheelchair access vehicle," and "cargo vehicle," but other judgment values ​​such as vehicle class and luxury car may also be used. When vehicle class is used as the judgment value, threshold values ​​for vehicle width and length may be used as the judgment value. When luxury car is used as the judgment value, for example, whether the vehicle is a specific model may be used as the judgment value, and a threshold value for the general sales price of the vehicle may be used as the judgment value.

[0028] The item information "position of adjacent parked vehicle" indicates whether the adjacent parked vehicle is parked close to the adjacent parking space in the adjacent parking space, whether it is parked beyond the adjacent parking space, etc. For example, in the example of Figure 4, the judgment value "distance to boundary of parking space < A1 (m)" and the risk score "R5" indicate that the distance from the adjacent parked vehicle to the boundary of the adjacent parking space is used as the judgment value, and if the distance is less than A1 (m), the risk score is R2.

[0029] The item "predicted exit route" relates to the predicted exit route of an adjacent parked vehicle. For example, the overlap area between the predicted exit route and the parking space is used as a judgment value, and a risk score is set according to the overlap area.

[0030] The item information "Parking Space Shape" is an item related to the size of the parking space. For example, thresholds or ranges for the width and length of the parking space are recorded as judgment values, and a risk score corresponding to these conditions is set.

[0031] The item information "surrounding conditions of the parking space" includes, for example, the shape of the turning space when entering the parking space, obstacles around the parking space, steps around the parking space, traffic volume around the parking space, and the number of pedestrians. Regarding the shape of the turning space, for example, the width of the turning space parallel to the parking direction and the length of the turning space perpendicular to the parking direction are recorded as judgment values, and a corresponding risk score is set. Regarding obstacles around the parking space, for example, the number, position, shape, and other conditions of the obstacles are recorded as judgment values, and a risk score corresponding to these conditions is set. Regarding steps around the parking space, for example, the presence or absence of steps and thresholds for the step height are used as judgment values ​​to set a risk score. Traffic volume and the number of pedestrians refer to the traffic volume and number of pedestrians on roads around the parking space, and are used, for example, when a parking space in a parking lot faces a road. For example, thresholds for the number of passing vehicles per specified time and the number of pedestrians are set as judgment values ​​to set a risk score.

[0032] The score calculation unit 726 determines whether or not each item of information, such as the attributes of the detected adjacent parked vehicles, the positions of the adjacent parked vehicles, the predicted exit route, the shape of the parking space, and the surrounding conditions, satisfies the judgment value of the risk judgment information 711, and if the judgment value is satisfied, adds up the corresponding risk score.The total score for each item of information is then calculated as the final risk score for the parking space.

[0033] The display control unit 727 causes the display unit 40 to display a parking guidance screen including the detected parking spaces around the host vehicle and the parked vehicles. At this time, the display control unit 727 superimposes an attribute image indicating the attributes of each parked vehicle on each parked vehicle. Furthermore, the display control unit 727 superimposes a risk score, including the parking difficulty level and the possibility of contact with another vehicle, on vacant parking spaces where no parked vehicles are present, highlighting the displayed image. The calculated risk score may be displayed as is, or the risk score for each parking space may be displayed using an image or color corresponding to the risk score. For example, if the risk score is calculated as a number from 0 to 100, the risk score may be divided into a minimum risk level of 0 to 25, a low risk level of 25 to 50, a medium risk level of 50 to 75, and a high risk level of 75 to 100. Parking spaces with a minimum risk level are displayed with a blue frame, parking spaces with a low risk level with a green frame, parking spaces with a medium risk level with a yellow frame, and parking spaces with a high risk level with a red frame.

[0034] Furthermore, when the user operates the input operation unit 50 to input an input to select one of the parking spaces, the display control unit 727 causes the display unit 40 to display the predicted exit route of the adjacent parked vehicle. Furthermore, when an attribute of a vehicle for getting on or off is detected in the adjacent parked vehicle, the display control unit 727 causes the display unit 40 to display the work area.

[0035] [Parking Assistance Method] Next, the parking assistance method of this embodiment will be described. FIG. 5 is a flowchart illustrating the parking assistance method of this embodiment. In this embodiment, the surrounding information acquisition unit 721 drives the image capture unit 10, the surroundings detection sensor 20, and the position detection sensor 30 at a predetermined cycle to capture an image using the image capture unit 10, detect surrounding objects using the surroundings detection sensor 20 (acquire point cloud data), and perform positioning processing using the position detection sensor 30. As a result, at the predetermined cycle, the image capture unit 721A acquires captured images, the point cloud acquisition unit 721B acquires point cloud data, and the vehicle's position calculation unit 721C calculates the vehicle's position using dead reckoning (step S1). At the same time, the data synchronization unit 721D synchronizes the captured images, point cloud data, and vehicle position acquired at the same time. The acquisition of the captured images, point cloud data, and vehicle position in step S1 is subsequently performed continuously at the predetermined cycle.

[0036] Next, the parking state detection unit 722 detects parking spaces and parked vehicles already parked in the parking spaces based on the captured images periodically acquired in step S1 (step S2). The detection of parking spaces and parked vehicles may be performed by image recognition based on image feature points of the captured images, or may be performed using point cloud data, parking lot map data acquired from external device 80, and the vehicle's position in addition to the captured images.

[0037] Next, the vehicle attribute determination unit 724 determines the attributes of each detected parked vehicle (step S3). In step S3, as described above, the vehicle attribute determination unit 724 detects predetermined marks (such as a beginner mark, an elderly mark, or a wheelchair mark) on the parked vehicles in the captured image and determines their attributes. The vehicle attribute determination unit 724 also estimates the vehicle rank of the parked vehicle through image analysis of the captured image. Point cloud data may be used in addition to the captured image, in which case the vehicle rank of the parked vehicle can be estimated more accurately. Furthermore, the vehicle attribute determination unit 724 communicates with the parked vehicle via the communication unit 60 through vehicle-to-vehicle communication to directly receive attributes from the parked vehicle. In this case, although only parked vehicles capable of vehicle-to-vehicle communication can be used, the attributes of the parked vehicle can be accurately determined. The attribute determination based on the captured image and the attribute determination based on vehicle-to-vehicle communication described above may be used in combination, whereby attributes are acquired directly from the parked vehicle for parked vehicles capable of vehicle-to-vehicle communication, and attribute determination based on the captured image for parked vehicles for which vehicle-to-vehicle communication is not possible. Furthermore, the vehicle attribute determination unit 724 can estimate the vehicle type of each parked vehicle by performing image recognition processing using AI on each parked vehicle detected from the captured image. In this case, the class and price of the parked vehicle can be estimated from the estimated vehicle type.

[0038] The area estimation unit 725 also estimates a predicted exit route and a work area when the parked vehicle leaves the parking lot based on the attributes of the detected parked vehicle (step S4). In step S4, the area estimation unit 725 reads out a predicted exit route corresponding to the attributes of each parked vehicle based on the exit route information 712 recorded in the storage unit 71, as described above, and applies it to the parking space detected based on the captured image. Furthermore, if attributes of boarding and alighting objects (wheelchairs, strollers, luggage, etc.) are recorded as vehicle attributes, the area estimation unit 725 estimates a work area for boarding and alighting objects.

[0039] Furthermore, the condition extraction unit 723 extracts the surrounding conditions (surrounding conditions) for each parking space detected in step S2 (step S5). In step S5, as described above, the condition extraction unit 723 extracts the shape of the parking space, the shape of the turning space, the number of obstacles, the positions and shapes of the obstacles, the position and height of any step, etc. as surrounding conditions using image analysis based on image feature points in the captured image and object detection based on point cloud data. The order of the processing of steps S3 and S4 and the processing of step S5 may be reversed, or they may be processed simultaneously.

[0040] Next, the score calculation unit 726 uses the risk assessment information 711 stored in the memory unit 71 to calculate a risk score for each parking space based on the attributes of the parked vehicle (adjacent parked vehicle) detected in step S3, the predicted exit route and work area estimated in step S4, and the surrounding conditions extracted in step S5 (step S6).

[0041] After the above, the display control unit 727 causes the display unit 40 to display a surrounding image including parking spaces where parking is possible and parked vehicles (step S7). FIG. 6 is a diagram showing an example of a display image displayed on the display unit 40 in this embodiment. In this embodiment, the display control unit 727 causes the display unit 40 to display an overhead view 101 looking down on the host vehicle. The overhead view 101 then displays detected parked vehicles 91, and also displays attribute images 92 indicating the attributes of the parked vehicles 91 superimposed on the parked vehicles 91. The display control unit 727 also displays risk scores for vacant parking spaces 90 where no parked vehicles 91 are present in the overhead view 101. The risk scores are displayed, for example, by surrounding the parking spaces with frame images 93 and highlighting the frame images 93 in different colors according to the range of the risk score. 6 shows an example in which the risk score is displayed by the color of the frame image 93, but alternatively, an icon image corresponding to the risk score may be superimposed on the parking space 90, or the numerical value of the risk score may be displayed as is. In the present embodiment, the bird's-eye view 101 is shown as an example of a display on the display unit 40, but this is not limiting. For example, a captured image (forward image) of the vehicle viewed from the host vehicle may be displayed on the display unit 40, and the risk score of the parking space 90 and the attribute image 92 of the parked vehicle 91 may be displayed on the forward image.

[0042] Furthermore, the parking spaces 90 displayed on the display unit 40 can be selected by the user. When the user selects one of the parking spaces 90 by operating the input operation unit 50, the parking state detection unit 722 identifies the selected parking space 90 as a parking space candidate (step S8). The display control unit 727 may, for example, superimpose a selectable button on the parking space 90 to indicate that the parking space 90 is selectable. In this case, the user can select the parking space 90 by selecting the selectable button.

[0043] After step S8, the display control unit 727 displays the risk factors for the parking space candidate, the predicted exit route for the adjacent parked vehicle, and the work area for the parking space candidate (step S9). Specifically, the display control unit 727 displays the items used in calculating the risk score on the parking space 90 of the selected parking space candidate. FIG. 7 is a diagram showing an example of the display unit 40 displaying the risk factors, the predicted exit route for the adjacent parked vehicle, and the work area. In FIG. 7, the attributes of the adjacent parked vehicle are displayed as an attribute image 92 superimposed on the parked vehicle 91. Therefore, the display control unit 727 displays the risk factors 94 based on the surrounding conditions, the predicted exit route 95, and the work area 96. In the example of FIG. 7, the attributes of the parked vehicle adjacent to the parking space 90 selected by the user are a novice driver and a wheelchair-accessible vehicle. However, if the adjacent parked vehicle is a general vehicle without these attributes, the predicted exit route 95 and the work area 96 do not need to be displayed. Furthermore, the display control unit 727 highlights the risk factors 94 as shown in FIG. 7. The highlighting method is not particularly limited, and the causes of parking difficulty may be displayed in text format using a speech bubble as shown in Figure 7, or the causes of parking difficulty may be displayed using an icon image, which may be made to flash or displayed in a highlighted color.

[0044] [Effects of the Present Embodiment] The parking assistance device 1 of the present embodiment includes an imaging unit 10 (camera) that captures an image of the surroundings of the host vehicle to acquire the captured image, a display unit 40 (display) that displays information, and a controller 70 (computer). The controller 70 acquires the captured image from the imaging unit 10 by having the processor 72 read and execute a program stored in the storage unit 71. The controller 70 then acquires the captured image from the imaging unit 10, detects parking spaces 90 and parked vehicles 91 parked in the parking spaces 90 from the captured image, and determines the attributes of the parked vehicles 91. The controller 70 then displays the parking spaces 90 and parked vehicles 91 around the host vehicle on the display unit 40, and superimposes an attribute image 92 indicating the attributes of the parked vehicles 91. This allows the user (the driver of the host vehicle) to check the attributes of the parked vehicles 91 parked in each parking space 90 and predict whether or not the parked vehicles 91 may collide with the host vehicle when exiting. This not only indicates the parking difficulty level but also provides suitable parking assistance that takes into account the possibility of collision with other vehicles.

[0045] In the parking assistance device 1 of this embodiment, the vehicle attribute determination unit 724 acquires attributes from the parked vehicle 91 through vehicle-to-vehicle communication with the parked vehicle 91. This allows for highly accurate attribute determination by receiving attributes directly from parked vehicles that are capable of vehicle-to-vehicle communication.

[0046] In the parking assistance device 1 of this embodiment, the vehicle attribute determination unit 724 acquires attributes by image analysis of the parked vehicle 91 included in the captured image. This makes it possible to accurately determine the attributes of the parked vehicle 91 by reading a beginner mark, elderly mark, or wheelchair mark attached to the parked vehicle 91. Image recognition using AI may also be performed, in which case the model and class of the parked vehicle can be estimated with high accuracy.

[0047] In the parking assistance device 1 of this embodiment, the processor 72 also functions as an area estimation unit 725, which estimates a predicted exit path 95 of the parked vehicle, and the display control unit 727 displays the predicted exit path 95 on the display unit 40. This allows the user to check the predicted exit path of the parked vehicle adjacent to the parking space 90 selected as a parking space candidate, thereby confirming risks while parking the user's own vehicle and enabling the user to select a safer parking space.

[0048] In the parking assistance device 1 of this embodiment, the storage unit 71 stores exit route information 712, which records the attributes of the parked vehicle and the predicted exit route from the parking space. The area estimation unit 725 then acquires the predicted exit route corresponding to the attributes of the parked vehicle from the exit route information 712. This makes it possible to easily acquire the predicted exit route corresponding to the attributes of the parked vehicle, thereby improving processing efficiency.

[0049] In the parking assistance device 1 of this embodiment, the processor 72 also functions as a score calculation unit 726, which calculates a risk score indicating the degree of risk when parking in a parking space and during parking, based on the attributes of the parked vehicle. The display control unit 727 displays the calculated risk score on the display unit 40. This allows the user to select a parking space 90 with low risk in both respects, by referring to the risk score corresponding to the parking difficulty and the possibility of contact with another vehicle.

[0050] In the parking assistance device 1 of this embodiment, the score calculation unit 726 calculates a risk score based on the surrounding conditions (surrounding conditions) of the parking space extracted by the condition extraction unit 723. This allows the score calculation unit 726 to calculate a risk score that reflects not only the risk while the vehicle is parked, but also the parking difficulty when parking the vehicle in the parking space. Therefore, when selecting a parking space, the user can easily find a parking space with a low parking difficulty and avoid risks when parking.

[0051] In the parking assistance device 1 of this embodiment, the display control unit 727 causes the display unit 40 to highlight and display the risk causes 94 for the parking space 90 of the parking space candidate selected by the user. This allows the user to easily check the risk causes 94 and reduce risks when parking.

[0052] In this embodiment, the condition extraction unit 723 detects, as peripheral conditions, at least one of the following: the shape of the turning space around the parking space 90, the shape of the parking space 90, the position of obstacles around the parking space 90, the shape of the obstacles, the presence or absence of steps around the parking space 90, and the traffic volume and number of pedestrians around the parking space 90, and the score calculation unit 726 calculates a risk score based on these peripheral conditions. This makes it possible to calculate a risk score based on multiple conditions that increase the difficulty of parking when parking the vehicle in a parking space, and to provide parking assistance to the user based on a more accurate risk score. Furthermore, by displaying these peripheral conditions as risk causes 94, the difficulty of parking for the user can be reduced.

[0053] [Modifications] The present invention is not limited to the above-described embodiment, and includes the following modifications within the scope of achieving the object of the present invention.

[0054] [Variation 1] In the above embodiment, the area estimation unit 725 acquires a predicted exit route corresponding to the attributes of the detected parked vehicle based on the exit route information 712 stored in the storage unit 71. However, this is not limiting. For example, the external device 80 may record the vehicle attributes and the corresponding predicted exit route, and the area estimation unit 725 may receive the predicted exit route of the parked vehicle from the external device 80. For example, when each vehicle leaves a parking space, it transmits the vehicle attributes, a captured image (exit video) of the vehicle leaving the parking space, and location information of the parking space from which it left to the external device 80. Upon receiving these vehicle attributes, exit video, and parking space location information, the external device 80 averages the exit video for each attribute for each parking space to create a predicted exit route. This makes it possible to accumulate a predicted exit route for each parking space and each attribute. In this case, the area estimation unit 725 receives the corresponding predicted exit route from the external device 80 by sending a route transmission request including the attributes of the detected parked vehicle and the location information of the parking space.

[0055] In the above embodiment, the parked vehicle attributes are exemplified as predicted exit routes for novice drivers and elderly drivers. However, predicted exit routes may also be acquired based on the driver's driving history (e.g., accident history). For example, when recording the exit route information 712 in the external device 80 as described above, the external device 80 acquires the driver's driving history when receiving from each vehicle the vehicle's attributes, a captured image (exit video) of the vehicle exiting a parking space, and location information of the parking space from which the vehicle exited. Upon receiving the vehicle attributes, the exit video, and location information of the parking space, the external device 80 averages the exit video for each attribute and driving history for each parking space to create a predicted exit route. The parking assistance device 1 also acquires the driver's driving history from the parked vehicle, for example, via vehicle-to-vehicle communication. The area estimation unit 725 then receives the corresponding predicted exit route from the external device 80 by transmitting a route transmission request including the detected parked vehicle's attributes, the location information of the parking space, and the driving history of the driver of the parked vehicle. This allows the display unit 40 to display a predicted route of departure based on the attributes of the parked vehicle and the driving history of the driver.

[0056] Furthermore, when recording the exit route information 712 in the external device 80, the accumulated vehicle attributes and the exit video may be used as training data to perform machine learning of a predicted exit route for the vehicle attributes, and a model that outputs a predicted exit route in response to input of the vehicle attributes may be generated. In this case, the area estimation unit 725 may send an exit route transmission request including the attributes of the detected parked vehicle to the external device 80, so that the external device 80 can input the received parked vehicle attributes into the model and return the output predicted exit route to the parking assistance device 1 that sent the exit route transmission request.

[0057] [Variation 2] In the above embodiment, when a parking space candidate is selected by the user, the display control unit 727 displays the risk causes 94 for the selected parking space candidate. Alternatively, the risk causes 94 may be displayed for all parking spaces regardless of the user's selection of a parking space candidate. In this case, by replacing the risk causes with icon images and displaying them, the user can be informed of the causes of parking difficulty for each parking space without making the display screen difficult to view. The same applies to the predicted exit routes 95 and working areas 96 of adjacent parked vehicles adjacent to the parking space candidate. The predicted exit routes 95 and working areas 96 of all parked vehicles may be displayed regardless of the user's selection of a parking space candidate.

[0058] [Variant 3] In the above embodiment, the display control unit 727 displayed a frame image 93, risk cause 94, predicted exit route 95, and work area 96 based on the risk score of each parking space, but these are not required and do not need to be displayed.

[0059] [Variation 4] In the above embodiment, the user drives his / her own vehicle and parks it in a parking space candidate, but this is not limiting. For example, the present invention may also be applied to an autonomous vehicle that parks its own vehicle in a parking space candidate by autonomous driving. In this case, an autonomous driving controller that automatically drives the own vehicle controls the driving force of the engine or drive motor, gear control, steering control, and braking control to park the own vehicle in the parking space candidate selected by the user. In this case, as described above, by displaying the risk score of each parking space and the attributes of adjacent parked vehicles, the user can be effectively assisted in selecting a parking space candidate.

[0060] 1...parking assistance device, 10...imaging unit (camera), 20...surroundings detection sensor, 30...position detection sensor, 40...display unit (display), 50...input operation unit, 60...communication unit, 70...controller (computer), 71...memory unit, 72...processor, 90...parking space, 91...parked vehicle, 92...attribute image, 93...frame image, 94...risk cause, 95...predicted exit route, 96...work area, 101...bird's-eye view, 711...risk assessment information, 712...exit route information, 721...surroundings information acquisition unit, 721A...image acquisition unit, 721B...point cloud acquisition unit, 721C...self-position calculation unit, 722...parking condition detection unit, 723...condition extraction unit, 724...vehicle attribute determination unit, 725...area estimation unit, 726...score calculation unit, 727...display control unit.

Claims

1. A parking assistance method in which a computer assists in parking a host vehicle, the method comprising the steps of: acquiring an image from a camera that captures images of the surroundings of the host vehicle; detecting a parking space and a parked vehicle parked in the parking space from the image; determining attributes of the parked vehicle; displaying the parking space and the parked vehicle around the host vehicle on a display; and superimposing an attribute image indicating the attributes on the parked vehicle.

2. The parking assistance method according to claim 1, wherein the computer acquires the attributes from the parked vehicle through vehicle-to-vehicle communication between the host vehicle and the parked vehicle.

3. A parking assistance method according to claim 1 or claim 2, wherein the computer acquires the attributes from an image of the parked vehicle included in the captured image.

4. A parking assistance method according to any one of claims 1 to 3, wherein the computer obtains a predicted exit route for the parked vehicle based on the attributes of the parked vehicle, and displays the predicted exit route on the display.

5. The parking assistance method according to claim 4, wherein the computer acquires the predicted exit route corresponding to the attributes of the parked vehicle from a route storage unit that records the attributes and the predicted exit route.

6. A parking assistance method as described in any one of claims 1 to 5, wherein the computer calculates a risk score indicating the degree of risk when parking in the parking space and while parking, based on the attributes, and displays the risk score on the display.

7. The parking assistance method described in claim 6, wherein the computer detects the surrounding conditions of the parking space based on the captured image, and calculates the risk score based on the parking difficulty based on the surrounding conditions and the attributes.

8. The parking assistance method according to claim 7, wherein the computer displays the causes of the parking difficulty on the display in an emphasized manner.

9. The parking assistance method according to claim 7, wherein the computer detects, as the conditions around the parking space, at least one of the shape of the turning space around the parking space, the shape of the parking space, the position of obstacles around the parking space, the shape of the obstacles around the parking space, the presence or absence of steps around the parking space, and the volume of traffic and the number of pedestrians around the parking space.

10. A parking assistance device comprising: a camera that captures an image of the area around the host vehicle and acquires the captured image; a display that displays information to a user of the host vehicle; and a computer that detects parking spaces and parked vehicles in the parking spaces from the captured image, determines attributes of the parked vehicles, displays the parking spaces and parked vehicles around the host vehicle on the display, and superimposes an attribute image indicating the attributes of the parked vehicles.

Images