PARKING AID DEVICE AND PARKING AID METHOD

Description

[Field of Technology]

[0001] The present invention relates to a parking assisting device and to a parking assisting method.[Prior Art]

[0002] Conventionally there have been disclosures of technologies relating to assisting in vehicle parking.

[0003] Japanese Unexamined Patent Application Publication 2016-101778 describes a parking assisting device comprising: an imaging portion for imaging outside of a vehicle; and a controlling portion for performing image recognition of a license plate or a wheel of a parked vehicle that is parked in a parking area imaged by the imaging portion, and, based on the license plate and / or the wheel that has been recognized through image recognition, evaluating also at least whether the type of parking space in the parking area is of the perpendicular parking type or the parallel parking type.

[0004] US 2009 / 128364 describes a parking device is for supporting a parking procedure, in which sensors record a vehicle environment and an evaluation unit ascertains an alignment of the parking space with the aid of the parking space environment.

[0005] The article Nebiker Stephan et al: "Outdoor mobile mapping and AI-Based 3D object detection with low-cost RGB-D Cameras: The Use Case of On-Street Parking Statistics", Remote Sensing, vol. 13, no. 16, 5 August 2021, page 3099, XP 093035310 describes a system and approach for creating on-street parking statistics.[Summary of the Invention][Problem Solved by the Present Invention]

[0006] With the parking assisting device described in Patent Document 1, at least whether the type of parking space in a parking area is of a perpendicular parking type or of a parallel parking type is evaluated based on at least a number plate or a wheel, recognized through image recognition. Given this, if it is not possible to acquire an image of a number plate or a wheel it will not be possible to evaluate the type of parking.

[0007] The object of the present invention is to provide a parking assisting device and parking assisting method able to determine correctly the type of parking in a region wherein a vehicle is parked.[Means for Solving the Problem]

[0008] In order to achieve the object set forth above, the invention relates to a parking assisting device according to claim 1.[Effects of the Invention]

[0009] Given the parking assisting device and parking assisting method according to the present invention, the type of parking, either parallel parking, perpendicular parking, or angle parking, is evaluated, and based on the result of that evaluation, the type of parking for a region wherein a plurality of other vehicles is parked is evaluated as being the type of parking of the largest number of vehicles in the plurality of other vehicles.

[0010] Consequently, because the type of parking in a region wherein a plurality of other vehicles is parked is evaluated as being the type of parking of the greatest number of other vehicles, it is possible to determine correctly the type of parking of a region wherein a plurality of other vehicles is parked.[Brief Descriptions of the Drawings]

[0011] FIG. 1 is a diagram showing an example of a structure of an onboard device that includes a parking assisting device. FIG. 2 is a captured image showing an example of a region dividing portion and a process for a type evaluating portion. FIG. 3 is a captured image showing an example of a process for evaluating a parking type. FIG. 4 is a diagram showing the relationship between the parking type and the angle formed between the direction of travel of the vehicle and the direction of another vehicle. FIG. 5 is a captured image showing an example of a process of a type determining portion. FIG. 6 is a diagram showing an example of the process of the type determining portion. FIG. 7 is a flowchart showing an example of a process of the parking assisting device. [Forms for Carrying Out the Present Invention]

[0012] An embodiment according to the present invention will be explained below in reference to the drawings. FIG. 1 is a diagram showing an example of a structure of an onboard device 3 that includes a parking assisting device 100. The onboard device 3 is installed in a vehicle 1.

[0013] The onboard device 3 comprises a location detecting unit 10, a detecting portion 20, an operating portion 50, a displaying portion 60, and the parking assisting device 100.

[0014] The vehicle 1 corresponds to an example of a "vehicle."

[0015] The location detecting unit 10 detects the location of the vehicle 1. The location detecting unit 10 comprises a GNSS receiver for receiving a GNSS (Global Navigation Satellite System) signal and a processor for calculating the location of the vehicle 1 based on the GNSS signal received by the GNSS receiver. The GNSS receiver and processor are omitted from the drawings. The location detecting unit 10 outputs, to the parking assisting device 100, location information that indicates the location of the vehicle 1.

[0016] The detecting portion 20 comprises one or more sensors for detecting a plurality of other vehicles 2 (referencing FIG. 2) around the vehicle 1. The detecting portion 20 in the present embodiment comprises an imaging portion 30 as the sensor.

[0017] The detecting portion 20 outputs an image captured by the imaging portion 30, as detection information, to the parking assisting device 100.

[0018] In the present embodiment, the explanation will be for a case wherein the detecting portion 20 comprises a camera; however, the detecting portion 20 may comprise a radar or sonar sensor, or LiDAR, that is able to measure the distance to another vehicle 2 using radio waves, light, or the like.

[0019] The imaging portion 30 is equipped with a front camera 31 for imaging forward of the vehicle 1, a rear camera 33 for imaging rearward of the vehicle 1, a left side camera 35 for imaging toward the left side of the vehicle 1, and a right side camera 37 for imaging toward the right side of the vehicle 1.

[0020] Each of these cameras comprises an image sensor such as a CCD (Charge-coupled Device), CMOS (Complementary Metal-Oxide-Semiconductor), or the like, and a data processing circuit for generating a captured image from the image sensor.

[0021] The front camera 31 produces a captured image PF of a plurality of other vehicles 2 that are parked around the vehicle 1.

[0022] Although in the present embodiment a case is explained wherein the front camera 31 produces a captured image PF of a plurality of other vehicles 2 that are parked around the vehicle 1, the captured image PF may be generated by the imaging portion 30 instead.

[0023] The operating portion 50 receives an operation from a user that is an occupant of the vehicle 1. The user may be, for example, the driver. The operating portion 50 outputs, to the parking assisting device 100, an operating signal that corresponds to the operation that was received. The operation received by the operating portion 50 may be an operation for starting a parking type determining process, an operation for terminating a parking type determining process, or the like.

[0024] The operating portion 50 comprises, for example, a parking type determination ON switch, not shown, and a parking type determination OFF switch, not shown, where, when the parking type determination ON switch is pressed, the parking assisting device 100 receives an operation for starting the parking type determining process. Moreover, when the parking type determining process OFF switch is pressed during execution of the parking type determining process, the parking assisting device 100 receives an operation for terminating the parking type determining process.

[0025] Note that the "parking type determining process" indicates a process for the parking assisting device 100 to evaluate the parking type PK for each of the plurality of other vehicles 2, to evaluate the parking type PK of a region RN wherein the plurality of other vehicles 2 are parked.

[0026] Note that the parking type PK indicates either parallel parking PA, perpendicular parking PC, or angle parking PB.

[0027] The displaying portion 60 comprises a display panel 61 and a touch sensor 63. A liquid crystal display, an organic EL display, or the like, is used in the display panel 61.

[0028] The displaying portion 60 displays a display image on a display panel 61 based on display data inputted from the parking assisting device 100.

[0029] The touch sensor 63 uses a resistance film-type sensor or an electrostatic capacitance-type sensor. The displaying portion 60 detects, through the touch sensor 63, a touch operation by the finger of the user on the display panel 61, to generate a location signal indicating the operating location of the touch operation that has been detected. The displaying portion 60 outputs, to the parking assisting device 100, the location signal that has been generated.

[0030] The parking assisting device 100 is a computer comprising: a processor 130 such as a CPU (Central Processing Unit), an MPU (Micro Processing Unit), or the like; and a memory 110, such as a ROM (Read-Only Memory), a RAM (Random Access Memory), or the like. Moreover, the memory 110 comprises a storage device such as an HDD (Hard Disk Drive), an SSD (Solid State Drive), or the like.

[0031] The parking assisting device 100 comprises, in addition to these devices: an interface circuit for connecting to sensors, peripheral devices, and the like; an onboard network communication circuit for communicating with other onboard devices through an onboard network; and the like. The parking assisting device 100 achieves various types of functional structures through the processor 130 executing a control program PGM that is stored in the memory 110.

[0032] The parking assisting device 100 is structured from, for example, an ECU (Electronic Control Unit).

[0033] The parking assisting device 100 comprises an image acquiring portion 131, a region dividing portion 132, a vehicle detecting portion 133, a type evaluating portion 134, and a type determining portion 135. Specifically, through executing the control program PGM that is stored in the memory 110, the processor 130 functions as the image acquiring portion 131, the region dividing portion 132, the vehicle detecting portion 133, the type evaluating portion 134, and the type determining portion 135.

[0034] The image acquiring portion 131 acquires a captured image PF around the vehicle 1. The image acquiring portion 131 acquires a captured image PF produced by, for example, the front camera 31.

[0035] The region dividing portion 132 divides the captured image PF into a plurality of regions RN based on the travel direction D1 of the vehicle 1. The region dividing portion 132 extracts, from the captured image PF, a road image RD that is an image of the road traveled by the vehicle 1, and, based on the extracted road image RD, divides the captured image PF into a plurality of regions RN. In the present embodiment, a region RN is structured from a left side region RL and a right side region RR.

[0036] The process of the region dividing portion 132 will be explained further in reference to FIG. 2 and FIG. 4.

[0037] The vehicle detecting portion 133 uses the captured image PF to detect a plurality of other vehicles 2 that are parked around the vehicle 1. In the present embodiment, the vehicle detecting portion 133 detects a plurality of other vehicles 2 that are parked around the vehicle 1 through, for example, using a trained model MD1 that inputs the captured image PF and outputs the plurality of other vehicles 2. Note that the vehicle detecting portion 133 may detect a plurality of other vehicles 2 that are parked around the vehicle 1 by, for example, performing image processing on the captured image PF.

[0038] For example, the memory 110 stores the trained model MD1 that inputs a variety of captured images PF and outputs the locations or areas of the plurality of other vehicles 2. The trained model MD1 is a model that was trained through, for example, machine learning, such as deep learning, or the like. The trained model MD1 is structured from a neural network. The trained model MD1 is trained through supervised training that uses an input data set wherein various captured images PF of vehicles 2 are associated with locations of the vehicles 2 in the captured images PF or areas in the captured images PF. The locations of the vehicles 2 in the captured image PF, or the areas in the captured images PF, correspond to labels in supervised learning.

[0039] While in the present embodiment the explanation is for a case wherein the memory 110 provides a trained model MD1, there is no limitation thereto. For example, a server device that is connected so as to be able to communicate with the parking assisting device 100 may provide the trained model MD1 instead. In this case, the vehicle detecting portion 133 sends the captured image PF to the server device. The server device then finds the position or area of each of the vehicles 2 through the trained model MD2. The type evaluating portion 134 acquires the location and area of each of the vehicles 2 from the server device.

[0040] The type evaluating portion 134 evaluates the parking type PK, which indicates parallel parking PA, perpendicular parking PC, or angle parking PB, for each of the plurality of other vehicles 2. The type evaluating portion 134 detects, for example, the vehicle body direction DB for each of the plurality of other vehicles 2, and evaluates the parking type PK based on the vehicle body direction DB.

[0041] The process in the type evaluating portion 134 will be explained further in reference to FIG. 3 and FIG. 4.

[0042] Furthermore, the memory 110 stores the trained model MD2 that inputs captured images PF of each of the plurality of other vehicles 2 and outputs the vehicle body direction DB of each of the plurality of other vehicles 2. The type evaluating portion 134 may extract, from the captured image PF, captured images of each of the plurality of other vehicles 2, to find the vehicle body direction DB of each of the plurality of other vehicles 2 through inputting, into the trained model MD2, the captured image of each of the plurality of other vehicles 2.

[0043] The trained model MD2 is a model that was trained through machine learning such as, for example, deep learning. The trained model MD2 is structured from a neural network. The trained model MD2 is trained through supervised training that uses an input data set wherein various captured images PF of vehicles 2 are associated with vehicle body directions DB of the vehicles 2. The vehicle body direction DB of a vehicle 2 corresponds to a label in supervised learning.

[0044] While in the present embodiment the explanation is for a case wherein the memory 110 provides a trained model MD2, there is no limitation thereto. For example, a server device that is connected to be able to communicate with the parking assisting device 100 may provide the trained model MD2 instead. In this case, the type evaluating portion 134 sends the captured image PF of each vehicle 2 to the server device. The server device then finds the vehicle body direction DB of each of the vehicles 2 through the trained model MD2. The type evaluating portion 134 acquires the vehicle body direction DB of each of the vehicles 2 from the server device.

[0045] Based on the evaluation results by the type evaluating portion 134, the type determining portion 135 determines, as the parking type PK for the region RN in which the plurality of other vehicles 2 are parked, the parking type PK for the largest number of vehicles among the plurality of other vehicles 2. For each of a plurality of regions RN, for example, the type determining portion 135 determines, as the parking type PK, the parking type PK with the greatest number of vehicles.

[0046] The process in the type determining portion 135 will be explained further in reference to FIG. 2, FIG. 5, and FIG. 6.

[0047] The parking assisting device 100 displays, on the display panel 61, the evaluation result of the type determining portion 135, for example. The parking assisting device 100 displays, on the display panel 61, the various parking types PK for the plurality of regions RN in correspondence with a map image, for example.

[0048] The processes in the region dividing portion 132 and the type evaluating portion 134 will be explained next in reference to FIG. 2. FIG. 2 is a captured image 700 showing an example of processes in the region dividing portion 132 and the type evaluating portion 134. The captured image 700 corresponds to an example of a captured image PF.

[0049] In the captured image 700, the vehicle 1 travels on a road that corresponds to a road image RD. In FIG. 2 there is a curve toward the right with respect to the travel direction D1 of the vehicle 1. A left edge line EL indicates the location of the left edge of the road. A right edge line ER indicates the location of the right edge of the road. A driving line DL shows an example of the path traveled by the vehicle 1.

[0050] The region dividing portion 132 divides the captured image PF into a left side region RL and a right side region RR.

[0051] The left side region RL is the region to the left of the left edge line EL. The right side region RR is the region to the right of the right edge line ER. The left side region RL and right side region RR each corresponds to an example of the plurality of regions RN.

[0052] As illustrated in FIG. 2, other vehicles 2 are parked in the left side region RL. The vehicles 2 are detected by the vehicle detecting portion 133. The vehicles 2 include a first vehicle VA1, a second vehicle VA2, and a third vehicle VA3.

[0053] The type evaluating portion 134 detects the vehicle body direction DB of each of the other vehicles 2. In FIG. 2, the vehicle body direction DB of the first vehicle VA1 is indicated by the arrow. The travel direction D1 is the direction of travel of the vehicle 1 at the position PA1 opposite the first vehicle VA1 on the driving line DL. The type evaluating portion 134 evaluates the parking type PK depending on the angle θ formed between the travel direction D1 and the direction DB. In FIG. 2, the angle θ formed between the travel direction D1 and the direction DB is near zero, and thus the evaluation is that the parking type PK is parallel parking PA.

[0054] The relationship between the angle θ and the parking type PK will be explained further in reference to FIG. 4.

[0055] Although in FIG. 2, the explanation was for a case wherein the type evaluating portion 134 evaluated the parking type PK depending on the angle θ formed between the travel direction D1 and the direction DB, there is no limitation thereto. Instead, the parking type PK may be evaluated depending on the direction θ formed between the vehicle body direction DB of each other vehicle 2 and the travel direction D1 of the vehicle 1. The travel direction D1 of the vehicle 1 when the captured image 700 was captured, for example, may be used for the travel direction D1 of the vehicle 1.

[0056] An example of a process by which the type evaluating portion 134 evaluates the parking type PK of the vehicle 2 will be explained next in reference to FIG. 3. FIG. 3 is a captured image 710 showing an example of a process by which the type evaluating portion 134 evaluates the parking type PK of a vehicle 2. The captured image 710 corresponds to an example of a captured image PF.

[0057] The captured image PF includes a vehicle VB as another vehicle 2 that is parked on the road. The road image RD is an image of the road on which the vehicle VB is parked. The road extends in the direction that is perpendicular to the plane of the paper.

[0058] A horizontal axis U and a vertical axis V are noted in FIG. 3. The horizontal axis U is parallel to the horizontal direction in the captured image PF. The vertical axis V is parallel to the vertical axis in the captured image PF. The rectangular region RB indicated by the heavy lines is the rectangular region that minimizes the area that includes an image that corresponds to the vehicle VB. The four corners of the rectangular region RB are point P1, point P2, point P3, and point P4. Point P1 is the lower right corner of the rectangular region RB. Point P2 is the lower left corner of the rectangular region RB. Point P3 is the upper right corner of the rectangular region RB. Point P4 is the upper left corner of the rectangular region RB. The (U, V) coordinates of point P1, point P2, point P3, and point P4, respectively, are (P1U, P1V), (P2U, P2V), (P3U, P3V), and (P4U, P4V).

[0059] The type evaluating portion 134 evaluates that the vehicle VB is in the crosswise direction when the following Equation (1) is satisfied, evaluating that the vehicle VB is perpendicular parked PC. P 1 U − P 2 U / P 1 V − P 3 V ≥ TH 1

[0060] The left side of the equation is the ratio of the size of the rectangular region RB in the crosswise direction to the size of the rectangular region RB in the vertical direction. A first threshold value TH1 is, for example, 1.5. When Equation (2), below, is satisfied, the type evaluating portion 134 evaluates that the vehicle VB is in the lengthwise direction, and that the vehicle VB is parallel parked PA. The second threshold value TH2 is a threshold value that is less than the first threshold value TH1, at, for example, 1.2. P 1 U − P 2 U / P 1 V − P 3 V < TH 2

[0061] Moreover, when the following equation (3) is satisfied, the type evaluating portion 134 evaluates that the vehicle VB is in an angled direction, and evaluates that the vehicle VB is angle parked PB. TH 2 ≤ P 1 U − P 2 U / P 1 V − P 3 V < TH 1

[0062] In this way, because the parking type PK is determined based on the ratio of the size of the rectangular region RB in the crosswise direction to the size of the rectangular region RB in the lengthwise direction, the type determining portion 134 can determine the parking type PK through a simple process.

[0063] The process for evaluating the parking type PK will be explained next based on the vehicle body direction DB for each of the vehicles 2, in reference to FIG. 4. FIG. 4 is a diagram showing the relationship between the parking type PK and the angle θ formed between the vehicle 1 travel direction D1 and the direction DB of the vehicle 2.

[0064] As illustrated in FIG. 4, when, for example, the angle θ satisfies, for example, Equation (4) or Equation (5), the type evaluating portion 134 evaluates that the parking type PK of the vehicle 2 is parallel parking PA: − 25 ° < θ ≤ 25 ° 155 < θ ≤ 205 °

[0065] Additionally, if the angle θ satisfies, for example, Equation (6) or Equation (7), the type evaluating portion 134 evaluates that the parking type PK of the vehicle 2 is perpendicular parking PC: 55 ° < θ ≤ 115 ° 245 < θ ≤ 295 °

[0066] Additionally, if the angle θ does not satisfy any of Equation (4) through Equation (7), the type evaluating portion 134 evaluates that the parking type PK of the vehicle 2 is angle parking PB.

[0067] In this way, the type evaluating portion 134 evaluates the parking type PK depending on the angle θ formed between the travel direction D1 of the vehicle 1 and the direction DB of the vehicle 2, enabling evaluation of the parking type PK through a simple process.

[0068] Next, with reference to FIG. 5 and FIG. 6, the process of the type determining portion 135 will be explained. FIG. 5 is a captured image 720 showing an example of a process for the type determining portion 135. The captured image 720 corresponds to an example of a captured image PF.

[0069] In the captured image 720 the vehicle 1 is traveling on a road that corresponds to a road image RD. The road that corresponds to the road image RD is a road for one-way travel, and extends along the travel direction D1 of the vehicle 1. A left edge line EL indicates the location of the left edge of the road. A right edge line ER indicates the location of the right edge of the road.

[0070] The region dividing portion 132 divides the captured image PF into a left side region RL and a right side region RR.

[0071] The left side region RL is the region to the left of the left edge line EL. The right side region RR is the region on the right side of the right edge line ER. The left side region RL and right side region RR each corresponds to an example of the plurality of regions RN.

[0072] A vehicle VC1, a vehicle VC2, and a vehicle VC3 are parked in the left side region RL. The vehicle VC1, the vehicle VC2, and the vehicle VC3 correspond to an example of a plurality of other vehicles 2. A vehicle VD1 and a vehicle VD2 are parked in the right side region RR. The vehicle VD1 and the vehicle VD2 correspond to an example of a plurality of other vehicles 2.

[0073] The type evaluating portion 134 evaluates, for example, the parking type PK depending on the angle θ formed between the travel direction D1 of the vehicle 1 and the direction DB of the vehicle 2. For example, the direction DB of the vehicle VC1 is shown in FIG. 5. Because of the angle θ formed between the direction of travel D1 of the vehicle 1 and the direction DB of the vehicle VC1 is 90°, the type evaluating portion 134 evaluates that the parking type PK of the vehicle VC1 is perpendicular parking PC.

[0074] FIG. 6 is a diagram showing an example of a process of the type determining portion 135. The time TM indicates the time at which the type evaluating portion 134 evaluates the parking type PK. In FIG. 6, the type evaluating portion 134 evaluates the parking type PK four times, when the time TM is (t), (t +1), (t +2), and (t +3). The totals CM for the numbers of vehicles are shown on the right in FIG. 6.

[0075] The number of vehicles 2 evaluated by the type evaluating portion 134 as perpendicular parked PC, the number of vehicles 2 evaluated by the type evaluating portion 134 as parallel parked PA, and the number of vehicles 2 evaluated by the type evaluating portion 132evaluating portion 134 as angle parked PB are shown in FIG. 6 for each of the vehicles 2 that are parked in the left side region RL.

[0076] For example, at time TM (t), the number of vehicles 2 evaluated as perpendicular parked PC is 2, the number of vehicles 2 evaluated as parallel parked PA is 1, and the number of vehicles 2 evaluated as angle parked PB is 0. In addition, at time TM (t +1), the number of vehicles 2 evaluated as perpendicular parked PC is 1, the number of vehicles 2 evaluated as parallel parked PA is 1, and the number of vehicles 2 evaluated as angle parked PB is 1.

[0077] For the total number of vehicles CM in the left side region RL, the number of vehicles 2 evaluated as perpendicular parked PC is 6, the number of vehicles 2 evaluated as parallel parked PA is 3, and the number of vehicles 2 evaluated as angle parked PB is 1. Because the number of vehicles 2 evaluated as perpendicular parked PC is the greatest, the type determining portion 135 determines that the parking type PK of the left side region RL is perpendicular parking PC.

[0078] Moreover, for the total number of vehicles CM in the right side region RR, the number of vehicles 2 evaluated as perpendicular parked PC is 0, the number of vehicles 2 evaluated as parallel parked PA is 8, and the number of vehicles 2 evaluated as angle parked PB is 0. Because the number of vehicles 2 evaluated as parallel parked PA is the greatest, the type determining portion 135 determines that the parking type PK of the left side region RL is parallel parking PA.

[0079] In this way, for each of the plurality of regions RN, the type determining portion 135 determines, as the parking type PK, the parking type PK with the greatest number of vehicles for each of the plurality of regions RN, enabling the parking type PK to be determined correctly for each of the plurality of regions RN.

[0080] Note that the times between each of the four times (t), (t +1) (t +2), and (t +3) that are the times TM are, for example, 0.5 second intervals. In the present embodiment, a case was explained wherein the type determining portion 135 calculates, at each prescribed time, the number of vehicles 2 evaluated as perpendicular parked PC, the number of vehicles 2 evaluated as parallel parked PA, and the number of vehicles 2 evaluated as angle parked PB; however there is no limitation thereto. For example, the type determining portion 135 may instead calculate the number of vehicles 2 evaluated as perpendicular parked PC, the number of vehicles 2 evaluated as parallel parked PA, and the number of vehicles 2 evaluated as angle parked PB with each passing prescribed distance of travel, such as 3 m, of the vehicle 1.

[0081] Although, in the present embodiment, the type determining portion 135 determines the parking type PK for each of the plurality of regions RN based on the total numbers CM of the four evaluation results by the type evaluating portion 134, there is no limitation thereto. The type determining portion 135 may determine the parking types PK for each of the plurality of regions RN based on the total numbers CM that are the evaluation results for two times or three times by the type evaluating portion 134. Moreover, the type determining portion 135 may determine the parking type PK for each of the plurality of regions RN based on the total numbers CM of five or more evaluation results by the type evaluating portion 134. Moreover, the type determining portion 135 may determine the parking types PK for each of the plurality of regions RN based on the result of a single evaluation by the type evaluating portion 134.

[0082] FIG. 7 will be referenced next to explain an example of a process of the parking assisting device 100. FIG. 7 is a flowchart showing an example of a process of the parking assisting device 100.

[0083] As illustrated in FIG. 7, first, in Step S101, the image acquiring portion 131 acquires a captured image PF around the vehicle 1.

[0084] Next, in Step S103, the region dividing portion 132 divides the captured image PF into a plurality of regions RN.

[0085] Next, in Step S105, the vehicle detecting portion 133 detects the pluralities of other vehicles 2 in each of the plurality of regions RN.

[0086] Next, in Step S107, the type evaluating portion 134 detects the directions DB of the vehicle body for each of the plurality of other vehicles 2 in each of the plurality of regions RN.

[0087] Next, in Step S109, the type evaluating portion 134 evaluates the parking type PK for each of the plurality of other vehicles 2 in each of the plurality of regions RN. The type evaluating portion 134 evaluates the parking type PK for each of the plurality of other vehicles 2 depending on the angle θ formed between the travel direction D1 of the vehicle 1 and the direction DB of the vehicle body of each of the plurality of other vehicles 2.

[0088] Following this, in Step S111, the type determining portion 135 calculates the numbers of vehicles 2 for each of the parking types PK in each of the plurality of regions RN. The type determining portion 135, for example, calculates the number of vehicles 2 evaluated as perpendicular parked PC, the number of vehicles 2 evaluated as parallel parked PA, and the number of vehicles 2 evaluated as angle parked PB in each of the plurality of regions RN.

[0089] Following this, in Step S113, the type determining portion 135 determines the parking type PK in each of the plurality of regions RN. The type determining portion 135, for example, determines, as the parking type PK for each of the plurality of regions RN, the parking type PK that has the greatest number of vehicles, for each of the plurality of regions RN. In the present embodiment, the plurality of regions RN corresponds to the left side region RL and the right side region RR.

[0090] Next, in Step S115, based on an operation by the user on the operating portion 50 the parking assisting device 100 evaluates whether or not to terminate the parking type determining process.

[0091] If the parking assisting device 100 evaluates that the parking type determining process is to be terminated (Step S115: YES), the process is terminated. If the parking assisting device 100 evaluates that the parking type determining process is not to be terminated (Step S115: NO), processing returns to Step S111.

[0092] Step S101 corresponds to an example of an "image acquiring step."

[0093] Step S103 corresponds to an example of a "vehicle detecting step."

[0094] Step S109 corresponds to an example of a "type evaluating step."

[0095] Step S113 corresponds to an example of a "type determining step."

[0096] In the above, as explained in reference to FIG. 1 through FIG. 6, the parking assisting device 100 according to the present embodiment comprises: an image acquiring portion 131 for acquiring a captured image PF around the vehicle 1; a vehicle detecting portion 133 for using the captured image PF to detect a plurality of other vehicles 2, parked around the vehicle 1; a type evaluating portion 134 for evaluating the parking type PK, indicating parallel parking PA, perpendicular parking PC, or angle parking PB, for each of the plurality of other vehicles 2; and a type determining portion 135 for determining, as the parking type PK for a region RN in which a plurality of other vehicles 2 are parked, the parking type PK with the greatest number of vehicles, of the plurality of other vehicles 2, based on the evaluation result by the type evaluating portion 134.

[0097] That is, the parking type PK, indicating parallel parking PA, perpendicular parking PC, or angle parking PB, is evaluated for each of the plurality of other vehicles 2, and the parking type PK with the greatest number of vehicles, of the plurality of other vehicles 2, is determined as the parking type PK for the region RN wherein the plurality of other vehicles 2 are parked.

[0098] The result is that the parking type PK for the region RN wherein the plurality of other vehicles 2 are parked can be determined correctly.

[0099] Moreover, the parking assisting device 100 comprises a region dividing portion 132 for dividing the captured image PF into a plurality of regions RN based on the direction of travel D1 of the vehicle 1, and the type determining portion 135 determines the parking type PK with the greatest number of vehicles, for each of the plurality of regions RN, as the parking type PK for each of the plurality of regions RN.

[0100] The result is that the parking type PK can be determined correctly for each of the plurality of regions RN.

[0101] Moreover, in the parking assisting device 100, the region dividing portion 132 extracts, from the captured image PF, a road image RD that is an image of the road traveled by the vehicle 1, to divide the captured image PF into a plurality of regions RN based on the road image RD.

[0102] The result is that the captured image PF can be divided correctly into a plurality of regions RN.

[0103] Moreover, in the parking assisting device 100, the type evaluating portion 134 detects the direction DB of the vehicle body for each of the plurality of other vehicles 2, and evaluates the parking type PK based on the direction DB of the vehicle body. For example, as explained in reference to FIG. 4, the type evaluating portion 134 evaluates the parking type PK based on the angle θ formed between the vehicle body direction DB and the direction of travel D1 of the vehicle 1. The result is that the parking types PK can be evaluated correctly for each of the plurality of other vehicles 2.

[0104] Moreover, the parking assisting method according to the present embodiment includes: an image acquiring step for acquiring a captured image PF around the vehicle 1; a vehicle detecting step for using the captured image PF to detect a plurality of other vehicles 2 parked around the vehicle 1; a type evaluating step for evaluating the parking type PK, indicating either parallel parking PA, perpendicular parking PC, or angle parking PB, for each of the plurality of other vehicles 2; and a type determining step for determining, as the parking type PK for a region RN wherein the plurality of other vehicles 2 are parked, the parking type PK with the greatest number of vehicles, of the plurality of other vehicles 2, based on the evaluation result in the type evaluating step. As a result, the parking assisting method according to the present embodiment has the same effects as those of the parking assisting device 100 of the present embodiment.

[0105] The embodiment set forth above is no more than an illustration of one form of the present invention.

[0106] For example, for ease in understanding the present invention, FIG. 1 is a diagram wherein the structural elements are shown partitioned into the main processing details, and the structural elements may be partitioned into more structural elements depending on the processing details. Moreover, the partitioning may be such that more processes are carried out by a single structural element.

[0107] Moreover, the processes in any of the structural elements may be executed in a single hardware or executed by a plurality of hardware.

[0108] Moreover, the processes of each structural element may be achieved by a single program, or by a plurality of programs.

[0109] Moreover, in FIG. 1 the parking assisting device 100 may be provided integrated with the location detecting unit 10, the operating portion 50, and / or the displaying portion 60.

[0110] Moreover, while in the present embodiment the parking assisting device 100 comprises an image acquiring portion 131, a region dividing portion 132, a vehicle detecting portion 133, a type evaluating portion 134, and a type determining portion 135, there is no limitation thereto. A server device that can connect to the parking assisting device 100 through a network, such as the Internet, so as to enable communication may comprise the region dividing portion 132, the vehicle detecting portion 133, the type evaluating portion 134, and / or the type determining portion 135. The server device may, for example, comprise the region dividing portion 132, the vehicle detecting portion 133, the type evaluating portion 134, and the type determining portion 135.

[0111] Additionally, while in the present embodiment the explanation was for a case wherein the memory 110 stores the trained model MD1 and be trained model MD2, there is no limitation thereto. For example, a server device that is connected to the parking assisting device 100 so as to enable communication may store the trained model MD1 and / or trained model MD2.

[0112] For example, if the server device stores the trained model MD1, the vehicle detecting portion 133 transmits, to the server device, a captured image PF for each of the plurality of other vehicles 2. Given this, the server device, through inputting the captured image PF into the trained model MD1, outputs the locations of the vehicles 2 or the areas thereof in the images. The vehicle detecting portion 133 acquires, from the server device, the positions in the images, or areas in the images, of the vehicles 2.

[0113] Moreover, if, for example, the server device stores the trained model MD2, the type evaluating portion 134 transmits the captured image PF of each of the plurality of other vehicles 2 to the server device. The server device, through inputting the captured image PF of each of the plurality of other vehicles 2 into the trained model MD2, outputs the direction DB of the vehicle body of each of the vehicles 2. The type evaluating portion 134 acquires the direction DB of the vehicle body of each of the vehicles 2 from the server device.

[0114] Additionally, if the parking assisting method according to the present invention is achieved using a computer, it may be structured in the form of a medium on which is recorded a control program PGM that is executed by a computer, or a transmission medium for transmitting the control program PGM.

[0115] The recording medium may use a magnetic or optical recording medium, or a semiconductor memory device. Specifically, it may be a fixed recording medium or a portable recording medium such as a flexible disk, an HDD, a CD-ROM (Compact Disk Read-Only Memory), a DVD, a Blu-ray ®< disc, a magnetooptical disc, a flash memory, a card-type recording medium, or the like. Additionally, the recording medium may be a non-volatile storage device, such as a RAM, ROM, HDD, or the like, provided in the parking assisting device 100.

[0116] The control program PGM may be downloaded by the parking assisting device 100 from a server device that is connected to the parking assisting device 100 through a network so as to enable communication.

[0117] Additionally, for example, the processing units in the flowchart shown in FIG. 7 are divided depending on main process content, for ease in understanding the processing of the parking assisting device 100; however the present invention is not limited by the way in which the processing units are divided, nor by the names thereof. The processes of the parking assisting device 100 may be divided into a greater number of processing units depending on the process details. Moreover, in the processes of the parking assisting unit 100, the divisions may be such that a greater number of processes are included in a single processing unit.[Explanation of Reference Symbols]

[0118] 1: Vehicle 2: Other Vehicle 31: Front Camera 100: Parking Assisting Device 131: Image Acquiring Portion 133: Vehicle Detecting Portion 134: Type Evaluating Portion 135: Type Determining Portion PA: Parallel Parking PB: Angle Parking PC: Perpendicular Parking PF: Captured Image PK: Parking Type

Claims

1. A parking assisting device (100) comprising: an image acquiring portion (131) for acquiring a captured image (PF) around a vehicle (1); a vehicle detecting portion (133) for using the captured image (PF) to detect a plurality of other vehicles (2) that are parked around the vehicle (1); a type evaluating portion (134) for evaluating a parking type (PK), indicating parallel parking (PA), perpendicular parking (PC), or angle parking (PB), for each of the plurality of other vehicles (2) at each prescribed time or with each passing prescribed distance of travel ; and a type determining portion (135) for determining, as the parking type (PK) for a region (RN) wherein the plurality of other vehicles (2) is parked, the parking type (PK) of the greatest number of vehicles (2) among the plurality of other vehicles (2), based on the evaluation result by the type evaluating portion (134).

2. A parking assisting device (100) set forth in claim 1, further comprising: a region dividing portion (132) for dividing the captured image (PF) into a plurality of regions (RN) based on a direction of travel (D1) of the host vehicle (1), wherein: the type determining portion (135) determines, as the parking type (PK) for each of the plurality of regions (RN), the parking type (PK) of the greatest number of vehicles (2), for each of the plurality of regions (RN).

3. A parking assisting device (100) set forth in claim 2, wherein: the region dividing portion (132) extracts, from the captured image (PF), a road image (RD) that is an image of the road traveled by the vehicle (1), and divides the captured image into a plurality of regions (RN) based on the extracted road image.

4. A parking assisting device (100) set forth in any one of claim 1 through claim 3, wherein: the type evaluating portion (134) detects the direction (DB) of the vehicle body for each of the plurality of vehicles (2), and evaluates the parking type (PK) based on the direction (DB) of the vehicle body.

5. A parking assisting method, including: an image acquiring step (S101) for acquiring a captured image (PF) around a vehicle (1); a vehicle detecting step (S105) for using the captured image (PF) to detect a plurality of other vehicles (2) that are parked around the vehicle (1); a type evaluating step (S109) for evaluating a parking type (PK), indicating parallel parking, perpendicular parking, or angle parking, for each of the plurality of other vehicles (2) at each prescribed time or with each passing prescribed distance of travel; and a type determining step (S113) for determining, as the parking type (PK) for a region wherein the plurality of other vehicles (2) is parked, the parking type (PK) of the greatest number of vehicles (2) among the plurality of other vehicles (2), based on the evaluation result in the type evaluating step (S109).